1,995,150 events, 1,021,162 push events, 1,659,609 commit messages, 122,024,398 characters
Created Text For URL [naijanewsagency.com/weve-got-deets-from-the-annual-hvscares-friends-of-africa-charity-dinner-youll-love-it/]
Merge #80
80: Start moving SIR generation to LLVM codegen. r=ltratt a=vext01
Previously I'd tried implementing SIR generation as a new codegen from scratch. What I learned from this is:
-
There's a lot of boilerplate in the codegen API.
-
To get it right, you'd end up duplicating a lot of LLVM backend code, which would need to be kept in sync over time. This code often contains low-level details, which at this point in time I don't understand. The code is also hard to debug if you get it wrong.
-
Pretty much the whole codegen has to be implemented and working all at once. You cannot really incrementally implement parts.
I spent quite some time trying that approach. I generated about 2000 lines of backend code. It's slow going and relentless.
Due to frustration with all of the above, I took pause to see if I could find a better way.
This change shows an idea I had at the end of last week. The general idea is to -- rather than implement a whole new backend -- add calls to SIR generation from within the existing LLVM backend. This means that:
-
We can build our SIR IR incrementally over time.
-
We don't have to understand all of the low-level details right off the bat.
-
There will be no code duplication and when the LLVM backend changes, instead of our code going out of sync we will get merge conflicts (in a good way).
-
The code should be smaller.
This all sounds good, but to make it work, there's one small detail. The LLVM codegen is opaque-pointer-centric and such pointers are often used as identifiers in LLVM API calls. For example you "add a new block to the function behind the following opaque Value pointer". To generate SIR for such a scenario we have to keep a mapping of LLVM pointers to the corresponding SIR. For example so that we know which of the SIR functions to add a block to. This will raise memory consumption a bit, but I hope it isn't all that bad.
To demonstrate the approach, the following diff shows creation of SIR functions and blocks in-memory. Eventually we'd be serialising this someplace for picking up at runtime.
I'm raising a draft PR for comments and discussion before I proceed, as it's a bold deviation from what I've been doing previously.
Thoughts?
Co-authored-by: Edd Barrett [email protected]
"4:25pm. While I was doing chores, my brother nabbed me to help him and I've been doing manual labor for 4 hours or something.
Let me take a short breather and I will finally do some programming.
4:35pm. http://charlespetzold.com/etc/DoesVisualStudioRotTheMind.html
I'll leave this for later.
let test49: SpiralModule =
{
name="test49"
prerequisites=[]
opens=[]
description="Does changing layout type work?"
code=
"""
{a=1; b=2} |> dyn |> stack |> heap |> indiv
"""
}
This test is next so let me do it.
Yeah, it works fine. Next.
4:40pm.
let test52: SpiralModule =
{
name="test52"
prerequisites=[]
description="Does the binary . operator apply if it is directly next to an expression?"
code=
"""
inl main _ =
inl f = function
| .Hello as x -> .Bye
inl g = function
| .Bye -> "Bye"
dyn (g f.Hello)
"""
}
Making headway.
4:45pm. Focus me, what is next. I can get a lot done in the next hour. It is times like these that make me glad I did something in the morning.
let test54: SpiralModule =
{
name="test54"
prerequisites=[]
opens=[]
description="Does the prepass memoization work? Intended to be looked directly without the Core library."
code=
"""
inl f x =
match x with
| q,w,e,r,t,z,x,c,v,b,m -> 0
| (((),a,b) | ({q w e r t y z a b}, _, _)) ->
inl f a b = !Add(a, b)
f a b
| a,b -> !Add(a, b)
!Dynamize (f ({q=(); w=(); e=(); r=(); t=(); y=(); z=(); a=1; b=2},2,3))
"""
}
What is this thing? I have no idea.
let test54: SpiralModule =
{
name="test54"
prerequisites=[]
description="Does the prepass memoization work? Intended to be looked directly without the Core library."
code=
"""
inl main _ =
inl f x =
match x with
| q,w,e,r,t,z,x,c,v,b,m -> 0
| (((),a,b) | ({q w e r t y z a b}, _, _)) ->
inl f a b = !!!!Add(a, b)
f a b
| a,b -> !!!!Add(a, b)
!!!!Dynamize((f ({q=(); w=(); e=(); r=(); t=(); y=(); z=(); a=1; b=2},2,3)))
"""
}
Well, I'll keep this around as a keepsake. I remember this being some kind of performance issue, but I forgot what it was by this point.
5pm.
let test59: SpiralModule =
{
name="test59"
prerequisites=[]
description="Does returning from join points work on nested structures?"
code=
"""
inl main _ =
inl q = {q=1;w=2;e=3}
inl w = {a=q;b=q}
inl e = {z=w;x=w}
inl e = join e
inl e = join e
()
"""
}
This thing works. Everything is working remarkably well so far.
let test61: SpiralModule =
{
name="test61"
prerequisites=[]
description="Does the () record-with pattern work?"
code=
"""
inl main _ =
inl k = .q
inl m = { $k = { b = 2 }}
{(m).(k) with a = 1}
|> dyn
"""
}
Uh oh, this is broken because of the recent unary keyword changes in the tokenizer.
5pm. There is no way around this. I am going to have to change .(k) to something like ... $k.
{m$k with a = 1}
|> dyn
I'll make this work.
(many ((keyword_unary' |>> keyword_unary'') <|> (dollar' >>. (case_var <|> case_rounds expr))))
Something like this then?
((small_var |>> v) <|> rounds expr)
(many ((keyword_unary' |>> keyword_unary'') <|> (dollar' >>. (case_var <|> case_rounds expr))))
This small_var <|> rounds pattern should really be abstracted.
((small_var |>> v) <|> rounds expr)
(many ((keyword_unary' |>> keyword_unary'') <|> (dollar' >>. (case_var <|> case_rounds expr))))
No wait, haven't I decided that big vars should be reserved for union types? I cannot backtrack here.
let case_small_var_rounds expr = ((small_var |>> v) <|> case_rounds expr)
So this is what I am looking for then.
| SpecOpen -> d.Skip; pipe2 (open_ >>. big_var) (opt (curlies (with_ >>. many (let var = small_var <|> rounds op in ((var |>> OpenValue) <|> (grave >>. var |>> OpenType) ) .>>. opt (arr_fun >>. var))))) handle_open d
Ah, crap. Here the big vars should be allowed as modules might want to be renamed.
5:20pm.
let test61: SpiralModule =
{
name="test61"
prerequisites=[]
description="Does the $ record-with pattern work?"
code=
"""
inl main _ =
inl k = .q
inl m = { $k = { b = 2 }}
{m$k with a = 1}
|> dyn
"""
}
Ok, this now works.
Let me move to the next thing.
let test63: SpiralModule =
{
name="test63"
prerequisites=[]
description="Do the keyword arguments get parsed correctly?"
code=
"""
inl main _ =
inl add left:right: = left + right
add left:1 right: 2
+ add
left: 3
right: 7
|> dyn
"""
}
Keyword arguments are a joy to behold.
5:25pm.
let macro_dotnet1: SpiralModule =
{
name="macro_dotnet1"
prerequisites=[macro]
opens=[]
description="Do the macros work?"
code=
"""
Macro
type: ()
global_method: "System.Console.WriteLine"
args: "Hello, world!"
"""
}
It seems I am done with the standard tests. Now come macros. Let me skim from here to the end and then I will start making plans for what comes next tomorrow.
The whole test suite is nearly 1.8k LOC, and I did close to 700. So I breezed through it. Still, the language does not seem to be in too bad of a condition at the moment.
5:30pm. Tomorrow I will definitely do the closure tests along with the closure code. I've yet to test the let statements and that should definitely be done before I move on.
Now before I call it a day let me do a bit more. Let me rename the tests that I have here so they are properly ordered.
5:35pm. Done. It seems I've gone through 40 tests in total so far.
I've decided...I'll delay the macros for later, because I am not sure how they are going to interact with typechecking.
It is also likely that the language syntax will change significantly going forward, so I do not want to over commit myself to much to a particular way of doing things.
In particular the bottom-up segments might be real inl and real let.
5:45pm. Let me close here. While I wait for lunch that virgin vs chad thread is waiting for me."
Make major changes to directory structure
I think its really silly for the client package to be nested within the server package. This structure makes it hard to understand what is going on at a greater level within the application and leads to a false belief that the back-end server is somehow running the front-end server.
I restructured the directories such that the server and client are parallel in the packages directory. I believe this setup allows for a more intuitive understanding of how this project works.
-
Simplified directory structure allows for other people to jump into this project and understand whats going on a little easier.
-
Now you can install all the dependencies for both the client and the server through a single run of
yarnrather than having to doyarnornpm installand thenyarn frontend-installornpm run frontend-install. -
Scripts are significantly simplified in the root
package.jsonbecause most of the more complex parameters are in the server and client'spackage.json -
All of the same scripts still work.
yarn devis still all you need to do to run the dev server -
You have to use yarn, sorry but it would be exceptionally complicated to allow for both to continue to be used here given the third party tools I've setup to help simplify the installation process. Almost all of the same commands work for yarn though and its great, I promise you'll learn to love it 💖
-
The directory structure has changed and its going to take a little bit of getting used to.
-
We're going to have to make some major changes to the
Dockerfilein order to actually be able to deploy this. I'll gladly do this in the near future. I figure we don't have any urgent pushes that need to be made within the next 48 hours so theDockerfilecould probably wait a day or two so that y'all could check out the new structure and start getting used to it.
Directory structure will be very different after this is pulled in. If you hate this new structure I'd love to have a conversation about this. If you both hate it you can kill this PR and I promise I'll only cry about it for a couple days.
FFS, just push the in-progress shit.
I don't love teasing apart random diffs.
Add autoload magicks to the document derivatives
Dear Rails:
Thanks for making prod fail when things work fine in dev.
Thanks for making different environments so very very different that debugging things is always a thrilling adventure.
Love always,
Jeremy
Gosh. Total refactor. Super de-factor
Still need to remove error_logs all over the place but now it actuall allows to change between Scenes, and add dynamically hotspots, swap back and forth and only saves what we need, more and not less. Also works when starting from scratch. Well, looking good and not failing after so many tests...
Pretty tired, will just continue first hour in the morning. This is pretty sweet and i feel kinda the largest dev for realz in this panorama thingy we are into
Created Text For URL [www.iol.co.za/dailynews/news/kwazulu-natal/pathologist-confirms-kzn-soldiers-rifle-killed-his-girldfriends-sister-42256372]