there were a lot of events recorded by gharchive.org of which 2,294,323 were push events containing 3,397,945 commit messages that amount to 265,017,382 characters filtered with words.py@e23d022007... to these 19 messages:
Updating: 1/10/2023 11:00:00 PM
- Added: Is it worth encrypting? | Matthew Linkous (https://www.matthewlinkous.com/p/encryption-worthy)
- Added: Makefiles for Web Work – Ross Zurowski (https://rosszurowski.com/log/2022/makefiles)
- Added: Setting up ActiveStorage with Cloudflare R2 (https://kirillplatonov.com/posts/activestorage-cloudflare-r2/)
- Added: I wish JSON5 was more popular (https://evertpot.com/json5/)
- Added: Effective Jugaad: An Ideology for Navigating Complexity and Uncertainty in the 21st Century (https://mostwrong.github.io/2023/01/10/E-Jugaad.html)
- Added: How I went from 0 to Hacking in SF as a high-schooler (https://buildspace.so/notes/zero-to-hacking-in-sf)
- Added: Easy to Overlook Way to Break Eager Loading in Rails Apps (https://pawelurbanek.com/rails-eager-ordered)
- Added: How I hacked Gumroad and broke a bunch of After Effects tools (https://aescreens.com/blog/gumroad-hack)
- Added: Learning Spanish as a Software Developer - Week 1 thoughts (https://www.vincentntang.com/learning-spanish-software-developer/)
- Added: Microservices are a Big Ball of Mud (https://code-held.com/2022/07/28/microservices/)
- Added: Community engagement is so 2022. Here are 10 things you should obsess over instead. (https://rosie.land/posts/dont-obsess-over-community-engagement/)
- Added: Argon ONE NVMe Board Slower than SATA - Martin Rowan (https://www.martinrowan.co.uk/2023/01/argon-one-nvme-board-slower-than-sata/)
- Added: How to build your OCI images using Buildpacks (https://www.adaltas.com/en/2023/01/09/how-to-build-your-oci-images-using-buildpacks/)
- Added: Swimming in the Sydney CBD – Marrinawi Cove - Jake Coppinger (https://jakecoppinger.com/2023/01/swimming-in-the-sydney-cbd-marrinawi-cove/)
- Added: Console applications in C# (https://dev.to/karenpayneoregon/console-applications-in-c-6pg)
- Added: PolySharp/PolySharp.Package.msbuildproj at main · Sergio0694/PolySharp (https://github.com/Sergio0694/PolySharp/blob/main/src/PolySharp.Package/PolySharp.Package.msbuildproj)
- Added: MS Paint IDE (https://ms-paint-i.de/)
Generation took: 00:11:10.0824667
add JamesBrandon validator
Hi, let me introduce myself. My public nickname is JamesBrandon and I have been a node operator and validator since the winter of 2021. I have higher education as a translator (Russian, English, Ukrainian, German) and QA engineer. I never stop learning as you can only get experience. My main advantages are:
- Knowledge of four languages, respectively I can help the community in (almost) four times more effective;
- Proactivity. Since I’ve been dealing with nodes for a long time, I have accumulated a lot of experience in interaction, which I’m always happy to share.
- Speed. Thanks again to my experience, I’ve created (and continue to create) small, highly specialized bots, scripts and so on, which help me respond quickly to both network updates and the usual crash of a node. My uptime is close to 100%, almost never getting slashed because of me.
Discord: JamesBrandon#4096 Element: jamesbrandon3 Twitter: @JBTGrox
List of projects in which I am involved:
Nois: https://explorer.stavr.tech/nois/staking/noisvaloper1f6rp2zgas0000hevcexxndz5xj2cgmtd2r028s
Ollo: https://testnet.manticore.team/ollo/staking/ollovaloper14eft5kc2nwasm2z2vapjje7dp7pnz5vt2u0jvp
Empower: https://testnet.manticore.team/empower/staking/empowervaloper1s2m4kutdrlggtztl8vvxx249u7zdqnh6sufclx
Connext: https://testnet.amarok.connextscan.io/router/0x3ad5BEB9Aa2E894686B92722EeFC9199cf33a658
OKP4: https://okp4.explorers.guru/validator/okp4valoper1x2mnhrgkhwehjt3z0t05m9d65lcjq3pud2gw5f
Zeeka: light node
dYmension: local node
add Sandman
Hi! I want to apply to your project! I’m SandMan, an experienced node operator! I think that your project is worth my, and not only, worth the attention of all node operators. I think it is very promising, I think you need to invest time and effort, I believe it is not in vain. I have a lot of experience participating in testnets, devnets. Experience in solving various bugs, writing quality and detailed feedbacks I am aware of the principles of security and be sure to comply with them, as well as support decentralization, renting servers in very atypical places.
My experience:
Haqq https://haqq.explorers.guru/validator/haqqvaloper1mjtzxuu6959rhkm676nrxv2xc7mnu0k7tsztgl
Connext https://testnet.amarok.connextscan.io/router/0xd567a5B93D0452C213A1262DdB87f66eDe1f7271
Empower https://empower.explorers.guru/validator/empowervaloper13lufaty3wzt4c3yu0juehgc7v0l2587kh376cm
Contact: Email - [email protected] Element - sanddman Discord - sandman#2532 Twitter - notasandyman
Created Text For URL [nypost.com/2023/01/10/im-a-real-life-she-hulk-i-love-my-natural-curvy-girl-muscles/]
ITS IN THE FUCKING REPO FUCKING SHITTY ASS GRADLE FUCK YOU
Editor Object Rehash
- I'm actually coming back to write this after the commit that's after this one, a reflective message I guess
To start this typing journey off, I started with the Editor script, to improve the typing for the app with the base Editor object, which is a big ugly state handling thing, which may need to be poked a little bit, this guy has got to go. It's too big as of now, gonna make things way more modular after I get everything typed out. Everything relies on each other thing in a bunch of weird ways, and that's hard to keep track of. The typings are gonna do wonders to help with that, they already have while doing this even! Some of the original code was just ugly to my older coder eyes, this little reword tidied things up that I've been meaning to do for a while now.
The class-nature of the Editor object is merely for strict typing, it's not for inheritance or anything. Just to initially get things typed out faster, it works nicely to define all of the properties as static members on the class, rather than having to make a huge interface for the object, in JSDoc or a .d.ts file. It's all in once place.
As of now, STE isn't going to be TypeScript quite yet, but I can only imagine moving to it, inevitably. It's so cool what you can do with it, I'm definitely not gonna put it past me for doing that, once the codebase gets to a state to be able to move to it easily (that's not too far away actually).
I. Love. TypeScript. (or, TSC and JSDoc hehe)
Oh yeah, and the weird dynamic object generation setup on the old Editor object was because I didn't know how to make my own getter functions, so that's what I came up with to get "dynamic properties". I'm very happy with how smart I was (for myself back then at least, lol) to being able to figure that out, so it worked very nice as a fill in before finding the more streamlined way of achieving something like that.
[xmonad] Switch the keyboard remapping stuff back
Just had a Chesterton's Fence experience: running the hooks every minute in cron seemed stupid, and I didn't remember why I had done it that way, so I switched to the "better" solution of just running the hooks once at startup. However, the xmodmap changes don't persist across suspend and resume, which was the whole reason for the cron solution in the first place. It's also possible to use hooks in /etc/pm/sleep.d/, but that's much more complex than just using cron, since those scripts run as root and need logic to restrict them to only running on wake from suspend (altho it would be harmless to just run my hacks all the time, including pointlessly when entering suspend or hibernate).
Num Text Legacy Types
Somehow I'm only down to 7 type errors, this is incredible! I really did work all day.
Alright, with this one, I made type definitions for the old version of Num Text, and I added them as a devDependency here, so it will use them right in STE's TSC checks!
Installed it right from GitHub, super cool. I did that back with MCRegionJS and Gamedata-Parser I think, so this is a nice refresher to try that out again.
Listened to Stupid Dream for most of this one, just over 1 time around (1.25 maybe).
Thanks to the new type definitions, it pointed out a few other fixes I could also make, so I did those too. The last 7 errors all have to do with Menu Drop types, so I'm gonna do the same thing as Num Text for the legacy version of that too. That'll be for tomorrow though, I think I'm all brained out for the day.
It was rainy today after work, so I stayed inside and coded pretty much all day. It was a nice recharge on that front. I need some of those days every once in a while, to just chug out all of your ideas and things. I think after this big burst of coding energy, I may have one for music again too, at least that's what I want too. I've found that I work my best when I listen to what I want and don't want to do. If I try and force what I "need" to do, like in terms of my personal goals, then it doesn't hit home like it does when I really mean it. I try to follow those, and I seem to be working really well with that. I think it's important to give yourself time for things, there's so much going on, your mind just might not be set up for that thing yet. Just wait a little while! I like doing that with albums too.
I really have been planning an STE rework like this for more than a year now, I think having TypeScript in my toolbelt made this possible. There were way too many things to handle without it. I think my experience from the past year in general has really helped me step up a few levels. Even when you want to keep working on a cool thing, sometimes it just needs a breather from you. Just let it have it's time, see what happens.
I have buckets in the morning again tomorrow, so I'm making this the last commit for the night. That's it! It's time for that breather I was talking about. If you work on it too long, it starts to turn stale.
Started with Slave Called Shiver, listened all the way to that again, finishing with Baby Dream In Cellophane. I really like This Is No Rehearsal, the whole album for that matter. It also was one that had to grow on me. I think I listened to it first about a year ago or so (not sure when it was), but I didn't pick it up again for a few months/weeks. Now I own it! I'm gonna try out Future Bytes again eventually too, I really liked Personal Shopper.
Ok, gn! I'm so excited for all of the things I'm planning on! Even for buckets in the morning :)
*Edit: That was 7 errors in the main app script, there's 6 more in the Editor script, so down to 13, and they all are only from the Menu Drop types. Noice! Ok, closing down this commit, at 12:43.
Chore: add post-commit hook for Talisman
We still have some issues with our Talisman ckecksum updating process. Every time Talisman detects a potential secret in a file, a human has to very it is not. Then the hash of the current file content has to be configured as secure. As soon as the file gets touched, the hash changes and the checksum in the configuration is obsolete. The next time Talisman reports a secret in this file, it will check the hash, detect that is not the configured one where a human has verified the file last time, and reports an issue.
Talisman checks for secrets in files in two different ways. For commits, it searches only within the diff of the commit for secrets. If there is no secret, everything is fine. If it finds a secret, it checks if the content of the file is marked secure by its hash (which is never the case for a commit which changes the file). It reports the warning and the user has to verify the files, update the configured checksum and update the commit. The second case is for pre-push. Here is checks all files that have been touched within all commits that are pushed. But here is checks the whole file content, not just the diff(s). This means, because any commit that gets pushed has changed the file content, it automatically invalidates the configured checksum. But because Talisman checks the full file again, it also detects the old/already reported potential secrets. But as the hash is no more the same, it reports the old issues again. In such a case, the user simply has to update the checksum. But only under the strong assumption that every potential secret was properly reported and checked during a pre-commit. This mechanism of Talisman is a "more secure" one, because it actually does not rely on a history of always guaranteed security check by every developer.
Unfortunately it is still not possible to verify the whole file as staged in a pre-commit. But it is also annoying to have the pre-push hook failing for this. Especially if more than a single commit gets pushed. Because then it is hard to update the checksums incrementally for every file content hash as staged for each commit. This leads to stupid "talisman-update-commits" and checksum jumps. The used alternative now is to have a post-commit hook. This hooks "abuses" Talisman his pre-push functionality to check full files touched by commits. Just that it now runs only the always just created commit. This detects checksum changes early on commit basis and allows to update them immediately to amend the commit correctly without much effort. As Talisman relies on the default git pre-push hook arguments which are not existing for a post-commit hook, we have to artificially creating them. We kinda did so already in our pre-push hook too due to some Lefthook limitations. Finally, as this is still "experimental", we keep the pre-push hook too. If everything works fine, nothing is worse than before. If not, it saves us and allows us to improve the hooks.
regcomp.c etc - rework branch reset so it works properly
Branch reset was hacked in without much thought about how it might interact with other features. Over time we added named capture and recursive patterns with GOSUB, but I guess because branch reset is somewhat esoteric we didnt notice the accumulating issues related to it.
The main problem was my original hack used a fairly simple device to give multiple OPEN/CLOSE opcodes the same target buffer id. When it was introduced this was fine. When GOSUB was added later however, we overlooked at that this broke a key part of the book-keeping for GOSUB.
A GOSUB regop needs to know where to jump to, and which close paren to stop at. However the structure of the regexp program can change from the time the regop is created. This means we keep track of every OPEN/CLOSE regop we encounter during parsing, and when something is inserted into the middle of the program we make sure to move the offsets we store for the OPEN/CLOSE data. This is essentially keyed and scaled to the number of parens we have seen. When branch reset is used however the number of OPEN/CLOSE regops is more than the number of logical buffers we have seen, and we only move one of the OPEN/CLOSE buffers that is in the branch reset. Which of course breaks things.
Another issues with branch reset is that it creates weird artifacts like this: /(?|(?a)|(?b))(?&a)(?&b)/ where the (?&b) actually maps to the (?a) capture buffer because they both have the same id. Another case is that you cannot check if $+{b} matched and $+{a} did not, because conceptually they were the same buffer under the hood.
These bugs are now fixed. The "aliasing" of capture buffers to each other is now done virtually, and under the hood each capture buffer is distinct. We introduce the concept of a "logical parno" which is the user visible capture buffer id, and keep it distinct from the true capture buffer id. Most of the internal logic uses the "true parno" for its business, so a bunch of problems go away, and we keep maps from logical to physical parnos, and vice versa, along with a map that gives use the "next physical parno with the same logical parno". Thus we can quickly skip through the physical capture buffers to find the one that matched. This means we also have to introduce a logical_total_parens as well, to complement the already existing total_parens. The latter refers to the true number of capture buffers. The former represents the logical number visible to the user.
It is helpful to consider the following table:
Logical: $1 $2 $3 $2 $3 $4 $2 $5 Physical: 1 2 3 4 5 6 7 8 Next: 0 4 5 7 0 0 0 0 Pattern: /(pre)(?|(?a)(?b)|(?c)(?d)(?e)|(?))(post)/
The names are mapped to physical buffers. So $+{b} will show what is in physical buffer 3. But $3 will show whichever of buffer 3 or 5 matched. Similarly @{^CAPTURE} will contain 5 elements, not 8. But %+ will contain all 6 named buffers.
Since the need to map these values is rare, we only store these maps when they are needed and branch reset has been used, when they are NULL it is assumed that physical and logical buffers are identical.
Currently the way this change is implemented will likely break plug in regexp engines because they will be missing the new logical_total_parens field at the very least. Given that the perl internals code is somewhat poorly abstracted from the regexp engine, with parts of the abstraction leaking out, I think this is acceptable. If we want to make plug in regexp engines work properly IMO we need to add some more hooks that they need to implement than we currently do. For instance mg.c does more work than it should. Given there are only a few plug in regexp engines and that it is specialized work, I think this is acceptable. We can work with the authors to refine the API properly later.
Add cryptech as Pre-genesis validator
Good afternoon, I want to introduce you our small team of 6 people.
We are enthusiasts who believe that very soon blockchain technologies will become an integral part of the life of an ordinary person. We are specialists in the field of system administration and have extensive experience in setting up and maintaining decentralized nodes of various networks. Here are some of them: ALEO, Ironfish, Solana-TestNet, Neon-LAbs, Minima, KYVE, Asset Mantle, Game, Cosmic Horizon, KleverChain, TGRADE, Celestia, Archway, QUAI Network, APTOS, Penumbra, STARKNET, SUI, GNOLang, Sei, QuickSilver, OBOL, LAYER ZERO, Web3Auth, DWEB, Oasys, Algoracle, Covalent, Abacus, peaq, Crescent, DeFund, Laconic, Subspace, Gitopia, Massa, Kira, ANOMA, Humanode, ChainFlip, Masa Finance, Manta-Kalamari, GEAR, Supra Oracle, Cere Network, Secret Networt, MoonBeam, Ki-Chain, KOII, Spacemesh, Stargaze, BlockPI and many others.
Our team has many years of experience in network infrastructure, setting up and maintaining client servers based on Linux (Debian, Ubuntu, CentOS, etс.), creting monitoring and warning systems (Prometheus, Zabbix, Nagois, Grafana), writing and compiling Dockerfiles (above 4 years experience), writing scripts and programs in languages (Python, Bash, PowerShell), knows programming languages such as RUST, Go, Javascript, which allows us to fully participate in projects based on CosmosSDK, Ethereum, Polkadot, ZK-snark, work with Bridges, and Blockchain Relays. We carry out full-fledged work with Github and search for Bugs, make various kinds of Feedbacks (bugreports, etc.) For our part, we provide comprehensive assistance and support to development teams - we deploy and maintain nodes of their decentralized networks on our equipment, we conduct functional testing and search for bugs. We also promote projects and create educational content that reveals projects in simple terms from the technical, conceptual and practical sides. We write articles, translate blogs and technical documentation, create video materials in the form of short promos and lectures about projects (mostly in Russian, but we can do it in English). You can see our content and working projects at the links below:
http://cryptech.com.ua/ https://cryptech-nodes.medium.com/ https://www.youtube.com/channel/UCGwQIwKu1QuB9YxW-vElNbQ https://www.twitch.tv/projectcryptech (The last broadcast was quite a while ago because our speaker was ill, but we will be back to this activity soon)
We do not rent cloud servers, we purchased, configured and maintain all the capacities on our own. We have created our own data center and work with the following equipment: HPE Proliant DL580 Gen9 Server Quad 24-Core E7-8894 v4 **96 Cores 512GB RAM, 8 x Intel P3520 Series 2 TB SSD 4 x HP GEN 9, CPU - 2 x Intel Xeon E-4667 v3, RAM - DDR-4 368 GB, SSD - 4 x Intel P3520 Series 2 TB 4 x Quanta - 2 x Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30GHz, RAM - DDR-4 368 GB, SSD - 4 x Intel P3520 Series 2 TB 2 x Gigabyte 1U - 2 x Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30GHz, RAM - DDR-4 256 GB, SSD - 4 x Intel P3520 Series 2 TB 2 x SuperMicro with SGX processors Intel Xeon E2278G, RAM - DDR-4 128 GB, 3 x Intel P3520 Series 2 TB We get the Internet from three backbone providers in our country: Eurotranstelecom, Vega Telecom, DataLine. We have our own autonomous system and communication channels with a speed of 1 Gigabit / s each and a backup power supply system, which allows our data center to work without interruptions 24/7 with 100% uptime.
For the last 4 years we have been working in the blockchain industry and participating in a large number of full-time projects. Before that, we were involved in various projects and various branches of the IT industry. BUT gradually we became interested in blockchains, and our hobby became our main occupation. Our contacts: https://twitter.com/CryptechNodes https://github.com/dvjromashkin [email protected]
Update base for Update on "POC: Allow guarding on unbacked symints"
This PR demonstrates a simple (perhaps too simple) approach to dealing with guards on unbacked symints. We maintain a list of substitutions of expressions with unbacked symints, and then when we fail to statically evaluate an expression because it contains unbacked symints, we attempt to apply those substitutions to see if they can eliminate the unbacked symints. The test case manually programs the substitutions in the ShapeEnv, but in general we need some more user friendly API for doing so.
I am not sure if I should actually finish preparing this patch, however, because as you can see the user experience for what substitutions you need to add is very, very bad. It turns out, we do a LOT of 0/1 specializing.
Signed-off-by: Edward Z. Yang <ezyangfb.com>
[ghstack-poisoned]
"https://softwarecrisis.dev/letters/tech-is-a-pop-culture/
https://www.baldurbjarnason.com/2022/theory-building/
The building of the program is the same as the building of the theory of it by and in the team of programmers. During the program life a programmer team possessing its theory remains in active control of the program, and in particular retains control over all modifications. The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered. Revival of a program is the rebuilding of its theory by a new programmer team.
6:55pm. https://www.youtube.com/watch?v=GIGvHMtNlis&list=PLQHmK8j6zHB5AHkzSh-3tFhkUTtaa5mBS
This OST is not like the other ones.
https://boards.4channel.org/g/thread/90815751#p90816625
This is a really great ship.
I need to try CG unity at some point.
9:05pm. Ok, since I hadn't gotten a reply from Tenstorrent it is certain that I've fallen into the usual ghosting pattern.
What I am going to do tomorrow is pare down the article just to the main bullet points and post just some code. Then I will shove it into the Tenstorrent Discord. Then I'll forget about this path.
I need to go where the market is. For that reason I should at least write the City Of Dawn chapter to completion. After that I'll look whether there is any hope for me in the embedded space. It is not like I intend to stop writing it if I get a good offer. I'll do it over the weekends.
https://en.wikipedia.org/wiki/AoS_and_SoA
I should link to this article.
I am going to do it. I'll condence the much longer article into a series of bullet points. Then I will post it in the Tenstorrent discord. After that I might want to try posting it on HN. In truth, I am running out of patience, and I am curious what the reception would be.
1/11/2023
9:55am. Fired an email to the Tenstorrent customer success guy.
Now let me chill for a while. Kakeru and Mimicry Girls are out.
10:10am. After I deal with this I will condense the article I've made.
10:45am. Let me get started. First the article. I need to touch it up. While I did need to write the current one for the sake of the backend demo, nobody is going to read it.
https://www.youtube.com/watch?v=lRvsk8_MWt0&list=PLQHmK8j6zHB5AHkzSh-3tFhkUTtaa5mBS&index=27 26 NightMare
This is banger.
https://github.com/mrakgr/PIM-Programming-In-Spiral-UPMEM-Demo
Ok, here is the new article. Let me post it in the Tenstorrent Discord.
Oh it has a dev cloud. I registered for it a while ago, but didn't get a confirmation.
11:45am. What the hell is going on with my Discord account? I am trying to log in, but it says my email is already registered.
https://support.discord.com/hc/en-us/articles/360043677651--Email-is-Already-Registered-Errors
Therefore, if you ever see this “Email is Already Registered” error when claiming an email address, that means that there'a already another Discord account that is registered to this email address.
I must have logged in incorrectly.
///
https://github.com/mrakgr/PIM-Programming-In-Spiral-UPMEM-Demo
I did a Python + C backend for UPMEM devices as a demo of Spiral's capabilities. I'd like to do a similar thing for Tenstorrent, but it is difficult to even get the user guide for your devices, let alone a simulator. I registered for your cloud offering months ago, but still hadn't gotten an invite. If you'd like to speed up the process, don't hesitate to give me a hand.
///
Posted the above in the Tenstorrent Discord.
With that I am done. Not much I can do unless they come back to me. This is the kind of article that is suitable for posting on HN. That longer one would just get ignored.
12pm. With that out of the way...
I'll wait a few days before doing it. I might as well post in on Saturday and see what HN gets me. If this results in some bites, it could give me some good work.
Embedded, AI chips, whatever. Bring it.
12:10pm. Right now, let me go for breakfast and chores. After that I'll write some more of Heaven's Key.
2pm. Caught up with demon sword maiden. Let me do the chores. Then I will write.
3:15pm. Done with chores. Got an email from Juan Gomez Luna of the PIM course.
///
Hi Marko,
Thanks for your email.
I guess you have used the simulator in the UPMEM SDK to develop your Spiral backend?
Have you performed any comparison to hand-tuned code? Would Spiral code have similar performance? I would also be curious to know if Spiral has any limitations. For example, would you be able to implement all existing PrIM benchmarks?
///
Let me take a moment to reply. This is what I live for.
///
I guess you have used the simulator in the UPMEM SDK to develop your Spiral backend?
Yes.
Have you performed any comparison to hand-tuned code? Would Spiral code have similar performance?
I haven't performed any comparison for the UPMEM devices, nor have I done extensive benchmarks. But my performance claims for Spiral come from its design philosophy. Let me explain.
You can take a statically typed high level language like F# for example. And you can write it in C style. By that I mean, restrict yourself from using features like generics, closures or heap allocated tuples. At that point, because you are writing code that does not use any abstractions that have a runtime cost, you'd approach the general performance of C minus the platform differences.
In Spiral's case, it is really easy to verify that its abstractions do not have a runtime cost. It provides guarantees of how tuples, records, functions and the like are compiled. I absolutely needed this because originally the language started as a codegen for Cuda kernels when I was working on a ML library in F#. In Cuda code how could one allocate functions and tuples on the heap for example? It cannot be done, so I needed a language that gives guarantees regarding how it does inlining.
In most functional languages, when you pass in a function as an argument, they'd get passed in as a runtime closure which would have a performance impact as a result. In Spiral, you can specialize the join point for that particular instance and inline it instead. These kinds of guarantees are really great if you are writing monadic code; it was really useful when I was doing auto-differentiated Cuda kernels back in 2018.
I do not have anything on hand, but I have benchmarked almost identical F# vs Spiral code, and found Spiral to be much faster, purely because of the control it offers over inlining. I think that the old Spiral from 2018 might have some benchmarks on parser combinators. But otherwise, why Spiral is performant is not some kind of magic. It really comes into play when you are writing abstract code. The other functional languages simply do not provide the control needed to make their abstractions free, that is where its edge comes from.
I would also be curious to know if Spiral has any limitations.
It has much less than existing functional languages. It would be very hard to implement an efficient inverse array datatype in any that I can think of other than Spiral. There are some coding styles that the language is not suited for, for example in C and even C#, you can write code that makes heavy use of gotos. But if you really need such a style it would be smarter to write the functions in languages than support them and call out to them from Spiral. Its macro approach to interop is quite similar to how ReasonML makes use of existing Javascript, so it is quite flexible.
For example, would you be able to implement all existing PrIM benchmarks?
Definitely. Of course, compared to the map kernel demo, it would be necessary to modify the run function so it makes use of more than just a single DPU. And it would take effort to rewrite the benchmarks so that they are generic library functions, but it would be doable if one has the time.
Spiral's weaknesses aren't really regarding the kinds of code you can write in it.
I think its main weakness is that it is less scalable than more dynamic languages like the .NET and the JVM ones. This is inherent to its design, it simply has to do more at compile time than Java and C# do. If Spiral ever becomes popular enough that people are compiling 100k LOC and larger codebases in it, I am going to have to sit down and put in some work to make the partial evaluation incremental and concurrent, but I'll cross my bridge when it comes to that. I put in a lot of work to prepare for that.
Compared to the old 2018 version, the 2021 v2 version of Spiral is already a lot faster thanks to the prepass. Right now I am not sure what its limits are. With v2 I've only used the Cython backend seriously with it and what happened was that the Cython code took the vast majority of time to compile rather than Spiral.
///
4pm. This is good. I put it through docs.
4:15pm. It was a while since I wrote a rant like that. Maybe there is some interest for Spiral from academia? Who knows, maybe something will come from this direction. I didn't think I'd get a reply to the last email. It shows you never know what might happen in life.
Right...let me write Heaven's Key for a bit. I had a heads up start, but I've somehow spent the whole day inside my head instead.
But it is such a drag to write HK like it is my job. I admit I am less motivated since it became obvious the writer's route won't work. Right now I am seriously thinking how to branch out and actually make an income from my existing work. I am in a tough spot since I do not just want to dump Spiral for some rando job. I want to link my paid work to my past unpaid work if possible.
I need to make it clear to myself what I am doing for the sake of ego and what I am doing for the sake of money. I can't do both. Ego takes priority, and money will have to take its place only if I cannot met the goals for my main one.
5:20pm. 51.65k. I wrote about 0.7k.
5:45pm. Done with lunch. Let me see if I can write more.
6:50pm. 51.9k. Not particularly. Today I've been mostly inside my own head. I just didn't feel like writing much.
Oh, got a reply from Tenstorrent. I won't paste the email here, as it is more of a private sort. It says the expected timeline is 6 months before they open source their SDK and enable others to program their chips. Right now they are focused on deep learning functions. I feel a bit guilty for leading them on since he offered to book a meeting to explain their SW and HW and even offered to help implement 'our' custom kernels.
I said I'll wait until their software stack is in a state they can release to consumers before trying it out, and thanked the guy for taking the time to reply.
It seems I've been too negative. I got a sensible reply from Tenstorrent and the Safari group. With Tenstorrent I have no choice but to wait a couple of months at least.
7:10pm. There is no helping it. 1k words today will have to be enough. Maybe tomorrow I will manage to find some strength to write more.
Today I shortened the PIM article, but now I am worried that my sales pitch in the intro is too strong. I'll leave it as is for now, and do one last rewrite before to posting it to HackerNews after I am done with the current chapter of Heaven's Key.
Let me commit this here.
I know I should be writing, but I can't help by be concerned for the future and it is distracting me. I know that I can't maintain my current existence forever. If there is a problem I have to solve, I'd rather do it now that later. So I am refining my mindset. Right now my heart is wavering, and when it becomes firm, I will step forward and charge."
LFO pitch is broken. Adding tests to debug
Instead of a smooth sliding pitch change, LFO pitch is warbling. My theory is that I'm using a linear scale when I should be using logarithmic, or vice-versa. Welsh Angels sounds silly.
fix blocklight not working with advanced light propagation and no pixel perfect shadows
fuck you codium, I don't care if my commit messages are longer than 50 characters
Now my life has gained its meaning since those sinful eyes behold.
Sacred land with meadows greening whose renown was often told. This was granted me from God, to see the land and the holy sod, which in human form he trod.
Splendid lands of wealth and power, I've seen many far and near. Yet of all are you the flower, what a wonder happened here! That, a maid a child should bear, Lord of all the angels fair, was not this a wonder rare?
Here he was baptised the holy, that all people might be pure. Here he died betrayed and lowly, that our bonds should not endure. Else, our fate had been severe, hail oh cross oh thorns and spear! Heathens woe your rage is clear! https://www.youtube.com/watch?v=EXv8-hRB8j4
[HACK]: base: power: wakeup: create a dummy debugfs entry for trace_marker
ah shit you finally disabled debugfs only to see userspace scream at you for not having trace_marker this is the only driver which creates a debugfs entry which is essential for battery monitoring (see 1bdb13584fb7b5c6b7b741e4436a4dc4397df26e) adjust it's init function to create said dummy trace file inside tracing dir this will suppress the silly userspace errors
lib/sort: make swap functions more generic
Patch series "lib/sort & lib/list_sort: faster and smaller", v2.
Because CONFIG_RETPOLINE has made indirect calls much more expensive, I thought I'd try to reduce the number made by the library sort functions.
The first three patches apply to lib/sort.c.
Patch #1 is a simple optimization. The built-in swap has special cases for aligned 4- and 8-byte objects. But those are almost never used; most calls to sort() work on larger structures, which fall back to the byte-at-a-time loop. This generalizes them to aligned multiples of 4 and 8 bytes. (If nothing else, it saves an awful lot of energy by not thrashing the store buffers as much.)
Patch #2 grabs a juicy piece of low-hanging fruit. I agree that nice simple solid heapsort is preferable to more complex algorithms (sorry, Andrey), but it's possible to implement heapsort with far fewer comparisons (50% asymptotically, 25-40% reduction for realistic sizes) than the way it's been done up to now. And with some care, the code ends up smaller, as well. This is the "big win" patch.
Patch #3 adds the same sort of indirect call bypass that has been added to the net code of late. The great majority of the callers use the builtin swap functions, so replace the indirect call to sort_func with a (highly preditable) series of if() statements. Rather surprisingly, this decreased code size, as the swap functions were inlined and their prologue & epilogue code eliminated.
lib/list_sort.c is a bit trickier, as merge sort is already close to optimal, and we don't want to introduce triumphs of theory over practicality like the Ford-Johnson merge-insertion sort.
Patch #4, without changing the algorithm, chops 32% off the code size and removes the part[MAX_LIST_LENGTH+1] pointer array (and the corresponding upper limit on efficiently sortable input size).
Patch #5 improves the algorithm. The previous code is already optimal for power-of-two (or slightly smaller) size inputs, but when the input size is just over a power of 2, there's a very unbalanced final merge.
There are, in the literature, several algorithms which solve this, but they all depend on the "breadth-first" merge order which was replaced by commit 835cc0c8477f with a more cache-friendly "depth-first" order. Some hard thinking came up with a depth-first algorithm which defers merges as little as possible while avoiding bad merges. This saves 0.2*n compares, averaged over all sizes.
The code size increase is minimal (64 bytes on x86-64, reducing the net savings to 26%), but the comments expanded significantly to document the clever algorithm.
TESTING NOTES: I have some ugly user-space benchmarking code which I used for testing before moving this code into the kernel. Shout if you want a copy.
I'm running this code right now, with CONFIG_TEST_SORT and CONFIG_TEST_LIST_SORT, but I confess I haven't rebooted since the last round of minor edits to quell checkpatch. I figure there will be at least one round of comments and final testing.
This patch (of 5):
Rather than having special-case swap functions for 4- and 8-byte objects, special-case aligned multiples of 4 or 8 bytes. This speeds up most users of sort() by avoiding fallback to the byte copy loop.
Despite what ca96ab859ab4 ("lib/sort: Add 64 bit swap function") claims, very few users of sort() sort pointers (or pointer-sized objects); most sort structures containing at least two words. (E.g. drivers/acpi/fan.c:acpi_fan_get_fps() sorts an array of 40-byte struct acpi_fan_fps.)
The functions also got renamed to reflect the fact that they support multiple words. In the great tradition of bikeshedding, the names were by far the most contentious issue during review of this patch series.
x86-64 code size 872 -> 886 bytes (+14)
With feedback from Andy Shevchenko, Rasmus Villemoes and Geert Uytterhoeven.
Link: http://lkml.kernel.org/r/f24f932df3a7fa1973c1084154f1cea596bcf341.1552704200.git.lkml@sdf.org Signed-off-by: George Spelvin [email protected] Acked-by: Andrey Abramov [email protected] Acked-by: Rasmus Villemoes [email protected] Reviewed-by: Andy Shevchenko [email protected] Cc: Rasmus Villemoes [email protected] Cc: Geert Uytterhoeven [email protected] Cc: Daniel Wagner [email protected] Cc: Don Mullis [email protected] Cc: Dave Chinner [email protected] Signed-off-by: Andrew Morton [email protected] Signed-off-by: Linus Torvalds [email protected]
Merge: Performance improvements.
This patchset brings some performance improvements and the addition of the LZO-RLE algorithm to the kernel, also usable in zram (yup, tested, works but LZ4 is still ok for us).
The main performance improvement is for SWAP space: the locking has changed and the swap cache is now split in 64MB trunks. This gives us a reduction of the median page fault latency of 375%, from 15uS to 4uS, and an improvement of 192% on the swap throughput (this includes "virtual" swap devices, like zRAM!). The real world user experience improvement of this on a mobile device is seen after a day or two of usage, where it usually starts losing just a little performance due to the large amount of apps kept open in background: now I cannot notice any more performance loss and the user experience is now basically the same as if the phone was in its first 2 hours of boot life.
Other performance improvements include, in short:
UDP v4/v6: 10% more performance on single RX queue
Userspace applications will be faster when checking running time of threads
2-5% improvements on heavy multipliers (yeah, not a lot, but was totally free...)
Improvements on rare conditions during sparsetruncate of about 0.3% to a
way more rare around 20% improvement (that's never gonna happen, but there
is no performance drop anywhere).
Tested on SoMC Tama Akatsuki RoW
This was taken from Repo: https://github.com/sonyxperiadev/kernel PR: 2039 ([2.3.2.r1.4] Performance improvements)
Signed-off-by: TogoFire [email protected]