Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization & cleanup #54

Merged
merged 4 commits into from
Sep 15, 2024
Merged

Optimization & cleanup #54

merged 4 commits into from
Sep 15, 2024

Conversation

Lyfts
Copy link
Member

@Lyfts Lyfts commented Sep 15, 2024

Optimization

To preface this, the only times I've ever hit OOM in GTNH was from a nasty memory leak and whenever VP to recache.
Enklume is a great tool but whenever you load a region file with it, it decompresses and load every single chunk from that region into memory.
It ate GB's worth of ram every second and is really not ideal since VP loads every single region file in multiple threads, while only ever reading a fraction of the chunks contained within. Enklume's NBT re-implementation was also incredibly ram heavy and accounted for about 15% of the memory used whenever I ran OOM.

Enklume has been completely ditched and replaced with a slimmed down region file reader that decompresses and loads the chunks on demand.

This combined with a bunch of micro-optimizations meant a reduction in the time it took to re-cache of over 65% and about 1/3 of the ram usage. (It's still pretty ram hungry though)

There's also a bunch of smaller optimizations that aren't involved in the re-caching, but their impact is harder to measure.

Cache file change

VP's cache files have been converted from raw bytes to NBT. This is solely because it is significantly easier to work with, especially when you need to change how some things are stored. (which we may need to do whenever TE ores are gone)
This uses about 15% more storage space than before but in the majority cases we are still talking KB's worth and MB's in the worst cases.

The files are all auto-converted and no data will be lost.

Cleanup & bugfix

The localized names that we can grab from both BW & GT materials work fine on both server and client, so the old server translation system has been completely gutted. There's no point in keeping a "manual" system which will just cause problems if ore/fluid names are ever changed.

And finally the weird bug where BW ores showed that they contain random GT materials has been fixed

Before
bw_ores_pre

After
bw_ores

@Lyfts Lyfts requested a review from a team September 15, 2024 16:13
@Caedis
Copy link
Member

Caedis commented Sep 15, 2024

Dont see any issues with the implementation. Good job

@Dream-Master Dream-Master merged commit e927d4c into master Sep 15, 2024
1 check passed
@Dream-Master Dream-Master deleted the optimization branch September 15, 2024 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants