Small files / minimum allocation size issue

[rojer]

i think LFS is awesome, and i would be even more excited if it were less wasteful in case of small files. I realize this will increase complexity, but it just pains me to think i will be wasting 127K for a file of 1K on this chip, which uses 128K blocks.

[davidefer]

yes but on the other side if the code size increases a lot then it is not a "little"fs anymore. Out there there are many commercial file systems with such optimizations that are much bigger and ram hungry.
One should find a good compromise and most important set a maximum code/ram size limit for littlefs (for instance on cortex m4 processor with xxx compiler).
I know @geky is already working on that.
Greetings

[rojer]

i understand there is a tradeoff.
currently the code footprint of lfs seems to be ~10K. i personally wouldn't mind it going up to, say, 16 it it'd cut down on waste for small files, because for me personally it woudl mean i'd be able to drop SPIFFS (~16K code but better small file handling) which i currently have to carry to handle small (32 - 256K) on-device filesystems.
i'll leave it up to @geky to decide whether it's worth it, i'm just providing some data and expressing my wishes here :)

[geky]

Thanks for the interest! This is a good thread and you both have good points.

Actually, inline files (storing small files directly in the directory blocks) was one of the first features I wanted to add after getting the filesystem out there. The disk structures are already set up to support this fairly easily.

Unfortunately one of the internal mechanisms to get inline files working (resizable entrys) turned out to be surprisingly complex and required rewriting most of the implementation. It's mostly working here: #48.

This approach ended up with a code size of ~14K.

Right now, I'm looking into an alternative approach which will hopefully simplify things while introduce several nice features (inline files, custom attributes, metadata logging). If this works out well, I'd rather adopt the new approach than merge #48. The work is all on this branch, but it's not documented, tested, and kind of a mess:
https://github.com/ARMmbed/littlefs/commits/dir-log

Unfortunately, most new features are currently bottlenecked by this work, so that's the main reason progress has been slow on the master branch of littlefs for a while.

[rojer]

@geky this is good news. there is no particular rush but i am definitely looking forward to this happening! as i said, this will actually reduce complexity for my project because i will be able to go LFS-only (eventually).