Documenting file name limitations

[Xywzel]

Can't find anywhere in lfs.h, but do the filenames have any limitations other than their size? I'm guessing that because many functions take paths that the filename should not contain '/' and with the names being null-terminated there likely should not be nulls inside them either, but are all other char values acceptable?

[steverpalmer]

I've been looking for the same information. I've not read every document in full, but I did find a reference in the SPEC.md file that says for "0x0xx LFS_TYPE_NAME" that "The data contains the file name stored as an ASCII string (may be expanded to UTF8 in the future)." As @Xywzel suggests, I also guess that a filename cannot contain a '/' or a null, but what about other characters like '\', '*', control characters or whole filenames like ".", "..". It would be great if this detail could be specified somewhere.

[Xywzel]

If I'm reading the lfs_dir_findcorrectly in the lfs.c, then "." and ".." are indeed special cases. There also seems to be root directory for the file system, but it is hard to say if one would open that for reading with path "/" or with empty path.

[geky]

I'll make a note to add documentation on this. littlefs follows Linux/Unix tradition with '/' and NULL being the only illegal characters, and '.' and '..' being the only reserved filenames. "/" is the proper way to reference the root directory. If there's anything I've missed littlefs will try to follow Linux/Unix.

The fact that the empty string is also accepted for the root directory wasn't really intended, checking with Linux it should probably be an LFS_ERR_NOENT. But at this point it shouldn't be changed unless there is a major revision.

Also UTF8 in paths is fine, that line of the documentation should be changed. UTF8's use of the high bit for continuation bytes means that it's not possible to introduce NULL bytes without a true NULL character.

This scheme has the unfortunate downside of requiring base64 or hex encoding to use raw data for filenames, but it's a tradeoff for directories and OS portability.

[steverpalmer]

Thanks @geky,

I've been implementing a virtual file system to sit above littlefs, which is why I've been looking at the issue of paths fairly closely, and in particular the POSIX specification. (I understand that you are not trying to claim full compliance with POSIX, but those guys have had to think closely about some of these things, so I've found them a useful reference). When you consider the documentation, please could you also consider a few corner cases:

• I think I am right in saying that littlefs interprets sequences of "/" characters the same as a single "/" (e.g. line 1339 in lfs_dir_find);
• I think I am right in saying that all paths are absolute, whether prefixed by (possibly multiple) "/" characters or not (afterall, there is no working directory to be relative to :-);
• POSIX makes a special case in §3.271 and elsewhere of paths starting with "//", but I don't think littlefs does anything different in this case;
• POSIX makes some comments about (possibly multiple) trailing "/" characters in §4.13 para 4. My internet searches on trailing "/" characters in paths seems to indicate that some folk think it is a good idea and others who think it is a very bad idea. Any significance to littlefs of trailing "/" character(s) might be worth a sentence in the documentation.

Cheers, Steve

[geky]

I think matching the POSIX specification is a good goal, if anything to avoid surprises. But this is sometimes in conflict with backwards compatibility. I probably won't introduce a breaking change for POSIX compatibility alone for example.

And good comment on these, I overlooked some of the nuances. Goes to show even "simple" specs can hide some unexpectedly complex interactions.

• I think I am right in saying that littlefs interprets sequences of "/" characters the same as a single "/" (e.g. line 1339 in lfs_dir_find);

Yep

• I think I am right in saying that all paths are absolute, whether prefixed by (possibly multiple) "/" characters or not (afterall, there is no working directory to be relative to :-);

Yep, if anything littlefs may add a set of lfs_file_openat/lfs_mkdirat/lfs_statat/etc APIs, leaving this implementation of the cwd up to OSes/virtual layers. But I'm not sure adding the statefulness of the concept of cwd to embedded systems would be very popular...

• POSIX makes a special case in §3.271 and elsewhere of paths starting with "//", but I don't think littlefs does anything different in this case;

If I'm reading this correctly, in §4.13 the spec says '//' is implementation defined. So in a sense, littlefs is POSIX conforming here, by doing the implementation defined thing of treating '//' the same as '/', which I would argue is the least surprising option. :)

Of course this doesn't stop OSes/virtual layers from intercepting '//' and giving it special meaning.

• POSIX makes some comments about (possibly multiple) trailing "/" characters in §4.13 para 4. My internet searches on trailing "/" characters in paths seems to indicate that some folk think it is a good idea and others who think it is a very bad idea. Any significance to littlefs of trailing "/" character(s) might be worth a sentence in the documentation.

There is actually an in-progress PR to improve this: #679.

The intention is to allow trailing '/' for directories only, which I think matches the text in §4.13.

I know this behavior is at least nice for ensuring cp a.txt b/ fails if b does not exist, but that doesn't translate directly to any littlefs API.