-
Notifications
You must be signed in to change notification settings - Fork 815
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix incorrect cache reuse when seeking from end-of-block #1058
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Clang:
I guess this highlights how difficult it is to write warnings that are both useful and not obnoxious. |
In v2.5, we introduced an optimization to avoid rereading data when seeking inside the file cache. Unfortunately this used a slightly wrong condition to check if the cache was "live", which meant seeks from end-of-blocks could end up with invalid caches and wrong data. Not great. The problem is the nuance of when a file's cache is "live": 1. The file is marked as LFS_F_READING or LFS_F_WRITING. But we can't reuse the cache when writing, so we only care about LFS_F_READING. 2. file->off != lfs->cfg->block_size (end-of-block). This is an optimization to avoid eagerly reading blocks we may not actually care about. We weren't checking for the end-of-block case, which meant if you seeked _from_ the end of a block to a seemingly valid location in the file cache, you could end up with an invalid cache. Note that end-of-block may not be powers-of-two due to CTZ skip-list pointers. --- The fix is to check for the end-of-block case in lfs_file_seek. Note this now matches the need-new-block logic in lfs_file_flushedread. This logic change may also make lfs_file_seek call lfs_file_flush more often, but only in cases where lfs_file_flush is a noop. I've also extended the test_seek tests to cover a few more boundary-read cases and prevent a regression in the future. Found by wjl and lrodorigo
geky
force-pushed
the
fix-seek-eob-cache
branch
from
December 19, 2024 08:40
537dc24
to
366100b
Compare
Tests passed ✓, Code: 17136 B (+0.0%), Stack: 1440 B (+0.0%), Structs: 812 B (+0.0%)
|
geky
changed the title
Fixed incorrect cache reuse when seeking from end-of-block
Fix incorrect cache reuse when seeking from end-of-block
Dec 20, 2024
yamt
added a commit
to yamt/incubator-nuttx
that referenced
this pull request
Dec 25, 2024
2.10.1 contains a fix for a long-standing wrong-data issue. littlefs-project/littlefs#1058
acassis
pushed a commit
to apache/nuttx
that referenced
this pull request
Dec 25, 2024
2.10.1 contains a fix for a long-standing wrong-data issue. littlefs-project/littlefs#1058
linguini1
pushed a commit
to CarletonURocketry/nuttx
that referenced
this pull request
Jan 15, 2025
2.10.1 contains a fix for a long-standing wrong-data issue. littlefs-project/littlefs#1058
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In v2.5 (#632 specifically), we introduced an optimization to avoid rereading data when seeking inside the file cache. Unfortunately this used a slightly wrong condition to check if the cache was "live", which meant seeks from end-of-blocks could end up with invalid caches and wrong data. Not great.
The problem is the nuance of when a file's cache is "live":
The file is marked as LFS_F_READING or LFS_F_WRITING.
But we can't reuse the cache when writing, so we only care about LFS_F_READING.
file->off != lfs->cfg->block_size
(end-of-block).This is an optimization to avoid eagerly reading blocks we may not actually care about.
We weren't checking for the end-of-block case, which meant if you seeked from the end of a block to a seemingly valid location in the file cache, you could end up with an invalid cache.
Note that end-of-block may not be powers-of-two due to CTZ skip-list pointers.
The fix is to check for the end-of-block case in
lfs_file_seek
. Note this now matches the need-new-block logic inlfs_file_flushedread
.This logic change may also make
lfs_file_seek
calllfs_file_flush
more often, but only in cases wherelfs_file_flush
is a noop.I've also extended the
test_seek
tests to cover a few more boundary-read cases and prevent a regression in the future.Found by @wjl and @lrodorigo
Related: #1057, #728