Unwrap on None and JoinHandle error if tokio::fs::File read operations reaches a certain concurrency rate

[milen-denev]

Version
1.42.0

test_db_web v0.1.0 (Q:\Projects\Rust\rasterizeddb\test_db_web)
├── async-lazy v0.1.0
│ └── tokio v1.42.0 ()
├── axum v0.7.9
│ ├── async-trait v0.1.83 (proc-macro)
│ │ ├── proc-macro2 v1.0.92 ()
│ │ ├── quote v1.0.38 ()
│ │ └── syn v2.0.92 ()
│ ├── axum-core v0.4.5
│ │ ├── async-trait v0.1.83 (proc-macro) ()
│ │ ├── bytes v1.9.0
│ │ ├── futures-util v0.3.31 ()
│ │ ├── http v1.2.0
│ │ │ ├── bytes v1.9.0
│ │ │ ├── fnv v1.0.7
│ │ │ └── itoa v1.0.14
│ │ ├── http-body v1.0.1
│ │ │ ├── bytes v1.9.0
│ │ │ └── http v1.2.0 ()
│ │ ├── http-body-util v0.1.2
│ │ │ ├── bytes v1.9.0
│ │ │ ├── futures-util v0.3.31 ()
│ │ │ ├── http v1.2.0 ()
│ │ │ ├── http-body v1.0.1 ()
│ │ │ └── pin-project-lite v0.2.15
│ │ ├── mime v0.3.17
│ │ ├── pin-project-lite v0.2.15
│ │ ├── rustversion v1.0.19 (proc-macro)
│ │ ├── sync_wrapper v1.0.2
│ │ ├── tower-layer v0.3.3
│ │ ├── tower-service v0.3.3
│ │ └── tracing v0.1.41
│ │ ├── log v0.4.22
│ │ ├── pin-project-lite v0.2.15
│ │ └── tracing-core v0.1.33
│ │ └── once_cell v1.20.2
│ ├── bytes v1.9.0
│ ├── futures-util v0.3.31 ()
│ ├── http v1.2.0 ()
│ ├── http-body v1.0.1 ()
│ ├── http-body-util v0.1.2 ()
│ ├── hyper v1.5.2
│ │ ├── bytes v1.9.0
│ │ ├── futures-channel v0.3.31 ()
│ │ ├── futures-util v0.3.31 ()
│ │ ├── http v1.2.0 ()
│ │ ├── http-body v1.0.1 ()
│ │ ├── httparse v1.9.5
│ │ ├── httpdate v1.0.3
│ │ ├── itoa v1.0.14
│ │ ├── pin-project-lite v0.2.15
│ │ ├── smallvec v1.13.2
│ │ └── tokio v1.42.0 ()
│ ├── hyper-util v0.1.10
│ │ ├── bytes v1.9.0
│ │ ├── futures-util v0.3.31 ()
│ │ ├── http v1.2.0 ()
│ │ ├── http-body v1.0.1 ()
│ │ ├── hyper v1.5.2 ()
│ │ ├── pin-project-lite v0.2.15
│ │ ├── tokio v1.42.0 ()
│ │ └── tower-service v0.3.3
│ ├── itoa v1.0.14
│ ├── matchit v0.7.3
│ ├── memchr v2.7.4
│ ├── mime v0.3.17
│ ├── percent-encoding v2.3.1
│ ├── pin-project-lite v0.2.15
│ ├── rustversion v1.0.19 (proc-macro)
│ ├── serde v1.0.217 ()
│ ├── serde_json v1.0.134 ()
│ ├── serde_path_to_error v0.1.16
│ │ ├── itoa v1.0.14
│ │ └── serde v1.0.217 ()
│ ├── serde_urlencoded v0.7.1
│ │ ├── form_urlencoded v1.2.1
│ │ │ └── percent-encoding v2.3.1
│ │ ├── itoa v1.0.14
│ │ ├── ryu v1.0.18
│ │ └── serde v1.0.217 ()
│ ├── sync_wrapper v1.0.2
│ ├── tokio v1.42.0 ()
│ ├── tower v0.5.2
│ │ ├── futures-core v0.3.31
│ │ ├── futures-util v0.3.31 ()
│ │ ├── pin-project-lite v0.2.15
│ │ ├── sync_wrapper v1.0.2
│ │ ├── tokio v1.42.0 ()
│ │ ├── tower-layer v0.3.3
│ │ ├── tower-service v0.3.3
│ │ └── tracing v0.1.41 ()
│ ├── tower-layer v0.3.3
│ ├── tower-service v0.3.3
│ └── tracing v0.1.41 ()
├── once_cell v1.20.2
├── rand v0.8.5
│ ├── rand_chacha v0.3.1
│ │ ├── ppv-lite86 v0.2.20
│ │ │ └── zerocopy v0.7.35 ()
│ │ └── rand_core v0.6.4
│ │ └── getrandom v0.2.15 ()
│ └── rand_core v0.6.4 ()
├── rasterizeddb_core v0.0.7 (Q:\Projects\Rust\rasterizeddb\rasterizeddb_core) ()
├── tokio v1.42.0 ()
└── tower v0.5.2 (*)

Platform
Windows 11 64-bit 24H2 26100.2605

Description

I am getting the errors shown on the screenshot.

This happens then:

I have placed a database table (my own, global static) behind an Axum server, and I am bombarding it with requests, that in the end are reading a file by using tokio::fs::File.

Repo: https://github.com/milen-denev/rasterizeddb/blob/0.0.7/test_db_web/src/main.rs

Execution flow:

async fn query() -> async fn first_or_default_by_query() -> async fn read_chunk_sync() -> async fn read_data() which is the final function located in file_sync.rs:

async fn read_data(&mut self,
        position: &mut u64,  
        length: u32) -> Vec<u8> {
        self.read_file.seek(SeekFrom::Start(*position)).await.unwrap();
        let mut buffer: Vec<u8> = vec![0; length as usize];
        let read_result = self.read_file.read_exact(&mut buffer).await;
        if read_result.is_err() {
            return Vec::default()
        } else {
            *position += length as u64;
            return buffer;
        }
    }

The self.read_file is initialized by this:

tokio::fs::File::options().read(true).open(&file_str).await.unwrap()

The concurrency at which I was able to begin having issues is 1000 / requests per second, (1000 seek() and read_exact() per second).

There are 2 errors, 1 unwrap on None value and another one with JoinHandle related error.

Might be related to: #6424

[paolobarbolini]

Do you get the same error after removing the unsafe code in test_db_web/src/main.rs? If I'm reading it correctly it's all unsound, because you are creating multiple mutable references to the same variables, which Rust never allows (try running it in miri and it'll point that out - also explained by the static_mut_refs lint in your case). This could create a race condition within tokio (for example this area being executed concurrently) that sound code would never be able to reproduce, leading to the panics you've experienced.

[milen-denev]

I can confirm that by adding a Arc<RwLock> actually fixes the issue. Thanks.