42triangles opened a new issue, #714:
URL: https://github.com/apache/arrow-rs-object-store/issues/714
**Describe the bug**
Rapidly creating the same file (as seen in the reproduction section) results
in this error:
```
Generic LocalFileSystem error: Unable to open file test_dir\test\test.bin#2:
Access is denied. (os error 5)
Generic LocalFileSystem error: Unable to open file test_dir\test\test.bin#1:
Access is denied. (os error 5)
Generic LocalFileSystem error: Unable to open file test_dir\test\test.bin#3:
Access is denied. (os error 5)
Generic LocalFileSystem error: Unable to open file test_dir\test\test.bin#5:
Access is denied. (os error 5)
Generic LocalFileSystem error: Unable to open file test_dir\test\test.bin#2:
Access is denied. (os error 5)
[...]
```
**To Reproduce**
```rust
use object_store::local::LocalFileSystem;
use object_store::path::Path;
use object_store::{Error, ObjectStore, PutMode, PutPayload};
#[tokio::main]
async fn main() {
let store = LocalFileSystem::new_with_prefix("./test_dir").unwrap();
for _ in 0..16 {
let store = store.clone();
tokio::spawn(async move {
loop {
match store
.put_opts(
&Path::parse("test/test.bin").unwrap(),
PutPayload::new(),
PutMode::Create.into(),
)
.await
{
Ok(_) | Err(Error::AlreadyExists { .. }) => (),
Err(err) => println!("{err}"),
}
}
});
}
tokio::time::sleep(std::time::Duration::from_secs_f32(1.5)).await;
}
```
with
```toml
[package]
name = "object-store-test"
version = "0.1.0"
edition = "2024"
[dependencies]
object_store = "0.13.2"
tokio = { version = "1.43.0", features = ["bytes", "default", "fs",
"io-std", "io-util", "libc", "macros", "mio", "net", "rt", "rt-multi-thread",
"socket2", "sync", "time", "tokio-macros"] }
```
and creating the `test_dir` directory, then `cargo run`.
The `tokio` features can likely be reduced by a lot, this is just what I
copied from the `Cargo.toml` of the bigger project this occurred in, but I also
doubt they are related to the error.
**Expected behavior**
The only expected error would've been `Error::AlreadyExists`, but it
seemingly often fails before that when creating the staging file (so this is
probably unrelated to the `PutMode` as well and would show up when trying to
recreate the file rapidly, though that was not tested).
**Additional context**
I'm not sure if this may "really" be a bug in Rusts `std` or maybe even
Windows & not `object_store`; based on a little digging I assume the error
comes from `CreateFileW`, which does mention this:
> If you call CreateFile on a file that is pending deletion as a result of a
previous call to
[DeleteFile](https://learn.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-deletefilea),
the function fails. The operating system delays file deletion until all
handles to the file are closed.
[GetLastError](https://learn.microsoft.com/en-us/windows/desktop/api/errhandlingapi/nf-errhandlingapi-getlasterror)
returns ERROR_ACCESS_DENIED.
I'm not sure if trying to rename a file (to the final path) actually counts
as a deletion in Windows though, but at least on
https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew
I didn't find any more info that seems relevant to this exact issue, and I
didn't search further than that either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
