[
https://issues.apache.org/jira/browse/ARROW-18313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633805#comment-17633805
]
Miles Granger commented on ARROW-18313:
---------------------------------------
Okay, I suspect the OS isn't allowing it as (probably) there are open files and
when attempting to swap out temp files the system says no. :) From the pyarrow
side and checking {{lsof +D ./}} on the directory, I don't see any open file
handles. But I'll try to come back to this, or reference someone more familiar
with the R API and if file handles are expected to be open there when using
dataset.
> [R] Issues with open_dataset()
> ------------------------------
>
> Key: ARROW-18313
> URL: https://issues.apache.org/jira/browse/ARROW-18313
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
> Reporter: N Gautam Animesh
> Priority: Major
> Attachments: image-2022-11-11-09-19-16-065.png,
> image-2022-11-14-10-51-09-893.png
>
>
> On using open_dataset, it creates a connection due to which the files in the
> directory get blocked and we cannot perform other operations on the file like
> replace!
> Actual issue:
> # We are running an atomic operation on a bunch of files, which replaces the
> temp file names to the target file names.
> # But while this is happening, if we try to run open_dataset() on that
> particular directory, the atomic operation is failing and there are both
> target files and temp files in the directory.
> # It is blocking the files that have been read through open_dataset().
> # Please, provide me with more about how we can handle such problems.
> # Snapshot: !image-2022-11-11-09-19-16-065.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
