N Gautam Animesh created ARROW-18313: ----------------------------------------
Summary: Issues with open_dataset() Key: ARROW-18313 URL: https://issues.apache.org/jira/browse/ARROW-18313 Project: Apache Arrow Issue Type: Bug Reporter: N Gautam Animesh Attachments: image-2022-11-11-09-19-16-065.png On using open_dataset, it creates a connection due to which the files in the directory get blocked and we cannot perform other operations on the file like replace! Actual issue: # We are running an atomic operation on a bunch of files, which replaces the temp file names to the target file names. # But while this is happening, if we try to run open_dataset() on that particular directory, the atomic operation is failing and there are both target files and temp files in the directory. # It is blocking the files that have been read through open_dataset(). # Please, provide me with more about how we can handle such problems. # Snapshot: !image-2022-11-11-09-19-16-065.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)