[GitHub] [arrow] JasperSch opened a new issue #10809: Errors/warnings when saving open_dataset() to variable in R

[GitBox]

JasperSch opened a new issue #10809:
URL: https://github.com/apache/arrow/issues/10809


   Hello,
   
   When saving the result from `open_dataset()` to a variable, I get various 
sporadic warning and error messages throughout the code that is executed 
afterwards. It stops once I remove the variable.
   
   The warnings/errors vary, but here some examples:
   ```
   Error: Invalid <Schema>, external pointer to null
   
   Warning message:
   Number of rows unknown; returning NA 
   
   Error: IOError: Failed to open local file 'xxxx'. Detail: [Windows error 3] 
The system cannot find the path specified.
   
   ```
   
   Tested on following:
   Ubuntu 20.04.2 LTS - R 3.6.3 
   Windows 10 - R 4.1.0.
   
   Using arrow_4.0.1.
   
   MWE:
   ```
   library(arrow)
   library(dplyr)
   
   dir <- tempdir()
   
   df <- mtcars
   
   arrow::write_dataset(df, dir)
   
   # Directly using the object returned by open_dataset gives no messages
   result <- arrow::open_dataset(dir) %>%
       collect()
   1+1
   
   # Leaving dataset open causes messages
   result <- arrow::open_dataset(dir)
   
   1+1
   1+2
   1+3
   
   remove(result)
   # No messages anymore
   
   unlink(dir, recursive = TRUE)
   ```
   
   Thanks in advance for your feedback and for maintaining such a great package.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org