[ https://issues.apache.org/jira/browse/ARROW-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17256715#comment-17256715 ]
Neal Richardson commented on ARROW-11067: ----------------------------------------- The print method for Table doesn't print the values. {{tab$json_string}} just extracts the corresponding ChunkedArray, and its print method does print values (which show nulls on my machine). > [R] read_csv_arrow silently fails to read some strings and returns nulls > ------------------------------------------------------------------------ > > Key: ARROW-11067 > URL: https://issues.apache.org/jira/browse/ARROW-11067 > Project: Apache Arrow > Issue Type: Bug > Components: R > Reporter: John Sheffield > Priority: Major > Fix For: 3.0.0 > > Attachments: arrow_explanation.png, arrow_failure_cases.csv, > arrow_failure_cases.csv, arrowbug1.png, arrowbug1.png, demo_data.csv > > > A sample file is attached, showing 10 rows each of strings with consistent > failures (false_na = TRUE) and consistent successes (false_na = FALSE). The > strings are in the column `json_string` – if relevant, they are geojsons with > min nchar of 33,229 and max nchar of 202,515. > When I read this sample file with other R CSV readers (readr and data.table > shown), the files are imported correctly and there are no NAs in the > json_string column. > When I read with arrow::read_csv_arrow, 50% of the sample json_string column > end up as NAs. as_data_frame TRUE or FALSE does not change the behavior, so > this might not be limited to the R interface, but I can't help debug much > further upstream. > > > {code:java} > aaa1 <- arrow::read_csv_arrow("demo_data.csv", as_data_frame = TRUE) > aaa2 <- arrow::read_csv_arrow("demo_data.csv", as_data_frame = FALSE) > bbb <- data.table::fread("demo_data.csv") > ccc <- readr::read_csv("demo_data.csv") > mean(is.na(aaa1$json_string)) # 0.5 > mean(is.na(aaa2$column(1))) # Scalar 0.5 > mean(is.na(bbb$json_string)) # 0 > mean(is.na(ccc$json_string)) # 0{code} > > > * arrow 2.0 (latest CRAN) > * readr 1.4.0 > * data.table 1.13.2 > * R version 4.0.1 (2020-06-06) > * MacOS Catalina 10.15.7 / x86_64-apple-darwin17.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)