[ https://issues.apache.org/jira/browse/ARROW-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544415#comment-17544415 ]
Antoine Pitrou commented on ARROW-16692: ---------------------------------------- By the backtrace, it looks like a recursion-induced stack blowup. > [C++] Segfault in datasets > -------------------------- > > Key: ARROW-16692 > URL: https://issues.apache.org/jira/browse/ARROW-16692 > Project: Apache Arrow > Issue Type: Bug > Components: C++ > Reporter: Jonathan Keane > Priority: Major > Attachments: backtrace.txt > > > I'm still working to make a minimal reproducer for this, though I can > reliably reproduce it below (though that means needing to download a bunch of > data first...). I've cleaned out much of the unnecessary code (so this query > below is a bit silly, and not what I'm actually trying to do), but haven't > been able to make a constructed dataset that reproduces this. > Working on some example with the new | more cleaned taxi dataset at > {{s3://ursa-labs-taxi-data-v2}}, I've run into a segfault: > {code} > library(arrow) > library(dplyr) > ds <- open_dataset("path/to/new_taxi/") > ds %>% > filter(!is.na(pickup_location_id)) %>% > summarise(n = n()) %>% collect() > {code} > Most of the time ends in a segfault (though I have gotten it to work on > occasion). I've tried with smaller files | constructed datasets and haven't > been able to replicate it yet. One thing that might be important is: > {{pickup_location_id}} is all NAs | nulls in the first 8 years of the data or > so. > I've attached a backtrace in case that's enough to see what's going on here. -- This message was sent by Atlassian Jira (v8.20.7#820007)