paleolimbot commented on PR #13635: URL: https://github.com/apache/arrow/pull/13635#issuecomment-1192623666
I did some more testing, and the only thing this works for is CSV reading (probably because the CSV reader takes its own `StopToken` and apparently doesn't care about the `IOContext`'s. In fact, I can't find anywhere that the `IOContext`'s `StopToken` actually works (for example, collecting a dataset). I attempted a few combinations of inserting the `StopToken` into some query engine things in compute-exec.cpp, none of which actually resulted in the ability to stop a query plan. We could either merge this because it contains some useful infrastructure and will make follow-up PRs have a smaller scope or wait until there's a clearer path to everything getting cancelled properly (happy with either). Reprex that I'm working with: ``` r library(arrow, warn.conflicts = FALSE) #> Some features are not enabled in this build of Arrow. Run `arrow_info()` for more information. tf <- tempfile() if (!file.exists(tf)) readr::write_csv(vctrs::vec_rep(mtcars, 7e5), tf) # try to slow down CSV reading set_cpu_count(1) set_io_thread_count(2) # turn on the stop source thing enable_cancel_from_interrupt() # Control-C during this works read_csv_arrow(tf, as_data_frame = FALSE) #> Table #> 22400000 rows x 11 columns #> $mpg <double> #> $cyl <int64> #> $disp <double> #> $hp <int64> #> $drat <double> #> $wt <double> #> $qsec <double> #> $vs <int64> #> $am <int64> #> $gear <int64> #> $carb <int64> # Control-C during this doesn't system.time(open_dataset(tf, format = "csv") |> dplyr::collect(as_data_frame = FALSE)) #> user system elapsed #> 3.900 0.259 4.072 ``` <sup>Created on 2022-07-22 by the [reprex package](https://reprex.tidyverse.org) (v2.0.1)</sup> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
