viirya commented on PR #59582: URL: https://github.com/apache/spark/pull/59582#issuecomment-6561237694
@HyukjinKwon, following up on your [question about the default for unmarked vectors](https://github.com/apache/spark/pull/59581#pullrequestreview-5565698195) and [my reply](https://github.com/apache/spark/pull/59581#issuecomment-6276911959) on #59581: `ReusedColumnVector` now lives in this PR, which also enables columnar input by default again. As it stands, the Arrow columnar path holds a column only if it is an `ArrowColumnVector` without the marker. Which would you prefer? 1. Keep the marker only, as now. 2. Keep the marker, and also treat a vector whose class overrides `closeIfFreeable` as reused, documenting the hold/copy protocol on the public `ColumnVector.closeIfFreeable`. 3. Drop the marker, and use only the override. Options 2 and 3 change the two checks in `ColumnarArrowEvalPythonEvaluatorFactory` (the first batch and the later ones) and the doc of `ColumnVector.closeIfFreeable`. I'd like to settle this before merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
