yaooqinn opened a new pull request, #12148:
URL: https://github.com/apache/gluten/pull/12148
### What changes were proposed in this pull request?
Drop `ep/build-velox/src/modify_arrow_dataset_scan_option.patch` and the two
shell call sites that apply / stage it:
- `dev/build-arrow.sh` — `patch -p1 <
.../modify_arrow_dataset_scan_option.patch`
- `ep/build-velox/src/get-velox.sh` — `cp` + `git add` that copy the patch
into the Velox source tree
### Why are the changes needed?
The patch added a native CSV / dataset scan-option API
(`CsvFragmentScanOptions::from`, `DeserializeMap`, `mapToExpressionLiteral`,
…)
that was originally consumed by Gluten's native Arrow CSV reader path.
That path is gone:
- #11190 fell CSV back to vanilla Spark
- `0658e906f` / `97f463813` / `9ea8290a` / `d206c5e20` removed the JVM-side
callers, the `ArrowUtil` reader path, and the
`spark.gluten.sql.native.arrow.reader.enabled` config
Nothing inside Gluten or Velox now references the symbols the patch
introduces, so it's dead code in the build. This PR is the final cleanup in
that chain.
### How was this patch tested?
Local verification on Ubuntu 24.04 / x86_64:
- `grep` across the repo — 0 callers of `CsvFragmentScanOptions`,
`CsvFragmentScanOptions::from`, `DeserializeMap`, `mapToExpressionLiteral`,
`createNative(... FragmentScanOptions)`, `CsvConvertOptions`,
`testCsvConvertOptions`
- `nm -C` on freshly built `libarrow_dataset.a` (both `java-dist` and
`cpp-jni` outputs) — none of the patch's symbols are present
- `arrow_ep/cpp/src/arrow/dataset/file_csv.cc` on disk is the unmodified
upstream source — the patch was never applied with this change in place
- `dev/buildbundle-veloxbe.sh --enable_vcpkg=ON` → BUILD SUCCESS for all 5
Spark profiles (3.3 / 3.4 / 3.5 / 4.0 / 4.1)
- `spark-shell` on the resulting bundle, reading a CSV file, prints
`GlutenFallbackReporter: Validation failed for plan: Scan csv , due to:
Unsupported file format TextReadFormat` and produces a vanilla
`FileScan csv` physical plan — confirming CSV is fallback-by-design and
never enters the native path the dropped patch fed.
Generated-by: Claude claude-opus-4.7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
