sunchao commented on code in PR #3726:
URL: https://github.com/apache/parquet-java/pull/3726#discussion_r4318390344
##########
parquet-hadoop/README.md:
##########
@@ -558,6 +558,26 @@ if not found then the library will use the classic
non-vectored reads: it is saf
---
+**JVM system property:** `parquet.hadoop.vectored.io.threads`
+
+**Description:** Maximum number of shared vectored-read workers. Set a
positive integer
Review Comment:
Agreed — added this in effbc38496207ddcb4c9a0b3204c4ff8c9d1e023.
Callers can now wrap an application-owned pool with a shared admission limit:
```java
VectoredReadExecutor reads = new VectoredReadExecutor(engineIoPool, 8);
ParquetReadOptions options = ParquetReadOptions.builder()
.withVectoredReadExecutor(reads)
.build();
```
The option is also available through `HadoopReadOptions` and
`ParquetReader.Builder`; copied options retain the same wrapper. Leaving it
unset keeps the existing JVM shared pool as the default. Parquet never shuts
down the supplied pool, and accepted operations retain capacity until
submission, the ownership decision, and failure cleanup finish.
The supplied executor must dispatch asynchronously, accept or reject rather
than silently drop work, and drain accepted tasks during shutdown. Inline
execution (including saturated `CallerRunsPolicy`) is rejected before
filesystem submission, leaving the reader usable for retry.
Added regressions for both options builders and copies, reader closure and
error cleanup without pool shutdown, inline rejection, permit recovery, and
retrying the same reader. Validation passed: 877 module tests (26
Hadoop-capability skips), 17 integration tests, and 118 focused Hadoop 3.4.1
tests with no skips, plus API compatibility, dependency analysis, Spotless, and
RAT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]