This is an automated email from the ASF dual-hosted git repository.
Fokko pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/parquet-java.git
The following commit(s) were added to refs/heads/master by this push:
new 2ea192a41 Bump minimum Java version to 17 (#3579)
2ea192a41 is described below
commit 2ea192a411af766629e229c021eba84456e6a60f
Author: Ismaël Mejía <[email protected]>
AuthorDate: Tue Aug 11 22:29:44 2026 +0200
Bump minimum Java version to 17 (#3579)
Raise the minimum build/runtime requirement to Java 17 and clean up
workarounds and tooling that are no longer needed.
Java 17 baseline:
- Set maven.compiler.release to 17 and add an enforcer rule that
requires Java 17+ to build.
- Remove the extraJavaTestArgs (--add-opens / -XX flags) that were
needed to run the tests on newer JDKs from a Java 11 baseline.
- Drop the per-module maven.compiler.release=17 override in
parquet-encoding-vector; it is now the project-wide default.
- Simplify the CleanUtil javadoc.
- CI: build only on JDK 17 (drop the JDK 11 matrix entry) and stop
exporting extraJavaTestArgs.
- Update README to state the Java 17 requirement.
Spotless upgrade (2.46.1 -> 3.8.0):
- Spotless 2.46.1 calls com.sun.tools.javac.util.Log methods that were
removed in JDK 25, causing NoSuchMethodError at format time. Spotless
3.8.0 is compatible with JDK 25. The switch/case comment indentation
changes are produced by the new formatter version.
Remove obsolete japicmp exclude for IndexIterator:
- The exclude worked around a false-positive source incompatibility
reported by japicmp 0.23.1 on the synthetic/bridge methods of
PrimitiveIterator.OfInt (#3388). japicmp was upgraded to 0.26.1
(#3592), which handles those methods correctly, so the workaround is
no longer needed. Verified that japicmp:cmp (with
breakBuildOnSourceIncompatibleModifications=true) passes with the
exclude removed.
Note: the Hadoop 3.3.0 -> 3.4.3 bump and the related vectored-I/O
ByteBuffer leak fix that previously lived here have been split out; the
Hadoop upgrade is tracked separately and will come via #3559.
---
.github/workflows/ci-hadoop3.yml | 6 +--
.github/workflows/vector-plugins.yml | 4 --
.gitignore | 2 +
README.md | 2 +-
.../format/converter/ParquetMetadataConverter.java | 2 +-
.../org/apache/parquet/hadoop/CodecFactory.java | 2 +-
.../parquet/hadoop/ColumnIndexValidator.java | 2 +-
.../apache/parquet/hadoop/DirectCodecFactory.java | 4 +-
.../org/apache/parquet/hadoop/codec/CleanUtil.java | 7 +---
parquet-plugins/parquet-encoding-vector/pom.xml | 5 ---
.../thrift/BufferedProtocolReadToWrite.java | 2 +-
.../apache/parquet/thrift/ProtocolReadToWrite.java | 2 +-
pom.xml | 43 +++++++++-------------
13 files changed, 31 insertions(+), 52 deletions(-)
diff --git a/.github/workflows/ci-hadoop3.yml b/.github/workflows/ci-hadoop3.yml
index 179300929..04aa1bd02 100644
--- a/.github/workflows/ci-hadoop3.yml
+++ b/.github/workflows/ci-hadoop3.yml
@@ -31,7 +31,7 @@ jobs:
strategy:
fail-fast: false
matrix:
- java: [ { setup: '11', maven: '11' }, { setup: '17', maven: '17' } ]
+ java: [ { setup: '17', maven: '17' } ]
name: Build Parquet with JDK ${{ matrix.java.setup }}
steps:
@@ -48,16 +48,12 @@ jobs:
bash dev/ci-before_install.sh
- name: install
run: |
- EXTRA_JAVA_TEST_ARGS=$(./mvnw help:evaluate
-Dexpression=extraJavaTestArgs -q -DforceStdout)
- export MAVEN_OPTS="$MAVEN_OPTS $EXTRA_JAVA_TEST_ARGS"
./mvnw install --batch-mode -DskipTests=true
-Dmaven.javadoc.skip=true -Dsource.skip=true -Djava.version=${{
matrix.java.maven }}
- name: verify
env:
TEST_CODECS: 'uncompressed,brotli,gzip,snappy,zstd'
JAVA_VERSION: ${{ matrix.java.setup }}
run: |
- EXTRA_JAVA_TEST_ARGS=$(./mvnw help:evaluate
-Dexpression=extraJavaTestArgs -q -DforceStdout)
- export MAVEN_OPTS="$MAVEN_OPTS $EXTRA_JAVA_TEST_ARGS"
./mvnw verify --batch-mode javadoc:javadoc
- name: generate published JARs and verify LICENSE and NOTICE
if: matrix.java.setup == '17'
diff --git a/.github/workflows/vector-plugins.yml
b/.github/workflows/vector-plugins.yml
index 0c48816bf..22e7b244f 100644
--- a/.github/workflows/vector-plugins.yml
+++ b/.github/workflows/vector-plugins.yml
@@ -49,8 +49,6 @@ jobs:
bash dev/ci-before_install.sh
- name: install
run: |
- EXTRA_JAVA_TEST_ARGS=$(./mvnw help:evaluate
-Dexpression=extraJavaTestArgs -q -DforceStdout)
- export MAVEN_OPTS="$MAVEN_OPTS $EXTRA_JAVA_TEST_ARGS"
# Spotless check uses palantir-java-format which relies on internal
javac APIs
# that are not available on all JDK versions (e.g. JDK 25+). Since
the formatting
# result is JDK-independent, running the check on JDK 17 alone is
sufficient.
@@ -64,8 +62,6 @@ jobs:
TEST_CODECS: ${{ matrix.codes }}
JAVA_VERSION: ${{ matrix.java }}
run: |
- EXTRA_JAVA_TEST_ARGS=$(./mvnw help:evaluate
-Dexpression=extraJavaTestArgs -q -DforceStdout)
- export MAVEN_OPTS="$MAVEN_OPTS $EXTRA_JAVA_TEST_ARGS"
# Spotless check uses palantir-java-format which relies on internal
javac APIs
# that are not available on all JDK versions (e.g. JDK 25+). Since
the formatting
# result is JDK-independent, running the check on JDK 17 alone is
sufficient.
diff --git a/.gitignore b/.gitignore
index c02d0f222..f0d310a4a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -22,3 +22,5 @@ mvn_install.log
.vscode/*
.DS_Store
+.sdkmanrc
+
diff --git a/README.md b/README.md
index 1ce03d5fd..aef3cbad9 100644
--- a/README.md
+++ b/README.md
@@ -36,7 +36,7 @@ You can find additional details about the format and intended
use cases in our [
## Building
-Parquet-Java uses Maven to build and depends on the thrift compiler (protoc is
now managed by maven plugin).
+Parquet-Java requires Java 17 or higher and uses Maven to build. It also
depends on the thrift compiler (protoc is now managed by maven plugin).
### Install Thrift
diff --git
a/parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java
b/parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java
index 473a15710..cf9c29f4d 100644
---
a/parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java
+++
b/parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java
@@ -778,7 +778,7 @@ public class ParquetMetadataConverter {
switch (stat.getPage_type()) {
case DATA_PAGE_V2:
builder.withV2Pages();
- // falls through
+ // falls through
case DATA_PAGE:
builder.addDataEncoding(getEncoding(stat.getEncoding()),
stat.getCount());
break;
diff --git
a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java
b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java
index c9391201f..9024954bb 100644
--- a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java
+++ b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java
@@ -456,7 +456,7 @@ public class CodecFactory implements
CompressionCodecFactory {
level = conf.get("parquet.compression.codec.zstd.level");
break;
default:
- // compression level is not supported; ignore it
+ // compression level is not supported; ignore it
}
String codecClass = codecName.getHadoopCompressionCodecClassName();
return level == null ? codecClass : codecClass + ":" + level;
diff --git
a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexValidator.java
b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexValidator.java
index 92f7db413..5142506d2 100644
---
a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexValidator.java
+++
b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnIndexValidator.java
@@ -546,7 +546,7 @@ public class ColumnIndexValidator {
prevMaxValue::toString);
break;
case UNORDERED:
- // No checks necessary.
+ // No checks necessary.
}
}
}
diff --git
a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/DirectCodecFactory.java
b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/DirectCodecFactory.java
index 312ccedf8..4788c5f04 100644
---
a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/DirectCodecFactory.java
+++
b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/DirectCodecFactory.java
@@ -103,8 +103,8 @@ class DirectCodecFactory extends CodecFactory implements
AutoCloseable {
return new SnappyCompressor();
case ZSTD:
return new ZstdCompressor();
- // todo: create class similar to the SnappyCompressor for zlib and
exclude it as
- // snappy is above since it also generates allocateDirect calls.
+ // todo: create class similar to the SnappyCompressor for zlib and
exclude it as
+ // snappy is above since it also generates allocateDirect calls.
default:
return super.createCompressor(codecName);
}
diff --git
a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/CleanUtil.java
b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/CleanUtil.java
index 3cf88db87..fe139d47e 100644
---
a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/CleanUtil.java
+++
b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/CleanUtil.java
@@ -26,11 +26,8 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
- * A helper class that uses {@code sun.misc.Unsafe#invokeCleaner(ByteBuffer)}
- * reflectively to clean up direct buffers.
- * <p>
- * Strongly inspired by:
- * <a
href="https://github.com/apache/tomcat/blob/master/java/org/apache/tomcat/util/buf/ByteBufferUtils.java";>Tomcat
ByteBufferUtils</a>
+ * A helper class which uses {@code sun.misc.Unsafe.invokeCleaner} to
explicitly free
+ * direct ByteBuffers.
*/
public class CleanUtil {
private static final Logger logger =
LoggerFactory.getLogger(CleanUtil.class);
diff --git a/parquet-plugins/parquet-encoding-vector/pom.xml
b/parquet-plugins/parquet-encoding-vector/pom.xml
index 06c4084be..4ceb0b43d 100644
--- a/parquet-plugins/parquet-encoding-vector/pom.xml
+++ b/parquet-plugins/parquet-encoding-vector/pom.xml
@@ -34,11 +34,6 @@
<name>Apache Parquet Encodings Vector</name>
<url>https://parquet.apache.org</url>
- <properties>
- <!-- Those properties prevent Java 11 to try and compile this code -->
- <maven.compiler.release>17</maven.compiler.release>
- </properties>
-
<dependencies>
<dependency>
<groupId>org.apache.parquet</groupId>
diff --git
a/parquet-thrift/src/main/java/org/apache/parquet/thrift/BufferedProtocolReadToWrite.java
b/parquet-thrift/src/main/java/org/apache/parquet/thrift/BufferedProtocolReadToWrite.java
index e1c2aae57..07c31c358 100644
---
a/parquet-thrift/src/main/java/org/apache/parquet/thrift/BufferedProtocolReadToWrite.java
+++
b/parquet-thrift/src/main/java/org/apache/parquet/thrift/BufferedProtocolReadToWrite.java
@@ -226,7 +226,7 @@ public class BufferedProtocolReadToWrite implements
ProtocolPipe {
writeShortAction(buffer, s);
break;
case TType.ENUM: // same as i32 => actually never seen in the protocol
layer as enums are written as a i32
- // field
+ // field
case TType.I32:
final int i = in.readI32();
checkEnum(expectedType, i);
diff --git
a/parquet-thrift/src/main/java/org/apache/parquet/thrift/ProtocolReadToWrite.java
b/parquet-thrift/src/main/java/org/apache/parquet/thrift/ProtocolReadToWrite.java
index 94c68b008..830a5c605 100644
---
a/parquet-thrift/src/main/java/org/apache/parquet/thrift/ProtocolReadToWrite.java
+++
b/parquet-thrift/src/main/java/org/apache/parquet/thrift/ProtocolReadToWrite.java
@@ -74,7 +74,7 @@ public class ProtocolReadToWrite implements ProtocolPipe {
out.writeI16(in.readI16());
break;
case TType.ENUM: // same as i32 => actually never seen in the protocol
layer as enums are written as a i32
- // field
+ // field
case TType.I32:
out.writeI32(in.readI32());
break;
diff --git a/pom.xml b/pom.xml
index 8fcacdeb1..b3344db73 100644
--- a/pom.xml
+++ b/pom.xml
@@ -69,7 +69,7 @@
</mailingLists>
<properties>
- <maven.compiler.release>11</maven.compiler.release>
+ <maven.compiler.release>17</maven.compiler.release>
<github.global.server>github</github.global.server>
<jackson.groupId>com.fasterxml.jackson.core</jackson.groupId>
<jackson.datatype.groupId>com.fasterxml.jackson.datatype</jackson.datatype.groupId>
@@ -80,7 +80,7 @@
<jackson-annotations.version>2.22</jackson-annotations.version>
<japicmp.version>0.26.1</japicmp.version>
<javax.annotation.version>1.3.2</javax.annotation.version>
- <spotless.version>2.46.1</spotless.version>
+ <spotless.version>3.8.0</spotless.version>
<shade.prefix>shaded.parquet</shade.prefix>
<!-- Guarantees no newer classes/methods/constants are used by parquet. -->
<hadoop.version>3.3.0</hadoop.version>
@@ -122,23 +122,6 @@
<!-- Resource intesive tests are enabled by default but disabled in the CI
envrionment -->
<enableResourceIntensiveTests>true</enableResourceIntensiveTests>
-
- <extraJavaTestArgs>
- -XX:+IgnoreUnrecognizedVMOptions
- --add-opens=java.base/java.lang=ALL-UNNAMED
- --add-opens=java.base/java.lang.invoke=ALL-UNNAMED
- --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
- --add-opens=java.base/java.io=ALL-UNNAMED
- --add-opens=java.base/java.net=ALL-UNNAMED
- --add-opens=java.base/java.nio=ALL-UNNAMED
- --add-opens=java.base/java.util=ALL-UNNAMED
- --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
- --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
- --add-opens=java.base/sun.nio.ch=ALL-UNNAMED
- --add-opens=java.base/sun.nio.cs=ALL-UNNAMED
- --add-opens=java.base/sun.security.action=ALL-UNNAMED
- --add-opens=java.base/sun.util.calendar=ALL-UNNAMED
- </extraJavaTestArgs>
</properties>
<modules>
@@ -318,6 +301,21 @@
</dependency>
</dependencies>
<executions>
+ <execution>
+ <id>enforce-java-version</id>
+ <goals>
+ <goal>enforce</goal>
+ </goals>
+ <configuration>
+ <rules>
+ <requireJavaVersion>
+ <version>[17,)</version>
+ <message>Java 17 or higher is required to build
parquet-java.</message>
+ </requireJavaVersion>
+ </rules>
+ <fail>true</fail>
+ </configuration>
+ </execution>
<execution>
<id>enforce-banned-dependencies</id>
<goals>
@@ -486,9 +484,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
- <configuration>
- <argLine>${extraJavaTestArgs}</argLine>
- </configuration>
<executions>
<execution>
<goals>
@@ -503,7 +498,7 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
- <argLine>${surefire.argLine} ${extraJavaTestArgs}</argLine>
+ <argLine>${surefire.argLine}</argLine>
<systemPropertyVariables>
<!-- Configure Parquet logging during tests
See http://www.slf4j.org/api/org/slf4j/impl/SimpleLogger.html
@@ -644,8 +639,6 @@
</excludeModules>
<excludes>
<exclude>${shade.prefix}</exclude>
- <!-- JDK 11 adds interface methods/bridges on
PrimitiveIterator.OfInt; ignore japicmp source incompatibility -->
-
<exclude>org.apache.parquet.internal.column.columnindex.IndexIterator</exclude>
<!-- Removal of a protected method in a class that's not
supposed to be subclassed by third-party code -->
<exclude>org.apache.parquet.column.values.bytestreamsplit.ByteStreamSplitValuesReader#gatherElementDataFromStreams(byte[])</exclude>
<!-- Removal of a protected internal field that should not have
been part of the public API -->
