nastra commented on a change in pull request #2826:
URL: https://github.com/apache/iceberg/pull/2826#discussion_r678045366
##########
File path: build.gradle
##########
@@ -247,13 +250,30 @@ project(':iceberg-data') {
exclude group: 'org.slf4j', module: 'slf4j-log4j12'
}
- testCompile("org.apache.hadoop:hadoop-client") {
+ implementation("org.apache.orc:orc-core::nohive") {
Review comment:
> This adds implementation dependencies for org.apache.avro:avro and
org.apache.orc:orc-core. Why isn't there one for org.apache.parquet?
Judging by the 3 parquet imports that are being used in `:iceberg-data`, we
have:
```
import org.apache.parquet.hadoop.ParquetFileReader;
import org.apache.parquet.hadoop.metadata.ParquetMetadata;
```
These 2 are satisfied by `org.apache.parquet:parquet-hadoop`. and then
`import org.apache.parquet.Preconditions` (which should probably be replaced
with `org.apache.iceberg.relocated.com.google.common.base.Preconditions`),
which is being satisfied by `org.apache.parquet:parquet-common`. Both of these
libs are part of `org.apache.parquet:parquet-avro` as can be seen below:
```
\--- org.apache.parquet:parquet-avro -> 1.12.0
+--- org.apache.parquet:parquet-column:1.12.0
| +--- org.apache.parquet:parquet-common:1.12.0
| | +--- org.apache.parquet:parquet-format-structures:1.12.0
| | | +--- org.slf4j:slf4j-api:1.7.22 -> 1.7.25
| | | \--- javax.annotation:javax.annotation-api:1.3.2
| | +--- org.slf4j:slf4j-api:1.7.22 -> 1.7.25
| | \--- org.apache.yetus:audience-annotations:0.12.0
| \--- org.apache.parquet:parquet-encoding:1.12.0
| \--- org.apache.parquet:parquet-common:1.12.0 (*)
+--- org.apache.parquet:parquet-hadoop:1.12.0
| +--- org.apache.parquet:parquet-column:1.12.0 (*)
| +--- org.apache.parquet:parquet-format-structures:1.12.0 (*)
| +--- org.apache.parquet:parquet-jackson:1.12.0
| +--- org.xerial.snappy:snappy-java:1.1.8
| +--- commons-pool:commons-pool:1.6
| \--- com.github.luben:zstd-jni:1.4.9-1
\--- org.apache.parquet:parquet-format-structures:1.12.0 (*)
```
Thus `org.apache.parquet:parquet-avro` carries everything we need for
`:iceberg-data` in this case. Does that answer your question?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
