[
https://issues.apache.org/jira/browse/IMPALA-9285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254227#comment-17254227
]
Joe McDonnell commented on IMPALA-9285:
---------------------------------------
As part of IMPALA-10198 the maven code that was in testdata is now in
java/datagenerator. It runs as part of the regular build rather than data load.
I think it is reasonable to close this issue.
> Maven compile error on testdata module
> --------------------------------------
>
> Key: IMPALA-9285
> URL: https://issues.apache.org/jira/browse/IMPALA-9285
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> I continuely hit this error when loading the test data:
> {code:java}
> Running mvn package
> Directory /home/quanlong/workspace/Impala/testdata
> ========================================================================
> [WARNING] Could not transfer metadata
> com.cloudera.cdh:cdh-root:6.x-SNAPSHOT/maven-metadata.xml from/to
> ${distMgmtSnapshotsId} (${distMgmtSnapshotsUrl}): Cannot access
> ${distMgmtSnapshotsUrl} with type default using the available connector
> factories: BasicRepositoryConnectorFactory
> [WARNING] The POM for
> org.apache.parquet:parquet-avro:jar:1.10.99-cdh6.x-20191031.203926-1582079 is
> invalid, transitive dependencies (if any) will not be available, enable debug
> logging for more details
> [ERROR] COMPILATION ERROR :
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[39,42]
> package org.apache.parquet.hadoop.metadata does not exist
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[79,46]
> cannot access org.apache.parquet.hadoop.ParquetWriter
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[80,26]
> cannot find symbol
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[81,26]
> cannot find symbol
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[78,47]
> cannot access org.apache.parquet.hadoop.api.WriteSupport
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[87,15]
> cannot find symbol
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[90,13]
> cannot find symbol
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[66,9]
> cannot find symbol
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[67,26]
> cannot find symbol
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[68,26]
> cannot find symbol
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[75,15]
> cannot find symbol
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[78,13]
> cannot find symbol
> [INFO] BUILD FAILURE
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile)
> on project impala-testdata: Compilation failure: Compilation failure:
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[39,42]
> package org.apache.parquet.hadoop.metadata does not exist
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[79,46]
> cannot access org.apache.parquet.hadoop.ParquetWriter
> [ERROR] class file for org.apache.parquet.hadoop.ParquetWriter not found
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[80,26]
> cannot find symbol
> [ERROR] symbol: variable DEFAULT_BLOCK_SIZE
> [ERROR] location: class org.apache.parquet.avro.AvroParquetWriter
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[81,26]
> cannot find symbol
> [ERROR] symbol: variable DEFAULT_PAGE_SIZE
> [ERROR] location: class org.apache.parquet.avro.AvroParquetWriter
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[78,47]
> cannot access org.apache.parquet.hadoop.api.WriteSupport
> [ERROR] class file for org.apache.parquet.hadoop.api.WriteSupport not found
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[87,15]
> cannot find symbol
> [ERROR] symbol: method write(org.apache.avro.generic.GenericRecord)
> [ERROR] location: variable writer of type
> org.apache.parquet.avro.AvroParquetWriter<org.apache.avro.generic.GenericRecord>
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/JsonToParquetConverter.java:[90,13]
> cannot find symbol
> [ERROR] symbol: method close()
> [ERROR] location: variable writer of type
> org.apache.parquet.avro.AvroParquetWriter<org.apache.avro.generic.GenericRecord>
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[66,9]
> cannot find symbol
> [ERROR] symbol: variable CompressionCodecName
> [ERROR] location: class
> org.apache.impala.datagenerator.RandomNestedDataGenerator
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[67,26]
> cannot find symbol
> [ERROR] symbol: variable DEFAULT_BLOCK_SIZE
> [ERROR] location: class org.apache.parquet.avro.AvroParquetWriter
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[68,26]
> cannot find symbol
> [ERROR] symbol: variable DEFAULT_PAGE_SIZE
> [ERROR] location: class org.apache.parquet.avro.AvroParquetWriter
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[75,15]
> cannot find symbol
> [ERROR] symbol: method write(org.apache.avro.generic.GenericData.Record)
> [ERROR] location: variable writer of type
> org.apache.parquet.avro.AvroParquetWriter<org.apache.avro.generic.GenericRecord>
> [ERROR]
> /home/quanlong/workspace/Impala/testdata/src/main/java/org/apache/impala/datagenerator/RandomNestedDataGenerator.java:[78,13]
> cannot find symbol
> [ERROR] symbol: method close()
> [ERROR] location: variable writer of type
> org.apache.parquet.avro.AvroParquetWriter<org.apache.avro.generic.GenericRecord>
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> mvn package exited with code 0
> ERROR in /home/quanlong/workspace/Impala/bin/create_testdata.sh at line 34:
> ${IMPALA_HOME}/bin/mvn-quiet.sh package
> Generated:
> /home/quanlong/workspace/Impala-CDH/logs/extra_junit_xml_logs/generate_junitxml.buildall.create_testdata.20200110_01_52_43.xml{code}
> I can work around it by adding these dependencies in testdata/pom.xml:
> {code}
> <dependency>
> <groupId>org.apache.parquet</groupId>
> <artifactId>parquet-hadoop</artifactId>
> <version>${parquet.version}</version>
> </dependency>
> <dependency>
> <groupId>org.apache.parquet</groupId>
> <artifactId>parquet-hadoop-bundle</artifactId>
> <version>${parquet.version}</version>
> </dependency>
> {code}
> Will spend time to dig into it later. The output of this JIRA may be a wiki
> for work around or a bug fix patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
