amnchauhan commented on issue #443:
URL:
https://github.com/apache/incubator-xtable/issues/443#issuecomment-2135110640
> @amnchauhan it looks like `-cp` and `-jar` do not work properly together,
can you try with just `-cp`? You may want to consider building a bundled jar
with all of the dependencies you require.
@the-other-tim-brown i have created bundled jar with by adding dependencies
in pom.xml and also tried adding all dependencies in my classpath as below but
still issue persist.
```
[root@incubator-xtable-main]# java -cp
./xtable-utilities/target/xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:/lib/iceberg-hive-metastore-1.4.2.jar:/lib/iceberg-hive-runtime-1.4.2.jar
org.apache.xtable.utilities.RunSync -d my_config.yaml -i
icebergCatalogConfig.yaml --hadoopConfig my_hadoop_conf.xml
2024-05-28 17:56:02 INFO org.apache.xtable.utilities.RunSync:148 - Running
sync for basePath
s3a://spark-iceberg-test/streaming_test_catalog/streaming_iceberg_test_pp for
following table formats [HUDI]
2024-05-28 17:56:02 INFO
org.apache.hudi.common.table.HoodieTableMetaClient:133 - Loading
HoodieTableMetaClient from
s3a://spark-iceberg-test/streaming_test_catalog/streaming_iceberg_test_pp/data
2024-05-28 17:56:02 WARN org.apache.hadoop.util.NativeCodeLoader:60 -
Unable to load native-hadoop library for your platform... using builtin-java
classes where applicable
2024-05-28 17:56:02 WARN org.apache.hadoop.metrics2.impl.MetricsConfig:136
- Cannot locate configuration: tried
hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
2024-05-28 17:56:03 INFO org.apache.xtable.hudi.HudiTableManager:73 - Hudi
table does not exist, will be created on first sync
2024-05-28 17:56:03 INFO
org.apache.xtable.conversion.ConversionController:240 - No previous
InternalTable sync for target. Falling back to snapshot sync.
2024-05-28 17:56:03 ERROR org.apache.xtable.utilities.RunSync:171 - Error
running sync for
s3a://spark-iceberg-test/streaming_test_catalog/streaming_iceberg_test_pp
java.lang.IllegalArgumentException: Cannot initialize Catalog implementation
org.apache.iceberg.hive.HiveCatalog: Cannot find constructor for interface
org.apache.iceberg.catalog.Catalog
Missing org.apache.iceberg.hive.HiveCatalog
[java.lang.NoClassDefFoundError:
org/apache/hadoop/hive/metastore/api/UnknownDBException]
at org.apache.iceberg.CatalogUtil.loadCatalog(CatalogUtil.java:224)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.iceberg.IcebergTableManager.lambda$getCatalog$6(IcebergTableManager.java:116)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
~[?:1.8.0_412]
at
org.apache.xtable.iceberg.IcebergTableManager.getCatalog(IcebergTableManager.java:113)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.iceberg.IcebergTableManager.getTable(IcebergTableManager.java:56)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.iceberg.IcebergConversionSource.initSourceTable(IcebergConversionSource.java:81)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.iceberg.IcebergConversionSource.getSourceTable(IcebergConversionSource.java:60)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.iceberg.IcebergConversionSource.getCurrentSnapshot(IcebergConversionSource.java:121)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.spi.extractor.ExtractFromSource.extractSnapshot(ExtractFromSource.java:38)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.conversion.ConversionController.syncSnapshot(ConversionController.java:183)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.conversion.ConversionController.sync(ConversionController.java:121)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at org.apache.xtable.utilities.RunSync.main(RunSync.java:169)
[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
Caused by: java.lang.NoSuchMethodException: Cannot find constructor for
interface org.apache.iceberg.catalog.Catalog
Missing org.apache.iceberg.hive.HiveCatalog
[java.lang.NoClassDefFoundError:
org/apache/hadoop/hive/metastore/api/UnknownDBException]
at
org.apache.iceberg.common.DynConstructors.buildCheckedException(DynConstructors.java:250)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.iceberg.common.DynConstructors.access$200(DynConstructors.java:32)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.iceberg.common.DynConstructors$Builder.buildChecked(DynConstructors.java:220)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at org.apache.iceberg.CatalogUtil.loadCatalog(CatalogUtil.java:221)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
... 11 more
Suppressed: java.lang.NoClassDefFoundError:
org/apache/hadoop/hive/metastore/api/UnknownDBException
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_412]
at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_412]
at
org.apache.iceberg.common.DynConstructors$Builder.impl(DynConstructors.java:149)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.iceberg.CatalogUtil.loadCatalog(CatalogUtil.java:221)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.iceberg.IcebergTableManager.lambda$getCatalog$6(IcebergTableManager.java:116)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
~[?:1.8.0_412]
at
org.apache.xtable.iceberg.IcebergTableManager.getCatalog(IcebergTableManager.java:113)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.iceberg.IcebergTableManager.getTable(IcebergTableManager.java:56)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.iceberg.IcebergConversionSource.initSourceTable(IcebergConversionSource.java:81)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.iceberg.IcebergConversionSource.getSourceTable(IcebergConversionSource.java:60)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.iceberg.IcebergConversionSource.getCurrentSnapshot(IcebergConversionSource.java:121)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.spi.extractor.ExtractFromSource.extractSnapshot(ExtractFromSource.java:38)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.conversion.ConversionController.syncSnapshot(ConversionController.java:183)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.conversion.ConversionController.sync(ConversionController.java:121)
~[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
at
org.apache.xtable.utilities.RunSync.main(RunSync.java:169)
[xtable-utilities-0.1.0-SNAPSHOT-bundled.jar:0.1.0-SNAPSHOT]
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hive.metastore.api.UnknownDBException
at
java.net.URLClassLoader.findClass(URLClassLoader.java:387) ~[?:1.8.0_412]
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
~[?:1.8.0_412]
at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) ~[?:1.8.0_412]
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
~[?:1.8.0_412]
... 15 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
