ryoma-nagata opened a new issue, #667: URL: https://github.com/apache/incubator-xtable/issues/667
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/incubator-xtable/issues?q=is%3Aissue) and found no similar issues. ### Please describe the bug 🐞 I ran into an issue while using Snowflake's (Azure) polaris catalog. Documenting here. ``` java -jar xtable-utilities/target/xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar --datasetConfig ./demo/config.yml -i ./demo/catalog.yaml ``` ## Error ``` 2025-03-17 16:41:17 INFO org.apache.iceberg.metrics.LoggingMetricsReporter:38 - Received metrics report: ScanReport{tableName=azure_ext.ICEBIRG_DEMO.PUBLIC.TEST_TABLE_MANAGED_ST, snapshotId=8300558301514482769, filter=true, schemaId=0, projectedFieldIds=[1], projectedFieldNames=[COL1], scanMetrics=ScanMetricsResult{totalPlanningDuration=TimerResult{timeUnit=NANOSECONDS, totalDuration=PT0.547000254S, count=1}, resultDataFiles=CounterResult{unit=COUNT, value=1}, resultDeleteFiles=CounterResult{unit=COUNT, value=0}, totalDataManifests=CounterResult{unit=COUNT, value=1}, totalDeleteManifests=CounterResult{unit=COUNT, value=0}, scannedDataManifests=CounterResult{unit=COUNT, value=1}, skippedDataManifests=CounterResult{unit=COUNT, value=0}, totalFileSizeInBytes=CounterResult{unit=BYTES, value=512}, totalDeleteFileSizeInBytes=CounterResult{unit=BYTES, value=0}, skippedDataFiles=CounterResult{unit=COUNT, value=0}, skippedDeleteFiles=CounterResult{unit=COUNT, value=0}, scannedDeleteManife sts=CounterResult{unit=COUNT, value=0}, skippedDeleteManifests=CounterResult{unit=COUNT, value=0}, indexedDeleteFiles=CounterResult{unit=COUNT, value=0}, equalityDeleteFiles=CounterResult{unit=COUNT, value=0}, positionalDeleteFiles=CounterResult{unit=COUNT, value=0}}, metadata={iceberg-version=Apache Iceberg 1.4.2 (commit f6bb9173b13424d77e7ad8439b5ef9627e530cb2)}} 2025-03-17 16:41:17 ERROR org.apache.xtable.utilities.RunSync:239 - Error running sync for abfss:///my-contai...@myaccount.dfs.core.windows.net/hub/public/test_table_managed_st java.lang.NullPointerException: null at org.apache.xtable.iceberg.IcebergColumnStatsConverter.lambda$fromIceberg$2(IcebergColumnStatsConverter.java:106) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?] at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) ~[?:?] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?] at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?] at org.apache.xtable.iceberg.IcebergColumnStatsConverter.fromIceberg(IcebergColumnStatsConverter.java:119) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at org.apache.xtable.iceberg.IcebergDataFileExtractor.fromIceberg(IcebergDataFileExtractor.java:64) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at org.apache.xtable.iceberg.IcebergDataFileExtractor.fromIceberg(IcebergDataFileExtractor.java:53) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at org.apache.xtable.iceberg.IcebergConversionSource.fromIceberg(IcebergConversionSource.java:176) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at org.apache.xtable.iceberg.IcebergConversionSource.getCurrentSnapshot(IcebergConversionSource.java:156) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at org.apache.xtable.spi.extractor.ExtractFromSource.extractSnapshot(ExtractFromSource.java:38) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at org.apache.xtable.conversion.ConversionController.syncSnapshot(ConversionController.java:282) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at org.apache.xtable.conversion.ConversionController.syncTableFormats(ConversionController.java:202) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at org.apache.xtable.conversion.ConversionController.sync(ConversionController.java:105) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at org.apache.xtable.utilities.RunSync.runSync(RunSync.java:237) [xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] at org.apache.xtable.utilities.RunSync.main(RunSync.java:160) [xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT] ``` ## config.yaml ```sourceFormat: ICEBERG targetFormats: - DELTA datasets: - tableBasePath: abfss://my-contai...@myaccount.dfs.core.windows.net/hub/public/test_table_managed_st tableDataPath: abfss://my-contai...@myaccount.dfs.core.windows.net/hub/public/test_table_managed_st/data # replace this with gs://path/to/delta-dataset/people if your data is in GCS. tableName: TEST_TABLE_MANAGED_ST namespace: ICEBIRG_DEMO.PUBLIC ``` ## catalog.yaml ``` catalogImpl: org.apache.iceberg.rest.RESTCatalog catalogName: azure_ext catalogOptions: warehouse: azure_ext uri: https://<id>.snowflakecomputing.com/polaris/api/catalog credential: <>:<> header.X-Iceberg-Access-Delegation: vended-credentials scope: PRINCIPAL_ROLE:ALL client.region: japan-east.azure ``` This error occurs with public access to the blob enabled, so I don't believe it's related to credential vending. ### Are you willing to submit PR? - [ ] I am willing to submit a PR! - [ ] I am willing to submit a PR but need help getting started! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@xtable.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
