alberttwong commented on issue #460:
URL:
https://github.com/apache/incubator-xtable/issues/460#issuecomment-2148585517
running sync again
```
root@spark:/opt/hudi/hudi-sync/hudi-hive-sync# cd /opt/xtable/jars/; java
-jar xtable-utilities-0.1.0-SNAPSHOT-bundled.jar --datasetConfig
xtable_delta.yaml -p core-site.xml
WARNING: Runtime environment or build system does not support multi-release
JARs. This will impact location-based features.
2024-06-04 23:40:55 INFO org.apache.xtable.utilities.RunSync:148 - Running
sync for basePath s3a://warehouse/people for following table formats [HUDI,
ICEBERG]
2024-06-04 23:40:55 WARN org.apache.hadoop.util.NativeCodeLoader:60 -
Unable to load native-hadoop library for your platform... using builtin-java
classes where applicable
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform
(file:/opt/xtable/jars/xtable-utilities-0.1.0-SNAPSHOT-bundled.jar) to
constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of
org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release
2024-06-04 23:40:56 WARN org.apache.hadoop.metrics2.impl.MetricsConfig:136
- Cannot locate configuration: tried
hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
2024-06-04 23:40:56 WARN org.apache.hadoop.fs.s3a.SDKV2Upgrade:39 -
Directly referencing AWS SDK V1 credential provider
com.amazonaws.auth.DefaultAWSCredentialsProviderChain. AWS SDK V1 credential
providers will be removed once S3A is upgraded to SDK V2
2024-06-04 23:40:57 INFO
org.apache.spark.sql.delta.storage.DelegatingLogStore:60 - LogStore
`LogStoreAdapter(io.delta.storage.S3SingleDriverLogStore)` is used for scheme
`s3a`
2024-06-04 23:40:57 INFO org.apache.spark.sql.delta.DeltaLog:60 - Loading
version 0.
2024-06-04 23:40:58 INFO org.apache.spark.sql.delta.DeltaLogFileIndex:60 -
Created DeltaLogFileIndex(JSON, numFilesInSegment: 1, totalFileSize: 3848)
2024-06-04 23:40:59 INFO org.apache.spark.sql.delta.Snapshot:60 -
[tableId=3ce0a1d7-8227-4adc-950d-4909c31ce4ec] Created snapshot
Snapshot(path=s3a://warehouse/people/_delta_log, version=0,
metadata=Metadata(f9b5140a-bd50-4095-9695-021c48c8e079,null,null,Format(parquet,Map()),{"type":"struct","fields":[{"name":"id","type":"integer","nullable":true,"metadata":{}},{"name":"name","type":"string","nullable":true,"metadata":{}},{"name":"age","type":"integer","nullable":true,"metadata":{}},{"name":"city","type":"string","nullable":true,"metadata":{}},{"name":"create_ts","type":"string","nullable":true,"metadata":{}}]},List(city),Map(),Some(1717543438122)),
logSegment=LogSegment(s3a://warehouse/people/_delta_log,0,WrappedArray(S3AFileStatus{path=s3a://warehouse/people/_delta_log/00000000000000000000.json;
isDirectory=false; length=3848; replication=1; blocksize=33554432;
modification_time=1717543440230; access_time=0; owner=root; group=root;
permission=rw-rw-rw-; isSymlink=false; hasAcl
=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE
eTag=b4ef5c900263aaf74ca626d27d79cc3c versionId=null),None,1717543440230),
checksumOpt=None)
2024-06-04 23:40:59 INFO
org.apache.hudi.common.table.HoodieTableMetaClient:133 - Loading
HoodieTableMetaClient from s3a://warehouse/people
2024-06-04 23:40:59 INFO org.apache.hudi.common.table.HoodieTableConfig:276
- Loading table properties from s3a://warehouse/people/.hoodie/hoodie.properties
2024-06-04 23:40:59 INFO
org.apache.hudi.common.table.HoodieTableMetaClient:152 - Finished Loading Table
of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from
s3a://warehouse/people
2024-06-04 23:40:59 INFO org.apache.iceberg.hadoop.HadoopTables:101 - Table
location loaded: s3a://warehouse/people
2024-06-04 23:40:59 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline:171 - Loaded
instants upto :
Option{val=[20240604232400230__replacecommit__COMPLETED__20240604232710563]}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
