>From Hussain Towaileb <[email protected]>:
Attention is currently required from: Hussain Towaileb.
Hello Jenkins, Peeyush Gupta,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20478?usp=email
to look at the new patch set (#6).
The following approvals got outdated and were removed:
Code-Review+1 by Hussain Towaileb, Verified+1 by Hussain Towaileb, Verified-1
by Jenkins
Change subject: [ASTERIXDB-3634][EXT]: Add support to Iceberg pt.2
......................................................................
[ASTERIXDB-3634][EXT]: Add support to Iceberg pt.2
Details:
- Supported AWS Glue catalog.
- Supported read path AWS Glue -> Parquet pipeline
- Tested types:
- boolean
- integer
- long
- float
- double
- string
- uuid
- decimal (to double)
- Supported projection
- Skeleton for supporting partition pruning and filtration
in place
- Custom AWS client providers to support all auth methods
Ext-ref: MB-63115
Change-Id: I531c613fe061367b2cb5c187bc985b39d6de17d1
---
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushValueAccessAndFilterDownRule.java
A
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/pushdown/processor/IcebergTableFilterPushdownProcessor.java
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/handlers/CatalogStatementHandler.java
M
asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/LocalCloudUtilAdobeMock.java
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/iceberg/catalog/all-supported-types/test.000.ddl.sqlpp
M
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java
M
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/metadata/IMetadataLockUtil.java
M asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties
M asterixdb/asterix-external-data/pom.xml
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/IIcebergRecordReaderFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/awsclient/EnsureCloseAWSClientFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/filter/IcebergTableFilterEvaluatorFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/filter/NoOpIcebergTableFilterEvaluatorFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/iceberg/IcebergFileRecordReader.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/iceberg/IcebergParquetRecordReaderFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/iceberg/converter/IcebergConverterContext.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/IcebergParquetDataParser.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/IcebergParserFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/IcebergTableParquetDataParserFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/HDFSUtils.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/aws/glue/GlueUtils.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/aws/s3/S3Utils.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/google/biglake_metastore/BiglakeMetastore.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/iceberg/IcebergConstants.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/iceberg/IcebergUtils.java
M
asterixdb/asterix-external-data/src/main/resources/META-INF/services/org.apache.asterix.external.api.IDataParserFactory
M
asterixdb/asterix-external-data/src/main/resources/META-INF/services/org.apache.asterix.external.api.IRecordReaderFactory
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataProvider.java
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/IndexUtil.java
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataLockUtil.java
M asterixdb/pom.xml
38 files changed, 1,745 insertions(+), 113 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/78/20478/6
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20478?usp=email
To unsubscribe, or for help writing mail filters, visit
https://asterix-gerrit.ics.uci.edu/settings?usp=email
Gerrit-MessageType: newpatchset
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I531c613fe061367b2cb5c187bc985b39d6de17d1
Gerrit-Change-Number: 20478
Gerrit-PatchSet: 6
Gerrit-Owner: Hussain Towaileb <[email protected]>
Gerrit-Reviewer: Hussain Towaileb <[email protected]>
Gerrit-Reviewer: Jenkins <[email protected]>
Gerrit-Reviewer: Peeyush Gupta <[email protected]>
Gerrit-Attention: Hussain Towaileb <[email protected]>
