>From Calvin Thomas Dani <[email protected]>: Calvin Thomas Dani has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21159?usp=email )
Change subject: [ASTERIXDB-3760] Training VTree index - user model changes: no - storage format changes: yes - interface changes: yes ...................................................................... [ASTERIXDB-3760] Training VTree index - user model changes: no - storage format changes: yes - interface changes: yes Enabled end-to-end VTree vector index creation from metadata (without DDL-only path) by adding clustering-based training, static structure build, grouped bulk load, and quantization (SQ4/SQ8) support across metadata, runtime, and storage layers. Change-Id: I1264840ffd536d91c0ee11634abfd35a0ad79ecd --- M asterixdb/asterix-common/pom.xml M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/DatasetConfig.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/OptimizedScalarQuantizationSampleFile.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/QuantizationConstants.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/QuantizationConstantsFileManager.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/ScalarVectorQuantizer.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/StaticStructureFileManager.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Index.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/ISecondaryIndexOperationsHelper.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/IndexUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SampleOperationsHelper.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryIndexOperationsHelper.java A asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryVectorOperationsHelper.java A asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/VCTreeResourceFactoryProvider.java A asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/AOrderedListVectorBinaryAccessor.java A asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/AOrderedListVectorBinaryAccessorFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java M asterixdb/asterix-runtime/pom.xml A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/QuantizationConstantsAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/CentroidsState.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/HierarchicalCentroidsState.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/HierarchicalClusterId.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/HierarchicalKMeansPlusPlusCentroidsOperatorDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/KMeansUtils.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/QuantileCalculatorOperatorDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/ScalarValueRunFileWriter.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/VCTreeBulkLoaderAndGroupingOperatorDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/VCTreePartitioner.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/VCTreeStaticStructureCreatorOperatorDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/VectorComponentExtractorOperatorDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/VectorDistanceArrCalculation.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/misc/MaterializerTaskState.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/misc/PartitionedUUID.java M hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/build/IndexBuilder.java A hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/vector/dataflow/QuantizedIndexCreateOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-storage-am-vtree/pom.xml A hyracks-fullstack/hyracks/hyracks-storage-am-vtree/src/main/java/org/apache/hyracks/storage/am/vector/impls/VectorClusteringTreeFactory.java 38 files changed, 11,884 insertions(+), 20 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/59/21159/1 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21159?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: newchange Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I1264840ffd536d91c0ee11634abfd35a0ad79ecd Gerrit-Change-Number: 21159 Gerrit-PatchSet: 1 Gerrit-Owner: Calvin Thomas Dani <[email protected]>
