>From Murtadha Hubail <[email protected]>: Attention is currently required from: Ritik Raj.
Murtadha Hubail has posted comments on this change by Ritik Raj. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366?usp=email ) Change subject: [ASTERIXDB-3702][RT][STO] LSM Sampling ...................................................................... Patch Set 1: (17 comments) File asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/16fd8f50_4b1214a3?usp=email : PS1, Line 5214: // Compile-time check: datasets created before theta sketch support cannot be randomly sampled : if (!fullScan && !dsDetails.isRandomSamplable()) { : LOGGER.warn("Dataset '{}' does not support random sampling " : + "(created before theta sketch support); forcing full scan.", datasetName); : fullScan = true; : } : Can we fall back to full scan when we know that the sample we tried to create didn't produce a good sample or an empty sample? File asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/GlobalVirtualBufferCacheTest.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/bb92a950_fd8f1030?usp=email : PS1, Line 103: ThetaSampler.setSamplingEnabled(false); Can we adjust the config and remove this so the new future we don't have to add this to every test? File asterixdb/asterix-app/src/test/resources/metadata/results/basic/dataset_with_meta-1/dataset_with_meta-1.1.adm: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/7b317fda_be5a6610?usp=email : PS1, Line 1: RandomSamplable we might not need this after all File asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/statement/AnalyzeStatement.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/c327300e_ef0a00f0?usp=email : PS1, Line 181: n n File asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/IndexTupleTranslator.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/c3a65435_3dfcc115?usp=email : PS1, Line 519: throw new AsterixException(ErrorCode.METADATA_ERROR, SAMPLE_IS_FULL_SCAN); This isn't backward compatible. Do we even need this flag? File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/DatasetStreamStatsOperatorDescriptor.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/0d1525f0_5248d7a7?usp=email : PS1, Line 129: Object totalNum = TaskUtil.get(ESTIMATE_CARDINALITY, ctx); : if (totalNum == null) { : totalNum = totalTupleCount; : } 🤮 File asterixdb/asterix-server/src/test/java/org/apache/asterix/test/server/AbstractExecutionIT.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/84c709fc_24176ec7?usp=email : PS1, Line 62: results_ncservice ? File asterixdb/asterix-server/src/test/java/org/apache/asterix/test/server/NCServiceExecutionIT.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/7b52e836_f885e2cd?usp=email : PS1, Line 138: results_ncservice ? File hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/9a0caac7_cd2fceb2?usp=email : PS1, Line 166: RANDOM_SAMPLE_LEAF_NOT_FOUND ? File hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/IPersistedResourceRegistry.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/9e554be3_2a94b14e?usp=email : PS1, Line 32: extends Serializable ? File hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/HyracksConstants.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/13577fa9_1774cc04?usp=email : PS1, Line 35: public static final String SAMPLE_CARDINALITY = "SAMPLE_CARDINALITY"; : : public static final String SAMPLE_SEED = "SAMPLE_SEED"; Hyracks probably doesn't need to know about samples File hyracks-fullstack/hyracks/hyracks-api/src/main/resources/errormsg/en.properties: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/a9b760f0_3a44a17f?usp=email : PS1, Line 156: 136 = Failed to find a random leaf page with tuples after %1$s attempts ? File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/dataflow/LSMColumnBTreeLocalResource.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/a36ded92_23f8a909?usp=email : PS1, Line 120: int thetaSketchK = getOrDefaultInt(json, "thetaSketchK", ThetaSampler.DEFAULT_K); : int maxSampleLeafAttempts = getOrDefaultInt(json, "maxSampleLeafAttempts", 500); : int sampleLeafDrawBatchSize = getOrDefaultInt(json, "sampleLeafDrawBatchSize", 32768); why are we persisting this info in the Btree local resource? if feels like those should be sampling runtime values that are passed during compilation File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSMBTreeLocalResource.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/ef327bf3_e199c085?usp=email : PS1, Line 179: json.put("thetaSketchK", thetaSketchK); : json.put("maxSampleLeafAttempts", maxSampleLeafAttempts); : json.put("sampleLeafDrawBatchSize", sampleLeafDrawBatchSize); why? File hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMTreeTupleReference.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/ec8d504d_5d4ffaeb?usp=email : PS1, Line 28: isAntimatter Do we need to make this interface change? File hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/IIndex.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/445bce82_a5471fa3?usp=email : PS1, Line 133: thetaSampler use the generic name in the API File hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/ISampler.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366/comment/d9787ee3_29926fd0?usp=email : PS1, Line 26: ISampler Is there a better name for this API? -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21366?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: asterixdb Gerrit-Branch: lumina Gerrit-Change-Id: If5c5b7eac5199b85fc40b722c87ae0bbb73eecbd Gerrit-Change-Number: 21366 Gerrit-PatchSet: 1 Gerrit-Owner: Ritik Raj <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-CC: Murtadha Hubail <[email protected]> Gerrit-Attention: Ritik Raj <[email protected]> Gerrit-Comment-Date: Mon, 22 Jun 2026 13:27:15 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No
