>From Hongyu Shi <[email protected]>: Attention is currently required from: Ian Maxon.
Hongyu Shi has posted comments on this change by Hongyu Shi. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744?usp=email ) Change subject: [WIP]Workload manager with configurable priority-based scheduling - user model changes: yes - storage format changes: no - interface changes: yes ...................................................................... Patch Set 4: (18 comments) File asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryResultApiServlet.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/cded632d_2da60382?usp=email : PS3, Line 249: > revert Done File asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/79d134d0_da362c00?usp=email : PS3, Line 25: SchedulerConfigMetadataEntity > the linter will freak out if you use star imports (and they're generally bad) Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/af8a95ef_37b5794e?usp=email : PS3, Line 6582: return ccs.getJobManager() instanceof WorkloadManager; > adding a method that returns an enum or something depending on whether or not > the implementation sup […] Done File asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/840a0704_64314b24?usp=email : PS3, Line 492: cc.getExecutor().submit(() -> { > this can't be done like this. […] Done File asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/8d17f39c_4cec6a17?usp=email : PS3, Line 485: if (cc.getJobManager() instanceof WorkloadManager) { > no instanceof here, and if you have that method that hides the enum check or > whatever, make it a sta […] Done File asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/SchedulerConfigUtil.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/8012e1ec_1336762b?usp=email : PS3, Line 36: //--------------------------------------- Scheduler config --------------------------------------// : : /* : Example of full-text config create statement : CREATE SCHEDULER CONFIG s_config_1 { : "defaultPriority": 1, : "shortMemoryPercent": 10.0, : "shortCPUQuota": 20, : "queryGroup": : [ : { : "priority": 4, : "grouplist": [ "ui"] : }, : : { : "priority" : 6, : "grouplist": ["analytics"] : : }, : : { : "priority": 8, : "grouplist": [ "management" ] : } : ] : }; : : UPSERT QGROUP INTO s_config_1 {[ : { : "priority": 10, : "name": "ingest" : }, : : { : "priority": 6, : "name": "management" : } : ]}; : : DELETE GROUP FROM s_config_1 {[ : "analytics", "ingest" : ]}; : : > delete this comment Done File asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataTransactionContext.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/0eeb0746_f7e86dca?usp=email : PS3, Line 178: public void addSchedulerConfig(SchedulerConfigMetadataEntity configMetadataEntity) { : // TODO : } > is this still todo? Done File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/scheduler/SchedulerConfigRecordDescriptor.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/a6a8983c_86b03a9b?usp=email : PS3, Line 39: //TODO(DB): database name should not be null > ? Done File hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/e109d12a_ed0f8fae?usp=email : PS3, Line 258: /* jobManager = new WorkloadManager(ccConfig, this, jobCapacityController); */ > ? Done File hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/job/WorkloadManager.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/16b95ecc_f624b211?usp=email : PS3, Line 170: logJobCapacity(jobRun, "released", Level.DEBUG); > might want this at TRACE Done File hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/CapacityControllerGuard.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/4b09aa35_c5ebf370?usp=email : PS3, Line 208: /* TODO: dynamically adjust resources limits from different categories */ : /* CPU Quota : short common : limit: 6 6 : used: 4 0 : available: 2 6 : -> set short to 3 : since 2 < 6 - 3 = 3 : limit: 3 9 : used: 4 0 : available: 0 8 : -> four used CPU Quota for short job is released : available: : 3 9 : */ > remove this comment? Done File hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/CompositeQueue.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/92b6a7f7_66e77914?usp=email : PS3, Line 38: //private final IJobCapacityController jobCapacityController; > ? Done File hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/DedicatedJobQueue.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/a99ac3fb_50205e13?usp=email : PS3, Line 89: // Removes the selected job. > remove Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/a12fa497_a6aa40eb?usp=email : PS3, Line 90: /* TODO: More bookkeeping for short jobs*/ > what would this bookkeeping be? Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/b8d94b3d_2c40a1f4?usp=email : PS3, Line 100: / Fails the job. > remove Done File hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/scheduler/DefaultJobQueue.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/0506fe78_50e52f67?usp=email : PS3, Line 51: //private final Map<JobId, JobRun> memoryQueue = new LinkedHashMap<>(); > ? Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/153404f4_eee18a47?usp=email : PS3, Line 54: //private final IJobCapacityController jobCapacityController; > ? Done https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744/comment/5ffb4301_097bcab6?usp=email : PS3, Line 140: if (ratio <= 0.05) { : return queues.get(1); : } else if (ratio <= 0.10) { : return queues.get(2); : } else if (ratio <= 0.15) { : return queues.get(3); : } else if (ratio <= 0.20) { : return queues.get(4); : } else if (ratio <= 0.25) { : return queues.get(5); : } else if (ratio <= 0.40) { : return queues.get(6); : } else if (ratio <= 0.55) { : return queues.get(7); : } else if (ratio <= 0.70) { : return queues.get(8); : } else if (ratio <= 0.85) { : return queues.get(9); : } : return queues.get(10); > maybe make this a switch/case? Done -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20744?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: comment Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I52748f8e62c29e3595898f765d00fd33320316c3 Gerrit-Change-Number: 20744 Gerrit-PatchSet: 4 Gerrit-Owner: Hongyu Shi <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-CC: Ian Maxon <[email protected]> Gerrit-Attention: Ian Maxon <[email protected]> Gerrit-Comment-Date: Fri, 16 Jan 2026 17:55:40 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Comment-In-Reply-To: Ian Maxon <[email protected]>
