viirya commented on PR #7692:
URL: 
https://github.com/apache/datafusion-comet/pull/7692#issuecomment-7020421482

   Benchmark of `collect` for the plans this PR admits, comparing Spark, 
ordinary Comet execution and Comet local execution.
   
   Setup: Apple M4 Max, `local[8]`, JDK 17, Spark 4.1, native library built 
with `--release -Ctarget-cpu=native`, 4 GB off-heap. Table: 8M rows (`id` long, 
`a` int, `b` double, `s` short string) in 64 Parquet files. Each case runs 10 
iterations. Times are averages in ms; best times are in parentheses.
   
   | Case | Partitions | Rows returned | Spark | Comet | Comet local |
   |---|---|---|---|---|---|
   | All rows | 8 | 8M | 1414 (1225) | 1178 (1089) | 1254 (1056) |
   | All rows | 320 | 8M | 1164 (1069) | 1279 (1129) | 1235 (1026) |
   | Selective filter | 8 | 8K | 62 (57) | 34 (33) | 33 (31) |
   | Selective filter | 320 | 8K | 93 (87) | 74 (71) | 107 (101) |
   | All rows, half in one file | 8 | 8M | 1791 (1590) | 1574 (1397) | 1124 
(1027) |
   | Range, all rows | 8 | 8M | 494 (396) | 541 (422) | 439 (237) |
   | Range, selective filter | 8 | 8K | 7 (7) | 17 (9) | 16 (9) |
   
   A second run showed the same pattern.
   
   - When files are uniform, local execution is within noise of ordinary Comet. 
These plans have no exchange, so the only work it removes is per-task overhead 
and the encoding of task results.
   - When one partition holds half the rows, local execution is about 30% 
faster than ordinary Comet. Ordinary execution encodes and compresses that 
partition's result in one task and decodes it on the driver, all on one thread. 
Local execution copies the rows straight into the driver's result slot.
   - Many tiny partitions are slower: with 320 partitions returning 8K rows in 
all, local execution takes about 45% longer than ordinary Comet, and the result 
was stable across both runs.
     - The cause is the rule from #6883 that the next partition starts only 
when the first one is delivered. A partition that has finished but is not first 
in line keeps its slot. So one slow partition idles the slots of the finished 
ones behind it.
     - Per-partition timing of this case shows that planning and starting a 
partition takes about 50 µs and running it a median of 340 µs (up to 2.9 ms). 
Finished partitions then waited a mean of 1.4 ms for delivery. On average only 
2.9 of the 8 slots were busy.
     - In an experiment where a finished partition gave up its slot, the case 
dropped from 104 ms to 44 ms (ordinary Comet: 83 ms), with 7.4 slots busy on 
average. The experiment set no bound on how many finished partitions wait for 
delivery, and it got as high as 57. #7693 adds that bound and fixes the 
scheduling separately, since the rule also applies to ranges. With #7693 
applied on top of this PR, the case takes 50 ms against 106 ms for ordinary 
Comet in the same run.
   
   The benchmark (`CometLocalExecutionBenchmark`) is not part of this PR.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to