Re: [PR] Refactor parquet row filter setup [datafusion]
xudong963 merged PR #22191: URL: https://github.com/apache/datafusion/pull/22191 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
xudong963 commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4469095886 > Yup agreed happy for that to happen as a follwup 😄 Thanks! Follow-up is on the way -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
xudong963 commented on code in PR #22191:
URL: https://github.com/apache/datafusion/pull/22191#discussion_r3253944588
##
datafusion/datasource-parquet/src/opener.rs:
##
@@ -1280,6 +1230,113 @@ impl RowGroupsPrunedParquetOpen {
}
}
+/// Builds row filters for decoder runs.
+///
+/// A [`RowFilter`] must be owned by a decoder, so scans split across multiple
+/// decoder runs need a fresh filter for each run that evaluates row
predicates.
+struct RowFilterGenerator<'a> {
+predicate: Option<&'a Arc>,
+physical_file_schema: &'a SchemaRef,
+file_metadata: &'a ParquetMetaData,
+reorder_predicates: bool,
+file_metrics: &'a ParquetFileMetrics,
+first_row_filter: Option,
+}
+
+impl<'a> RowFilterGenerator<'a> {
+fn new(
+prepared: &'a PreparedParquetOpen,
+file_metadata: &'a ParquetMetaData,
+) -> Self {
+let predicate = prepared
+.pushdown_filters
+.then_some(prepared.predicate.as_ref())
+.flatten();
+
+let mut generator = Self {
+predicate,
+physical_file_schema: &prepared.physical_file_schema,
+file_metadata,
+reorder_predicates: prepared.reorder_predicates,
+file_metrics: &prepared.file_metrics,
+first_row_filter: None,
+};
+generator.first_row_filter = generator.build_row_filter();
+generator
+}
+
+fn has_row_filter(&self) -> bool {
+self.first_row_filter.is_some()
+}
+
+fn next_filter(&mut self) -> Option {
+self.first_row_filter
+.take()
+.or_else(|| self.build_row_filter())
+}
+
+fn build_row_filter(&self) -> Option {
+let predicate = self.predicate?;
+match row_filter::build_row_filter(
+predicate,
+self.physical_file_schema,
+self.file_metadata,
+self.reorder_predicates,
+self.file_metrics,
+) {
+Ok(Some(filter)) => Some(filter),
+Ok(None) => None,
+Err(e) => {
+debug!("Ignoring error building row filter for
'{predicate:?}': {e}");
+None
+}
+}
+}
+}
+
+fn prepare_access_plan(
+plan: ParquetAccessPlan,
+rg_metadata: &[RowGroupMetaData],
+file_metadata: &ParquetMetaData,
+reverse_row_groups: bool,
+) -> Result {
+let mut prepared_access_plan = plan.prepare(rg_metadata)?;
+if reverse_row_groups {
+prepared_access_plan = prepared_access_plan.reverse(file_metadata)?;
+}
+Ok(prepared_access_plan)
+}
+
+struct DecoderBuilderConfig<'a> {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangb commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4463158979 Yup agreed happy for that to happen as a follwup 😄 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
alamb commented on code in PR #22191:
URL: https://github.com/apache/datafusion/pull/22191#discussion_r3248847301
##
datafusion/datasource-parquet/src/opener.rs:
##
@@ -1280,6 +1230,113 @@ impl RowGroupsPrunedParquetOpen {
}
}
+/// Builds row filters for decoder runs.
+///
+/// A [`RowFilter`] must be owned by a decoder, so scans split across multiple
+/// decoder runs need a fresh filter for each run that evaluates row
predicates.
+struct RowFilterGenerator<'a> {
+predicate: Option<&'a Arc>,
+physical_file_schema: &'a SchemaRef,
+file_metadata: &'a ParquetMetaData,
+reorder_predicates: bool,
+file_metrics: &'a ParquetFileMetrics,
+first_row_filter: Option,
+}
+
+impl<'a> RowFilterGenerator<'a> {
+fn new(
+prepared: &'a PreparedParquetOpen,
+file_metadata: &'a ParquetMetaData,
+) -> Self {
+let predicate = prepared
+.pushdown_filters
+.then_some(prepared.predicate.as_ref())
+.flatten();
+
+let mut generator = Self {
+predicate,
+physical_file_schema: &prepared.physical_file_schema,
+file_metadata,
+reorder_predicates: prepared.reorder_predicates,
+file_metrics: &prepared.file_metrics,
+first_row_filter: None,
+};
+generator.first_row_filter = generator.build_row_filter();
+generator
+}
+
+fn has_row_filter(&self) -> bool {
+self.first_row_filter.is_some()
+}
+
+fn next_filter(&mut self) -> Option {
+self.first_row_filter
+.take()
+.or_else(|| self.build_row_filter())
+}
+
+fn build_row_filter(&self) -> Option {
+let predicate = self.predicate?;
+match row_filter::build_row_filter(
+predicate,
+self.physical_file_schema,
+self.file_metadata,
+self.reorder_predicates,
+self.file_metrics,
+) {
+Ok(Some(filter)) => Some(filter),
+Ok(None) => None,
+Err(e) => {
+debug!("Ignoring error building row filter for
'{predicate:?}': {e}");
+None
+}
+}
+}
+}
+
+fn prepare_access_plan(
+plan: ParquetAccessPlan,
+rg_metadata: &[RowGroupMetaData],
+file_metadata: &ParquetMetaData,
+reverse_row_groups: bool,
+) -> Result {
+let mut prepared_access_plan = plan.prepare(rg_metadata)?;
+if reverse_row_groups {
+prepared_access_plan = prepared_access_plan.reverse(file_metadata)?;
+}
+Ok(prepared_access_plan)
+}
+
+struct DecoderBuilderConfig<'a> {
Review Comment:
It would be nice to add some comments here explaining what this represnets
(namely the state needed to build the ParquetPushDecoder)
##
datafusion/datasource-parquet/src/opener.rs:
##
@@ -1280,6 +1230,113 @@ impl RowGroupsPrunedParquetOpen {
}
}
+/// Builds row filters for decoder runs.
+///
+/// A [`RowFilter`] must be owned by a decoder, so scans split across multiple
+/// decoder runs need a fresh filter for each run that evaluates row
predicates.
+struct RowFilterGenerator<'a> {
+predicate: Option<&'a Arc>,
+physical_file_schema: &'a SchemaRef,
+file_metadata: &'a ParquetMetaData,
+reorder_predicates: bool,
+file_metrics: &'a ParquetFileMetrics,
+first_row_filter: Option,
+}
+
+impl<'a> RowFilterGenerator<'a> {
+fn new(
+prepared: &'a PreparedParquetOpen,
+file_metadata: &'a ParquetMetaData,
+) -> Self {
+let predicate = prepared
+.pushdown_filters
+.then_some(prepared.predicate.as_ref())
+.flatten();
+
+let mut generator = Self {
+predicate,
+physical_file_schema: &prepared.physical_file_schema,
+file_metadata,
+reorder_predicates: prepared.reorder_predicates,
+file_metrics: &prepared.file_metrics,
+first_row_filter: None,
+};
+generator.first_row_filter = generator.build_row_filter();
+generator
+}
+
+fn has_row_filter(&self) -> bool {
+self.first_row_filter.is_some()
+}
+
+fn next_filter(&mut self) -> Option {
+self.first_row_filter
+.take()
+.or_else(|| self.build_row_filter())
+}
+
+fn build_row_filter(&self) -> Option {
+let predicate = self.predicate?;
+match row_filter::build_row_filter(
+predicate,
+self.physical_file_schema,
+self.file_metadata,
+self.reorder_predicates,
+self.file_metrics,
+) {
+Ok(Some(filter)) => Some(filter),
+Ok(None) => None,
+Err(e) => {
+debug!("Ignoring error building row filter for
'{predicate:?}': {e}");
+None
+}
+}
+}
+}
+
+fn prepare_access_plan(
+plan: ParquetAcces
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangb commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4457658614 @xudong963 I left you some suggestions in https://github.com/xudong963/arrow-datafusion/pull/6. It goes a bit further but I think leaves things in a nicer state and is related enough to the other refactors to make sense to bundle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4457060669 🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456983296) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Details ``` Comparing HEAD and xudong963_prune-page-index-followup Benchmark clickbench_partitioned.json ┏━━━┳━━━┳━━━┳━━━┓ ┃ Query ┃ HEAD ┃ xudong963_prune-page-index-followup ┃Change ┃ ┡━━━╇━━━╇━━━╇━━━┩ │ QQuery 0 │ 1.19 / 4.61 ±6.78 / 18.16 ms │ 1.23 / 4.64 ±6.75 / 18.14 ms │ no change │ │ QQuery 1 │12.44 / 12.58 ±0.08 / 12.69 ms │12.26 / 12.62 ±0.22 / 12.89 ms │ no change │ │ QQuery 2 │35.26 / 35.59 ±0.26 / 35.91 ms │35.57 / 36.07 ±0.45 / 36.62 ms │ no change │ │ QQuery 3 │30.77 / 31.50 ±0.58 / 32.19 ms │30.60 / 30.98 ±0.35 / 31.60 ms │ no change │ │ QQuery 4 │236.49 / 262.19 ±14.05 / 275.12 ms │ 227.56 / 232.32 ±4.16 / 239.13 ms │ +1.13x faster │ │ QQuery 5 │ 291.22 / 301.17 ±6.32 / 307.22 ms │ 308.72 / 311.36 ±1.42 / 312.87 ms │ no change │ │ QQuery 6 │ 7.09 / 7.75 ±0.40 / 8.31 ms │ 7.01 / 7.79 ±0.40 / 8.17 ms │ no change │ │ QQuery 7 │13.89 / 14.03 ±0.10 / 14.20 ms │14.59 / 14.71 ±0.13 / 14.91 ms │ no change │ │ QQuery 8 │ 310.69 / 313.36 ±2.45 / 317.94 ms │329.08 / 340.94 ±10.31 / 354.67 ms │ 1.09x slower │ │ QQuery 9 │ 447.64 / 452.89 ±3.42 / 457.79 ms │449.21 / 460.04 ±11.98 / 482.13 ms │ no change │ │ QQuery 10 │68.37 / 69.38 ±0.74 / 70.39 ms │68.88 / 74.95 ±6.67 / 87.52 ms │ 1.08x slower │ │ QQuery 11 │78.96 / 79.86 ±0.67 / 80.95 ms │80.16 / 80.86 ±0.70 / 82.18 ms │ no change │ │ QQuery 12 │ 268.17 / 274.95 ±6.38 / 285.33 ms │ 271.65 / 276.58 ±4.14 / 284.20 ms │ no change │ │ QQuery 13 │371.70 / 384.39 ±10.08 / 400.95 ms │ 383.60 / 392.53 ±6.71 / 400.53 ms │ no change │ │ QQuery 14 │ 274.60 / 280.05 ±3.54 / 283.50 ms │ 275.97 / 281.01 ±4.39 / 287.84 ms │ no change
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4457052632 🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456985478) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Details ``` Comparing HEAD and xudong963_prune-page-index-followup Benchmark clickbench_partitioned.json ┏━━━┳━━━┳━━━┳━━━┓ ┃ Query ┃ HEAD ┃ xudong963_prune-page-index-followup ┃Change ┃ ┡━━━╇━━━╇━━━╇━━━┩ │ QQuery 0 │ 1.22 / 4.70 ±6.81 / 18.32 ms │ 1.23 / 4.69 ±6.83 / 18.35 ms │ no change │ │ QQuery 1 │12.65 / 13.05 ±0.29 / 13.55 ms │13.14 / 13.51 ±0.21 / 13.74 ms │ no change │ │ QQuery 2 │36.14 / 36.54 ±0.38 / 37.18 ms │35.95 / 36.24 ±0.27 / 36.72 ms │ no change │ │ QQuery 3 │31.07 / 31.85 ±0.58 / 32.75 ms │31.44 / 31.66 ±0.18 / 31.89 ms │ no change │ │ QQuery 4 │ 233.84 / 236.63 ±1.71 / 238.45 ms │ 234.60 / 239.82 ±6.61 / 251.68 ms │ no change │ │ QQuery 5 │ 278.23 / 280.45 ±2.38 / 284.76 ms │ 280.00 / 281.96 ±1.66 / 284.89 ms │ no change │ │ QQuery 6 │ 6.75 / 7.22 ±0.37 / 7.69 ms │ 6.54 / 7.52 ±0.57 / 8.27 ms │ no change │ │ QQuery 7 │13.76 / 13.91 ±0.15 / 14.18 ms │14.39 / 15.44 ±2.07 / 19.58 ms │ 1.11x slower │ │ QQuery 8 │ 315.23 / 317.92 ±2.07 / 320.72 ms │ 317.76 / 325.40 ±9.42 / 343.67 ms │ no change │ │ QQuery 9 │ 449.26 / 456.35 ±8.24 / 471.88 ms │ 458.81 / 464.95 ±7.35 / 474.45 ms │ no change │ │ QQuery 10 │69.25 / 69.74 ±0.50 / 70.70 ms │69.35 / 74.37 ±7.67 / 89.65 ms │ 1.07x slower │ │ QQuery 11 │79.94 / 80.36 ±0.38 / 80.84 ms │81.61 / 83.79 ±2.95 / 89.64 ms │ no change │ │ QQuery 12 │ 274.33 / 276.35 ±1.60 / 278.96 ms │275.42 / 285.32 ±10.56 / 305.25 ms │ no change │ │ QQuery 13 │ 382.11 / 391.40 ±9.49 / 406.39 ms │ 385.33 / 392.87 ±5.06 / 400.97 ms │ no change │ │ QQuery 14 │ 281.92 / 284.14 ±1.87 / 286.70 ms │ 281.40 / 285.06 ±4.01 / 292.86 ms │ no change
Re: [PR] Refactor parquet row filter setup [datafusion]
xudong963 commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4457049452 Benchmark follow-up: I reran after the initial result showed 4 slower queries. In the rerun, Q19/Q24/Q26 were no longer flagged slower, only Q41 was slower with high branch-side variance, and total time was effectively unchanged (20310.89ms base vs 20314.91ms branch). TPCH/TPCDS also do not show a consistent regression. Given this PR only refactors setup code and preserves the decoder/page-pruning behavior, I think the clickbench signal is noise rather than a real regression. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4457044653 🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456985478) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Details ``` Comparing HEAD and xudong963_prune-page-index-followup Benchmark tpcds_sf1.json ┏━━━┳━━━┳━━━┳━━━┓ ┃ Query ┃ HEAD ┃ xudong963_prune-page-index-followup ┃Change ┃ ┡━━━╇━━━╇━━━╇━━━┩ │ QQuery 1 │ 6.31 / 6.87 ±0.97 / 8.81 ms │ 6.20 / 6.69 ±0.89 / 8.46 ms │ no change │ │ QQuery 2 │80.94 / 81.48 ±0.34 / 81.93 ms │81.14 / 81.54 ±0.24 / 81.83 ms │ no change │ │ QQuery 3 │28.91 / 29.17 ±0.26 / 29.67 ms │28.62 / 29.11 ±0.44 / 29.89 ms │ no change │ │ QQuery 4 │ 509.85 / 515.59 ±5.68 / 522.80 ms │ 510.37 / 513.53 ±2.31 / 516.29 ms │ no change │ │ QQuery 5 │52.84 / 53.23 ±0.32 / 53.73 ms │52.45 / 52.94 ±0.37 / 53.52 ms │ no change │ │ QQuery 6 │35.77 / 35.96 ±0.13 / 36.18 ms │34.82 / 35.71 ±0.52 / 36.27 ms │ no change │ │ QQuery 7 │ 109.22 / 109.98 ±0.53 / 110.67 ms │ 109.06 / 111.19 ±2.42 / 114.58 ms │ no change │ │ QQuery 8 │39.62 / 39.84 ±0.20 / 40.21 ms │39.22 / 39.51 ±0.16 / 39.70 ms │ no change │ │ QQuery 9 │51.52 / 53.62 ±2.04 / 57.42 ms │52.88 / 54.39 ±1.14 / 56.20 ms │ no change │ │ QQuery 10 │81.11 / 82.24 ±1.12 / 84.29 ms │80.88 / 81.75 ±1.50 / 84.75 ms │ no change │ │ QQuery 11 │ 322.13 / 325.84 ±3.35 / 331.88 ms │ 310.11 / 316.68 ±4.69 / 322.27 ms │ no change │ │ QQuery 12 │28.88 / 29.05 ±0.14 / 29.29 ms │28.27 / 28.71 ±0.46 / 29.34 ms │ no change │ │ QQuery 13 │ 128.84 / 129.46 ±0.50 / 130.14 ms │ 128.86 / 129.54 ±0.50 / 130.35 ms │ no change │ │ QQuery 14 │ 513.81 / 516.78 ±2.16 / 518.93 ms │ 512.44 / 513.82 ±0.96 / 514.87 ms │ no change │ │ QQuery 15 │60.25 / 61.55 ±0.90 / 62.92 ms │61.39 / 62.47 ±0.88 / 63.67 ms │ no change │ │ QQuery
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4457047319 🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456976277) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Details ``` Comparing HEAD and xudong963_prune-page-index-followup Benchmark clickbench_partitioned.json ┏━━━┳━━━┳━━━┳━━━┓ ┃ Query ┃ HEAD ┃ xudong963_prune-page-index-followup ┃Change ┃ ┡━━━╇━━━╇━━━╇━━━┩ │ QQuery 0 │ 1.25 / 4.76 ±6.91 / 18.57 ms │ 1.23 / 4.81 ±7.01 / 18.82 ms │ no change │ │ QQuery 1 │13.15 / 13.58 ±0.41 / 14.35 ms │12.68 / 13.44 ±0.57 / 14.40 ms │ no change │ │ QQuery 2 │37.07 / 37.42 ±0.26 / 37.82 ms │36.48 / 36.91 ±0.33 / 37.29 ms │ no change │ │ QQuery 3 │31.75 / 32.32 ±0.45 / 33.07 ms │31.52 / 31.83 ±0.34 / 32.48 ms │ no change │ │ QQuery 4 │ 249.20 / 250.07 ±0.88 / 251.48 ms │ 248.21 / 251.20 ±2.10 / 253.87 ms │ no change │ │ QQuery 5 │ 284.49 / 290.03 ±3.48 / 294.81 ms │ 285.92 / 287.58 ±1.85 / 290.99 ms │ no change │ │ QQuery 6 │ 6.96 / 8.26 ±1.64 / 11.47 ms │ 6.81 / 7.18 ±0.30 / 7.62 ms │ +1.15x faster │ │ QQuery 7 │14.58 / 14.76 ±0.17 / 15.05 ms │14.15 / 14.21 ±0.06 / 14.31 ms │ no change │ │ QQuery 8 │ 329.05 / 334.65 ±4.45 / 341.06 ms │ 328.54 / 331.27 ±1.82 / 334.21 ms │ no change │ │ QQuery 9 │ 469.59 / 479.45 ±5.74 / 485.98 ms │ 460.69 / 473.00 ±7.94 / 480.02 ms │ no change │ │ QQuery 10 │70.27 / 73.29 ±3.32 / 79.41 ms │71.68 / 72.91 ±0.68 / 73.51 ms │ no change │ │ QQuery 11 │81.49 / 82.28 ±0.62 / 83.20 ms │80.71 / 81.46 ±0.51 / 82.25 ms │ no change │ │ QQuery 12 │ 290.74 / 293.88 ±2.61 / 297.56 ms │ 285.89 / 291.42 ±4.37 / 299.23 ms │ no change │ │ QQuery 13 │ 395.12 / 412.31 ±9.78 / 424.08 ms │ 391.90 / 400.95 ±5.02 / 407.31 ms │ no change │ │ QQuery 14 │ 290.16 / 292.20 ±2.29 / 296.52 ms │ 290.02 / 298.06 ±6.28 / 307.96 ms │ no change
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4457031030 🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456985478) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Details ``` Comparing HEAD and xudong963_prune-page-index-followup Benchmark tpch_sf1.json ┏━━━┳┳━┳━━━┓ ┃ Query ┃ HEAD ┃ xudong963_prune-page-index-followup ┃Change ┃ ┡━━━╇╇━╇━━━┩ │ QQuery 1 │ 39.62 / 40.43 ±0.88 / 41.79 ms │ 39.64 / 40.39 ±1.04 / 42.45 ms │ no change │ │ QQuery 2 │ 21.13 / 21.63 ±0.51 / 22.55 ms │ 21.14 / 21.29 ±0.21 / 21.71 ms │ no change │ │ QQuery 3 │ 35.79 / 38.05 ±1.47 / 39.84 ms │ 38.74 / 39.08 ±0.29 / 39.48 ms │ no change │ │ QQuery 4 │ 18.08 / 18.20 ±0.10 / 18.37 ms │ 17.70 / 18.18 ±0.64 / 19.46 ms │ no change │ │ QQuery 5 │ 43.44 / 44.09 ±0.39 / 44.40 ms │ 44.59 / 47.28 ±2.65 / 52.00 ms │ 1.07x slower │ │ QQuery 6 │ 16.70 / 16.90 ±0.12 / 17.05 ms │ 17.09 / 17.17 ±0.07 / 17.28 ms │ no change │ │ QQuery 7 │ 49.55 / 52.20 ±1.59 / 53.71 ms │ 50.59 / 51.84 ±1.12 / 53.67 ms │ no change │ │ QQuery 8 │ 46.41 / 47.12 ±0.47 / 47.72 ms │ 46.66 / 46.79 ±0.12 / 47.00 ms │ no change │ │ QQuery 9 │ 50.95 / 51.76 ±0.56 / 52.32 ms │ 52.31 / 52.59 ±0.22 / 52.99 ms │ no change │ │ QQuery 10 │ 63.28 / 63.59 ±0.19 / 63.85 ms │ 63.68 / 65.24 ±1.25 / 67.16 ms │ no change │ │ QQuery 11 │ 13.56 / 14.00 ±0.50 / 14.93 ms │ 14.11 / 14.27 ±0.16 / 14.54 ms │ no change │ │ QQuery 12 │ 25.14 / 25.41 ±0.21 / 25.74 ms │ 25.51 / 25.86 ±0.27 / 26.18 ms │ no change │ │ QQuery 13 │ 35.80 / 37.74 ±2.55 / 42.77 ms │ 36.94 / 37.85 ±0.66 / 39.00 ms │ no change │ │ QQuery 14 │ 25.82 / 26.06 ±0.22 / 26.33 ms │ 25.64 / 26.27 ±0.56 / 27.05 ms │ no change │ │ QQuery 15 │ 31.74 / 32.41 ±0.86 / 34.11 ms │ 31.26 / 31.69 ±0.26 / 32.02 ms │ no change │ │ QQuery 16 │ 15.08 / 15.27 ±0.24 / 15.73 ms │ 14.98 / 15.06 ±0.09 / 15.23 ms │ no change │ │ QQuery 17 │ 84.03 / 87.03 ±1.59 / 88.49 ms │ 74.71 / 76.06
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4457000689 🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456983296) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4456983296-116-x5mnx 6.12.68+ #1 SMP Wed Apr 1 02:23:28 UTC 2026 aarch64 GNU/Linux` CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Comparing xudong963/prune-page-index-followup (d6ac2fb2655d4710a62847a84495b763f06fc388) to 3f501f4 (merge-base) [diff](https://github.com/apache/datafusion/compare/3f501f4b9e6bf8616556fbf6c832da7e364680a4..d6ac2fb2655d4710a62847a84495b763f06fc388) using: clickbench_partitioned Results will be posted here when complete --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456998660 🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456985478) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4456985478-118-vj865 6.12.68+ #1 SMP Wed Apr 1 02:23:28 UTC 2026 aarch64 GNU/Linux` CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Comparing xudong963/prune-page-index-followup (d6ac2fb2655d4710a62847a84495b763f06fc388) to 3f501f4 (merge-base) [diff](https://github.com/apache/datafusion/compare/3f501f4b9e6bf8616556fbf6c832da7e364680a4..d6ac2fb2655d4710a62847a84495b763f06fc388) using: tpcds Results will be posted here when complete --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456994987 🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456985478) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4456985478-117-8b72x 6.12.68+ #1 SMP Wed Apr 1 02:23:28 UTC 2026 aarch64 GNU/Linux` CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Comparing xudong963/prune-page-index-followup (d6ac2fb2655d4710a62847a84495b763f06fc388) to 3f501f4 (merge-base) [diff](https://github.com/apache/datafusion/compare/3f501f4b9e6bf8616556fbf6c832da7e364680a4..d6ac2fb2655d4710a62847a84495b763f06fc388) using: clickbench_partitioned Results will be posted here when complete --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456992539 🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456985478) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4456985478-119-t8fbz 6.12.68+ #1 SMP Wed Apr 1 02:23:28 UTC 2026 aarch64 GNU/Linux` CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Comparing xudong963/prune-page-index-followup (d6ac2fb2655d4710a62847a84495b763f06fc388) to 3f501f4 (merge-base) [diff](https://github.com/apache/datafusion/compare/3f501f4b9e6bf8616556fbf6c832da7e364680a4..d6ac2fb2655d4710a62847a84495b763f06fc388) using: tpch Results will be posted here when complete --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456985554 🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456976277) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4456976277-115-99vv2 6.12.68+ #1 SMP Wed Apr 1 02:23:28 UTC 2026 aarch64 GNU/Linux` CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Comparing xudong963/prune-page-index-followup (d6ac2fb2655d4710a62847a84495b763f06fc388) to 3f501f4 (merge-base) [diff](https://github.com/apache/datafusion/compare/3f501f4b9e6bf8616556fbf6c832da7e364680a4..d6ac2fb2655d4710a62847a84495b763f06fc388) using: clickbench_partitioned Results will be posted here when complete --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
xudong963 commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456985478 run benchmarks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
xudong963 commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456983296 run benchmarks clickbench_partitioned -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
xudong963 commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456976277 run benchmark clickbench_partitioned -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456815818 🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456691699) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Details ``` Comparing HEAD and xudong963_prune-page-index-followup Benchmark clickbench_partitioned.json ┏━━━┳━━━┳━━━┳━━━┓ ┃ Query ┃ HEAD ┃ xudong963_prune-page-index-followup ┃Change ┃ ┡━━━╇━━━╇━━━╇━━━┩ │ QQuery 0 │ 1.20 / 4.60 ±6.72 / 18.04 ms │ 1.20 / 4.52 ±6.59 / 17.69 ms │ no change │ │ QQuery 1 │12.95 / 13.15 ±0.15 / 13.34 ms │12.52 / 12.86 ±0.48 / 13.79 ms │ no change │ │ QQuery 2 │35.42 / 35.73 ±0.27 / 36.11 ms │35.31 / 35.50 ±0.19 / 35.80 ms │ no change │ │ QQuery 3 │30.60 / 31.26 ±0.53 / 32.20 ms │30.41 / 30.80 ±0.23 / 31.08 ms │ no change │ │ QQuery 4 │ 226.79 / 231.17 ±2.91 / 235.36 ms │ 228.51 / 232.44 ±3.89 / 239.43 ms │ no change │ │ QQuery 5 │ 274.58 / 277.09 ±2.29 / 281.11 ms │ 272.98 / 275.51 ±1.73 / 278.25 ms │ no change │ │ QQuery 6 │ 6.10 / 6.60 ±0.44 / 7.40 ms │ 6.18 / 6.65 ±0.45 / 7.34 ms │ no change │ │ QQuery 7 │14.13 / 14.32 ±0.10 / 14.41 ms │13.35 / 13.41 ±0.03 / 13.44 ms │ +1.07x faster │ │ QQuery 8 │ 310.52 / 312.56 ±2.12 / 315.56 ms │ 307.80 / 312.06 ±3.52 / 318.18 ms │ no change │ │ QQuery 9 │ 437.26 / 443.84 ±4.76 / 448.22 ms │ 438.29 / 442.48 ±3.04 / 447.44 ms │ no change │ │ QQuery 10 │68.73 / 69.66 ±0.82 / 71.08 ms │67.85 / 68.69 ±0.63 / 69.64 ms │ no change │ │ QQuery 11 │80.00 / 81.59 ±2.33 / 86.22 ms │78.89 / 79.85 ±0.91 / 81.27 ms │ no change │ │ QQuery 12 │ 270.22 / 274.08 ±2.74 / 277.65 ms │ 266.75 / 271.47 ±4.68 / 278.97 ms │ no change │ │ QQuery 13 │ 379.92 / 388.80 ±8.98 / 405.82 ms │ 374.40 / 381.08 ±4.74 / 388.76 ms │ no change │ │ QQuery 14 │ 275.41 / 279.30 ±6.20 / 291.62 ms │ 273.33 / 275.96 ±2.26 / 279.58 ms │ no change
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456805515 🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456691699) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Details ``` Comparing HEAD and xudong963_prune-page-index-followup Benchmark tpcds_sf1.json ┏━━━┳━━━┳━━━┳━━━┓ ┃ Query ┃ HEAD ┃ xudong963_prune-page-index-followup ┃Change ┃ ┡━━━╇━━━╇━━━╇━━━┩ │ QQuery 1 │ 6.56 / 7.06 ±0.86 / 8.78 ms │ 6.41 / 6.95 ±0.86 / 8.65 ms │ no change │ │ QQuery 2 │82.61 / 82.90 ±0.35 / 83.51 ms │82.35 / 82.84 ±0.30 / 83.14 ms │ no change │ │ QQuery 3 │29.82 / 29.99 ±0.12 / 30.17 ms │29.92 / 30.11 ±0.11 / 30.24 ms │ no change │ │ QQuery 4 │ 537.38 / 542.23 ±4.22 / 549.34 ms │ 534.72 / 543.04 ±7.20 / 552.91 ms │ no change │ │ QQuery 5 │53.30 / 53.98 ±0.39 / 54.47 ms │54.29 / 55.04 ±0.45 / 55.68 ms │ no change │ │ QQuery 6 │36.33 / 36.70 ±0.30 / 37.16 ms │37.16 / 37.54 ±0.34 / 37.96 ms │ no change │ │ QQuery 7 │ 111.39 / 113.27 ±1.85 / 116.72 ms │ 112.04 / 114.10 ±2.47 / 118.82 ms │ no change │ │ QQuery 8 │39.77 / 40.02 ±0.22 / 40.28 ms │40.29 / 40.57 ±0.27 / 41.05 ms │ no change │ │ QQuery 9 │55.51 / 56.62 ±1.09 / 58.39 ms │54.41 / 56.42 ±1.84 / 59.57 ms │ no change │ │ QQuery 10 │81.76 / 83.18 ±1.48 / 85.97 ms │82.52 / 84.01 ±1.72 / 87.39 ms │ no change │ │ QQuery 11 │ 332.86 / 336.60 ±3.78 / 343.32 ms │ 339.14 / 343.37 ±3.87 / 350.64 ms │ no change │ │ QQuery 12 │29.85 / 30.64 ±0.66 / 31.56 ms │30.05 / 30.40 ±0.39 / 31.03 ms │ no change │ │ QQuery 13 │ 130.25 / 131.03 ±0.61 / 132.09 ms │ 132.28 / 132.46 ±0.15 / 132.65 ms │ no change │ │ QQuery 14 │ 522.42 / 524.66 ±1.55 / 527.28 ms │ 528.60 / 530.51 ±1.47 / 532.17 ms │ no change │ │ QQuery 15 │62.49 / 64.14 ±1.06 / 65.40 ms │64.32 / 65.17 ±0.87 / 66.34 ms │ no change │ │ QQuery 16 │ 7.10 / 7.25 ±0.15 / 7.52 ms │ 7.18 / 7.37 ±0.
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456788016 🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456691699) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Details ``` Comparing HEAD and xudong963_prune-page-index-followup Benchmark tpch_sf1.json ┏━━━┳┳━┳━━━┓ ┃ Query ┃ HEAD ┃ xudong963_prune-page-index-followup ┃Change ┃ ┡━━━╇╇━╇━━━┩ │ QQuery 1 │ 38.49 / 39.94 ±1.32 / 41.77 ms │ 38.54 / 40.22 ±1.15 / 41.64 ms │ no change │ │ QQuery 2 │ 20.29 / 20.68 ±0.30 / 21.20 ms │ 20.70 / 20.94 ±0.32 / 21.57 ms │ no change │ │ QQuery 3 │ 34.77 / 37.19 ±2.03 / 39.86 ms │ 36.23 / 37.91 ±0.89 / 38.83 ms │ no change │ │ QQuery 4 │ 17.19 / 17.34 ±0.10 / 17.48 ms │ 17.26 / 17.42 ±0.11 / 17.58 ms │ no change │ │ QQuery 5 │ 43.28 / 44.72 ±0.84 / 45.77 ms │ 42.63 / 44.42 ±1.10 / 45.51 ms │ no change │ │ QQuery 6 │ 16.35 / 16.65 ±0.35 / 17.33 ms │ 16.79 / 17.43 ±0.90 / 19.21 ms │ no change │ │ QQuery 7 │ 47.94 / 50.06 ±1.67 / 52.48 ms │ 49.87 / 50.70 ±0.93 / 52.42 ms │ no change │ │ QQuery 8 │ 45.64 / 45.88 ±0.16 / 46.07 ms │ 45.76 / 46.12 ±0.24 / 46.38 ms │ no change │ │ QQuery 9 │ 50.59 / 51.31 ±0.41 / 51.71 ms │ 50.97 / 51.28 ±0.24 / 51.60 ms │ no change │ │ QQuery 10 │ 63.77 / 64.63 ±0.98 / 66.41 ms │ 63.81 / 64.25 ±0.43 / 64.89 ms │ no change │ │ QQuery 11 │ 13.48 / 13.97 ±0.58 / 15.07 ms │ 13.48 / 14.11 ±0.74 / 15.56 ms │ no change │ │ QQuery 12 │ 25.01 / 25.26 ±0.17 / 25.52 ms │ 25.17 / 25.43 ±0.21 / 25.81 ms │ no change │ │ QQuery 13 │ 35.19 / 36.25 ±0.57 / 36.92 ms │ 35.23 / 36.39 ±0.77 / 37.31 ms │ no change │ │ QQuery 14 │ 25.96 / 26.07 ±0.09 / 26.18 ms │ 25.93 / 26.19 ±0.38 / 26.94 ms │ no change │ │ QQuery 15 │ 31.50 / 31.89 ±0.62 / 33.12 ms │ 31.69 / 31.98 ±0.35 / 32.66 ms │ no change │ │ QQuery 16 │ 15.13 / 15.20 ±0.08 / 15.33 ms │ 14.98 / 15.30 ±0.18 / 15.54 ms │ no change │ │ QQuery 17 │ 74.74 / 75.33 ±0.47 / 76.00 ms │ 74.96 / 75.84 ±0.77 / 77.02 ms │ no change │ │ QQuery 18 │ 68.01 / 69.21 ±0.83 / 70.58
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456707615 🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456691699) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4456691699-113-9ngm6 6.12.68+ #1 SMP Wed Apr 1 02:23:28 UTC 2026 aarch64 GNU/Linux` CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Comparing xudong963/prune-page-index-followup (d6ac2fb2655d4710a62847a84495b763f06fc388) to 3f501f4 (merge-base) [diff](https://github.com/apache/datafusion/compare/3f501f4b9e6bf8616556fbf6c832da7e364680a4..d6ac2fb2655d4710a62847a84495b763f06fc388) using: tpcds Results will be posted here when complete --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456704263 🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456691699) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4456691699-114-8wsp8 6.12.68+ #1 SMP Wed Apr 1 02:23:28 UTC 2026 aarch64 GNU/Linux` CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Comparing xudong963/prune-page-index-followup (d6ac2fb2655d4710a62847a84495b763f06fc388) to 3f501f4 (merge-base) [diff](https://github.com/apache/datafusion/compare/3f501f4b9e6bf8616556fbf6c832da7e364680a4..d6ac2fb2655d4710a62847a84495b763f06fc388) using: tpch Results will be posted here when complete --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456704294 🤖 Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/22191#issuecomment-4456691699) **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4456691699-112-sg94t 6.12.68+ #1 SMP Wed Apr 1 02:23:28 UTC 2026 aarch64 GNU/Linux` CPU Details (lscpu) ``` Architecture:aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: ARM Model name: Neoverse-V2 Model: 1 Thread(s) per core: 1 Core(s) per cluster: 16 Socket(s): - Cluster(s): 1 Stepping:r0p1 BogoMIPS:2000.00 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti L1d cache: 1 MiB (16 instances) L1i cache: 1 MiB (16 instances) L2 cache:32 MiB (16 instances) L3 cache:80 MiB (1 instance) NUMA node(s):1 NUMA node0 CPU(s): 0-15 Vulnerability Gather data sampling: Not affected Vulnerability Indirect target selection: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Reg file data sampling:Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1:Mitigation; __user pointer sanitization Vulnerability Spectre v2:Mitigation; CSV2, BHB Vulnerability Srbds: Not affected Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Not affected Vulnerability Vmscape: Not affected ``` Comparing xudong963/prune-page-index-followup (d6ac2fb2655d4710a62847a84495b763f06fc388) to 3f501f4 (merge-base) [diff](https://github.com/apache/datafusion/compare/3f501f4b9e6bf8616556fbf6c832da7e364680a4..d6ac2fb2655d4710a62847a84495b763f06fc388) using: clickbench_partitioned Results will be posted here when complete --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
xudong963 commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456691699 run benchmarks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
xudong963 commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456689064 run benchmark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Refactor parquet row filter setup [datafusion]
adriangbot commented on PR #22191: URL: https://github.com/apache/datafusion/pull/22191#issuecomment-4456689282 Hi @xudong963, `run benchmark` requires benchmark names (https://github.com/apache/datafusion/pull/22191#issuecomment-4456689064). Supported benchmarks: - Standard: clickbench_1, clickbench_extended, clickbench_partitioned, clickbench_pushdown, external_aggr, smj, sort_pushdown, sort_pushdown_inexact, sort_pushdown_inexact_overlap, sort_pushdown_inexact_unsorted, sort_pushdown_sorted, topk_tpch, tpcds, tpch, tpch10, tpch_mem, tpch_mem10 - Criterion: (any) Usage: ``` run benchmark# run specific benchmark(s) run benchmarks # run default suite run benchmarks # run specific benchmarks ``` Per-side configuration (`run benchmark tpch` followed by): ```yaml env: SHARED_SETTING: enabled baseline: ref: v45.0.0 env: DATAFUSION_RUNTIME_MEMORY_LIMIT: 1G changed: ref: v46.0.0 env: DATAFUSION_RUNTIME_MEMORY_LIMIT: 2G ``` --- [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
