This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/datafusion-site.git
The following commit(s) were added to refs/heads/asf-staging by this push:
new f9cb640 Commit build products
f9cb640 is described below
commit f9cb6403ff9e29ecb1c4148cb81f9a96dbcb07d2
Author: Build Pelican (action) <[email protected]>
AuthorDate: Thu Nov 20 11:27:28 2025 +0000
Commit build products
---
blog/2025/11/25/datafusion-51.0.0/index.html | 34 ++++++++++++++++------------
blog/feeds/all-en.atom.xml | 34 ++++++++++++++++------------
blog/feeds/blog.atom.xml | 34 ++++++++++++++++------------
blog/feeds/pmc.atom.xml | 34 ++++++++++++++++------------
4 files changed, 80 insertions(+), 56 deletions(-)
diff --git a/blog/2025/11/25/datafusion-51.0.0/index.html
b/blog/2025/11/25/datafusion-51.0.0/index.html
index 846d671..f6077cf 100644
--- a/blog/2025/11/25/datafusion-51.0.0/index.html
+++ b/blog/2025/11/25/datafusion-51.0.0/index.html
@@ -211,32 +211,38 @@ functions benefit from the same syntax.</p>
<p>The output of <a
href="https://datafusion.apache.org/user-guide/sql/explain.html#explain-analyze";>EXPLAIN
ANALYZE</a> has been improved to include more metrics
about execution time and memory usage of each operator in the query plan.
Read about these new metrics in the <a
href="https://datafusion.apache.org/user-guide/metrics.html";>metrics user
guide</a>.</p>
+<p>The <code>51.0.0</code> release adds:</p>
+<ul>
+<li><strong>Configuration</strong>: adds a new option
<code>datafusion.explain.analyze_level</code>, which can be set to
<code>summary</code> for a concise output or <code>dev</code> for the full set
of metrics (the previous default).</li>
+<li><strong>For all major operators</strong>: adds <code>output_bytes</code>,
reporting how many bytes of data each operator produces.</li>
+<li><strong>FilterExec</strong>: adds a <code>selectivity</code> metric
(<code>output_rows / input_rows</code>) to show how effective the filter
is.</li>
+<li><strong>AggregateExec</strong>: </li>
+<li>adds detailed timing metrics for group-ID computation, aggregate argument
evaluation, aggregation work, and emitting final results.</li>
+<li>adds a <code>reduction_factor</code> metric (<code>output_rows /
input_rows</code>) to show how much grouping reduces the data.</li>
+<li><strong>NestedLoopJoinExec</strong>: adds a <code>selectivity</code>
metric (<code>output_rows / (left_rows * right_rows)</code>) to show how many
combinations actually pass the join condition.</li>
+<li>Several display formatting improvements were added to make <code>EXPLAIN
ANALYZE</code> output easier to read.</li>
+</ul>
<p>For example, the following query</p>
-<pre><code class="language-sql">explain analyze
+<pre><code class="language-sql">set datafusion.explain.analyze_level = summary
+
+explain analyze
select count(*)
from
'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet'
where "URL" <> '';
-</code></pre>
-<p>Now shows easier-to-understand metrics such as:</p>
-<pre><code class="language-text"> metrics=[
+
+Now shows easier-to-understand metrics such as:
+
+```text
+ metrics=[
output_rows=1000000,
elapsed_compute=16ns,
output_bytes=222.5 MB,
files_ranges_pruned_statistics=16 total → 16 matched,
row_groups_pruned_statistics=3 total → 3 matched,
row_groups_pruned_bloom_filter=3 total → 3 matched,
- page_index_rows_pruned=0 total → 0 matched,
- batches_split=0,
+ page_index_rows_pruned=0 total → 0 matched,
bytes_scanned=33661364,
- ...
metadata_load_time=4.243098ms,
- page_index_eval_time=1.016µs,
- row_pushdown_eval_time=32ns,
- statistics_eval_time=601.477µs,
- time_elapsed_opening=6.816666ms,
- time_elapsed_processing=159.983752ms,
- time_elapsed_scanning_total=4.468917754s,
- time_elapsed_scanning_until_data=1.658900959s
]
</code></pre>
<h2 id="upgrade-guide-and-changelog">Upgrade Guide and Changelog<a
class="headerlink" href="#upgrade-guide-and-changelog" title="Permanent
link">¶</a></h2>
diff --git a/blog/feeds/all-en.atom.xml b/blog/feeds/all-en.atom.xml
index 96e0d21..95cd13b 100644
--- a/blog/feeds/all-en.atom.xml
+++ b/blog/feeds/all-en.atom.xml
@@ -167,32 +167,38 @@ functions benefit from the same syntax.</p>
<p>The output of <a
href="https://datafusion.apache.org/user-guide/sql/explain.html#explain-analyze">EXPLAIN
ANALYZE</a> has been improved to include more metrics
about execution time and memory usage of each operator in the query plan.
Read about these new metrics in the <a
href="https://datafusion.apache.org/user-guide/metrics.html">metrics user
guide</a>.</p>
+<p>The <code>51.0.0</code> release adds:</p>
+<ul>
+<li><strong>Configuration</strong>: adds a new option
<code>datafusion.explain.analyze_level</code>, which can be set to
<code>summary</code> for a concise output or
<code>dev</code> for the full set of metrics (the previous
default).</li>
+<li><strong>For all major operators</strong>: adds
<code>output_bytes</code>, reporting how many bytes of data each
operator produces.</li>
+<li><strong>FilterExec</strong>: adds a
<code>selectivity</code> metric (<code>output_rows /
input_rows</code>) to show how effective the filter is.</li>
+<li><strong>AggregateExec</strong>: </li>
+<li>adds detailed timing metrics for group-ID computation, aggregate
argument evaluation, aggregation work, and emitting final results.</li>
+<li>adds a <code>reduction_factor</code> metric
(<code>output_rows / input_rows</code>) to show how much grouping
reduces the data.</li>
+<li><strong>NestedLoopJoinExec</strong>: adds a
<code>selectivity</code> metric (<code>output_rows /
(left_rows * right_rows)</code>) to show how many combinations actually
pass the join condition.</li>
+<li>Several display formatting improvements were added to make
<code>EXPLAIN ANALYZE</code> output easier to read.</li>
+</ul>
<p>For example, the following query</p>
-<pre><code class="language-sql">explain analyze
+<pre><code class="language-sql">set
datafusion.explain.analyze_level = summary
+
+explain analyze
select count(*)
from
'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet'
where "URL" &lt;&gt; '';
-</code></pre>
-<p>Now shows easier-to-understand metrics such as:</p>
-<pre><code class="language-text"> metrics=[
+
+Now shows easier-to-understand metrics such as:
+
+```text
+ metrics=[
output_rows=1000000,
elapsed_compute=16ns,
output_bytes=222.5 MB,
files_ranges_pruned_statistics=16 total → 16 matched,
row_groups_pruned_statistics=3 total → 3 matched,
row_groups_pruned_bloom_filter=3 total → 3 matched,
- page_index_rows_pruned=0 total → 0 matched,
- batches_split=0,
+ page_index_rows_pruned=0 total → 0 matched,
bytes_scanned=33661364,
- ...
metadata_load_time=4.243098ms,
- page_index_eval_time=1.016µs,
- row_pushdown_eval_time=32ns,
- statistics_eval_time=601.477µs,
- time_elapsed_opening=6.816666ms,
- time_elapsed_processing=159.983752ms,
- time_elapsed_scanning_total=4.468917754s,
- time_elapsed_scanning_until_data=1.658900959s
]
</code></pre>
<h2 id="upgrade-guide-and-changelog">Upgrade Guide and Changelog<a
class="headerlink" href="#upgrade-guide-and-changelog" title="Permanent
link">¶</a></h2>
diff --git a/blog/feeds/blog.atom.xml b/blog/feeds/blog.atom.xml
index c91c163..367ce27 100644
--- a/blog/feeds/blog.atom.xml
+++ b/blog/feeds/blog.atom.xml
@@ -167,32 +167,38 @@ functions benefit from the same syntax.</p>
<p>The output of <a
href="https://datafusion.apache.org/user-guide/sql/explain.html#explain-analyze">EXPLAIN
ANALYZE</a> has been improved to include more metrics
about execution time and memory usage of each operator in the query plan.
Read about these new metrics in the <a
href="https://datafusion.apache.org/user-guide/metrics.html">metrics user
guide</a>.</p>
+<p>The <code>51.0.0</code> release adds:</p>
+<ul>
+<li><strong>Configuration</strong>: adds a new option
<code>datafusion.explain.analyze_level</code>, which can be set to
<code>summary</code> for a concise output or
<code>dev</code> for the full set of metrics (the previous
default).</li>
+<li><strong>For all major operators</strong>: adds
<code>output_bytes</code>, reporting how many bytes of data each
operator produces.</li>
+<li><strong>FilterExec</strong>: adds a
<code>selectivity</code> metric (<code>output_rows /
input_rows</code>) to show how effective the filter is.</li>
+<li><strong>AggregateExec</strong>: </li>
+<li>adds detailed timing metrics for group-ID computation, aggregate
argument evaluation, aggregation work, and emitting final results.</li>
+<li>adds a <code>reduction_factor</code> metric
(<code>output_rows / input_rows</code>) to show how much grouping
reduces the data.</li>
+<li><strong>NestedLoopJoinExec</strong>: adds a
<code>selectivity</code> metric (<code>output_rows /
(left_rows * right_rows)</code>) to show how many combinations actually
pass the join condition.</li>
+<li>Several display formatting improvements were added to make
<code>EXPLAIN ANALYZE</code> output easier to read.</li>
+</ul>
<p>For example, the following query</p>
-<pre><code class="language-sql">explain analyze
+<pre><code class="language-sql">set
datafusion.explain.analyze_level = summary
+
+explain analyze
select count(*)
from
'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet'
where "URL" &lt;&gt; '';
-</code></pre>
-<p>Now shows easier-to-understand metrics such as:</p>
-<pre><code class="language-text"> metrics=[
+
+Now shows easier-to-understand metrics such as:
+
+```text
+ metrics=[
output_rows=1000000,
elapsed_compute=16ns,
output_bytes=222.5 MB,
files_ranges_pruned_statistics=16 total → 16 matched,
row_groups_pruned_statistics=3 total → 3 matched,
row_groups_pruned_bloom_filter=3 total → 3 matched,
- page_index_rows_pruned=0 total → 0 matched,
- batches_split=0,
+ page_index_rows_pruned=0 total → 0 matched,
bytes_scanned=33661364,
- ...
metadata_load_time=4.243098ms,
- page_index_eval_time=1.016µs,
- row_pushdown_eval_time=32ns,
- statistics_eval_time=601.477µs,
- time_elapsed_opening=6.816666ms,
- time_elapsed_processing=159.983752ms,
- time_elapsed_scanning_total=4.468917754s,
- time_elapsed_scanning_until_data=1.658900959s
]
</code></pre>
<h2 id="upgrade-guide-and-changelog">Upgrade Guide and Changelog<a
class="headerlink" href="#upgrade-guide-and-changelog" title="Permanent
link">¶</a></h2>
diff --git a/blog/feeds/pmc.atom.xml b/blog/feeds/pmc.atom.xml
index fa7c980..cf4006b 100644
--- a/blog/feeds/pmc.atom.xml
+++ b/blog/feeds/pmc.atom.xml
@@ -167,32 +167,38 @@ functions benefit from the same syntax.</p>
<p>The output of <a
href="https://datafusion.apache.org/user-guide/sql/explain.html#explain-analyze">EXPLAIN
ANALYZE</a> has been improved to include more metrics
about execution time and memory usage of each operator in the query plan.
Read about these new metrics in the <a
href="https://datafusion.apache.org/user-guide/metrics.html">metrics user
guide</a>.</p>
+<p>The <code>51.0.0</code> release adds:</p>
+<ul>
+<li><strong>Configuration</strong>: adds a new option
<code>datafusion.explain.analyze_level</code>, which can be set to
<code>summary</code> for a concise output or
<code>dev</code> for the full set of metrics (the previous
default).</li>
+<li><strong>For all major operators</strong>: adds
<code>output_bytes</code>, reporting how many bytes of data each
operator produces.</li>
+<li><strong>FilterExec</strong>: adds a
<code>selectivity</code> metric (<code>output_rows /
input_rows</code>) to show how effective the filter is.</li>
+<li><strong>AggregateExec</strong>: </li>
+<li>adds detailed timing metrics for group-ID computation, aggregate
argument evaluation, aggregation work, and emitting final results.</li>
+<li>adds a <code>reduction_factor</code> metric
(<code>output_rows / input_rows</code>) to show how much grouping
reduces the data.</li>
+<li><strong>NestedLoopJoinExec</strong>: adds a
<code>selectivity</code> metric (<code>output_rows /
(left_rows * right_rows)</code>) to show how many combinations actually
pass the join condition.</li>
+<li>Several display formatting improvements were added to make
<code>EXPLAIN ANALYZE</code> output easier to read.</li>
+</ul>
<p>For example, the following query</p>
-<pre><code class="language-sql">explain analyze
+<pre><code class="language-sql">set
datafusion.explain.analyze_level = summary
+
+explain analyze
select count(*)
from
'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet'
where "URL" &lt;&gt; '';
-</code></pre>
-<p>Now shows easier-to-understand metrics such as:</p>
-<pre><code class="language-text"> metrics=[
+
+Now shows easier-to-understand metrics such as:
+
+```text
+ metrics=[
output_rows=1000000,
elapsed_compute=16ns,
output_bytes=222.5 MB,
files_ranges_pruned_statistics=16 total → 16 matched,
row_groups_pruned_statistics=3 total → 3 matched,
row_groups_pruned_bloom_filter=3 total → 3 matched,
- page_index_rows_pruned=0 total → 0 matched,
- batches_split=0,
+ page_index_rows_pruned=0 total → 0 matched,
bytes_scanned=33661364,
- ...
metadata_load_time=4.243098ms,
- page_index_eval_time=1.016µs,
- row_pushdown_eval_time=32ns,
- statistics_eval_time=601.477µs,
- time_elapsed_opening=6.816666ms,
- time_elapsed_processing=159.983752ms,
- time_elapsed_scanning_total=4.468917754s,
- time_elapsed_scanning_until_data=1.658900959s
]
</code></pre>
<h2 id="upgrade-guide-and-changelog">Upgrade Guide and Changelog<a
class="headerlink" href="#upgrade-guide-and-changelog" title="Permanent
link">¶</a></h2>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
