This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 7fb8d07800 Publish built docs triggered by
1037f0aa205e0d04ade7aa9973b4b4c485b696ab
7fb8d07800 is described below
commit 7fb8d078000e6616cd3dfecddb7cad0e6bde5a40
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Tue Jan 6 13:24:31 2026 +0000
Publish built docs triggered by 1037f0aa205e0d04ade7aa9973b4b4c485b696ab
---
_sources/user-guide/cli/functions.md.txt | 38 +++++++++++++++++++++
searchindex.js | 2 +-
user-guide/cli/functions.html | 57 ++++++++++++++++++++++++++++++++
user-guide/cli/index.html | 1 +
4 files changed, 97 insertions(+), 1 deletion(-)
diff --git a/_sources/user-guide/cli/functions.md.txt
b/_sources/user-guide/cli/functions.md.txt
index f3b0163534..11f61297ac 100644
--- a/_sources/user-guide/cli/functions.md.txt
+++ b/_sources/user-guide/cli/functions.md.txt
@@ -170,5 +170,43 @@ The columns of the returned table are:
| table_size_bytes | Utf8 | Size of the table, in bytes
|
| statistics_size_bytes | UInt64 | Size of the cached statistics in memory
|
+## `list_files_cache`
+
+The `list_files_cache` function shows information about the `ListFilesCache`
that is used by the [`ListingTable`] implementation in DataFusion. When
creating a [`ListingTable`], DataFusion lists the files in the table's location
and caches results in the `ListFilesCache`. Subsequent queries against the same
table can reuse this cached information instead of re-listing the files.
+
+You can inspect the cache by querying the `list_files_cache` function. For
example,
+
+```sql
+> select split_part(path, '/', -1) as folder, metadata_size_bytes, expires_in,
unnest(metadata_list)['file_size_bytes'] as file_size_bytes,
unnest(metadata_list)['e_tag'] as e_tag from list_files_cache();
++----------+---------------------+-----------------------------------+-----------------+-------------------------------+
+| folder | metadata_size_bytes | expires_in |
file_size_bytes | e_tag |
++----------+---------------------+-----------------------------------+-----------------+-------------------------------+
+| customer | 1592 | 0 days 0 hours 0 mins 18.488 secs | 1233969
| 7041136-643a7bfeeec9b-12d431 |
+| customer | 1592 | 0 days 0 hours 0 mins 18.488 secs | 1234756
| 7041137-643a7bfeef2df-12d744 |
+| customer | 1592 | 0 days 0 hours 0 mins 18.488 secs | 1232554
| 7041139-643a7bfeef86a-12ceaa |
+| customer | 1592 | 0 days 0 hours 0 mins 18.488 secs | 1238676
| 704113a-643a7bfeef914-12e694 |
+| customer | 1592 | 0 days 0 hours 0 mins 18.488 secs | 1232186
| 704113b-643a7bfeefb22-12cd3a |
+| customer | 1592 | 0 days 0 hours 0 mins 18.488 secs | 1237506
| 7041138-643a7bfeef775-12e202 |
+| customer | 1592 | 0 days 0 hours 0 mins 18.488 secs | 1228756
| 7041134-643a7bfeec2d8-12bfd4 |
+| customer | 1592 | 0 days 0 hours 0 mins 18.488 secs | 1228509
| 7041135-643a7bfeed599-12bedd |
+| lineitem | 1600 | 0 days 0 hours 0 mins 16.758 secs |
20124715 | 704114a-643a7c00bb560-133142b |
+| lineitem | 1600 | 0 days 0 hours 0 mins 16.758 secs |
20131024 | 7041149-643a7c00b90b7-1332cd0 |
+| lineitem | 1600 | 0 days 0 hours 0 mins 16.758 secs |
20179217 | 704114b-643a7c00bb93e-133e911 |
+| lineitem | 1600 | 0 days 0 hours 0 mins 16.758 secs |
20296819 | 704114f-643a7c00ccefd-135b473 |
+| lineitem | 1600 | 0 days 0 hours 0 mins 16.758 secs |
20110730 | 7041148-643a7c00b9832-132dd8a |
+| lineitem | 1600 | 0 days 0 hours 0 mins 16.758 secs |
20128346 | 704114c-643a7c00bc00a-133225a |
+| lineitem | 1600 | 0 days 0 hours 0 mins 16.758 secs |
20130133 | 7041147-643a7c00b3901-1332955 |
+| lineitem | 1600 | 0 days 0 hours 0 mins 16.758 secs |
20139830 | 7041146-643a7c00abbe8-1334f36 |
++----------+---------------------+-----------------------------------+-----------------+-------------------------------+
+```
+
+The columns of the returned table are:
+| column_name | data_type | Description |
+| ------------------- | ------------ |
-----------------------------------------------------------------------------------------
|
+| path | Utf8 | File path relative to the object store / filesystem root |
+| metadata_size_bytes | UInt64 | Size of the cached metadata in memory (not
its thrift encoded form) |
+| expires_in | Duration(ms) | Last modified time of the file |
+| metadata_list | List(Struct) | List of metadatas, one for each file under
the path. |
+
[`listingtable`]:
https://docs.rs/datafusion/latest/datafusion/datasource/listing/struct.ListingTable.html
[entity tag]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag
diff --git a/searchindex.js b/searchindex.js
index beca48554c..e67b6addea 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"!=":[[61,"op-neq"]],"!~":[[61,"op-re-not-match"]],"!~*":[[61,"op-re-not-match-i"]],"!~~":[[61,"id19"]],"!~~*":[[61,"id20"]],"#":[[61,"op-bit-xor"]],"%":[[61,"op-modulo"]],"&":[[61,"op-bit-and"]],"(relation,
name) tuples in logical fields and logical columns are
unique":[[13,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[61,"op-multiply"]],"+":[[61,"op-plus"]],"-":[[61,"op-minus"]],"/":[[61,"op-divide"]],"<":[[61,"op-lt"]],"<
[...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"!=":[[61,"op-neq"]],"!~":[[61,"op-re-not-match"]],"!~*":[[61,"op-re-not-match-i"]],"!~~":[[61,"id19"]],"!~~*":[[61,"id20"]],"#":[[61,"op-bit-xor"]],"%":[[61,"op-modulo"]],"&":[[61,"op-bit-and"]],"(relation,
name) tuples in logical fields and logical columns are
unique":[[13,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[61,"op-multiply"]],"+":[[61,"op-plus"]],"-":[[61,"op-minus"]],"/":[[61,"op-divide"]],"<":[[61,"op-lt"]],"<
[...]
\ No newline at end of file
diff --git a/user-guide/cli/functions.html b/user-guide/cli/functions.html
index 466df398c7..e5d4c81788 100644
--- a/user-guide/cli/functions.html
+++ b/user-guide/cli/functions.html
@@ -683,6 +683,62 @@ enabled.</p>
</table>
</div>
</section>
+<section id="list-files-cache">
+<h2><code class="docutils literal notranslate"><span
class="pre">list_files_cache</span></code><a class="headerlink"
href="#list-files-cache" title="Link to this heading">#</a></h2>
+<p>The <code class="docutils literal notranslate"><span
class="pre">list_files_cache</span></code> function shows information about the
<code class="docutils literal notranslate"><span
class="pre">ListFilesCache</span></code> that is used by the <a
class="reference external"
href="https://docs.rs/datafusion/latest/datafusion/datasource/listing/struct.ListingTable.html";><code
class="docutils literal notranslate"><span
class="pre">ListingTable</span></code></a> implementation in DataFusion [...]
+<p>You can inspect the cache by querying the <code class="docutils literal
notranslate"><span class="pre">list_files_cache</span></code> function. For
example,</p>
+<div class="highlight-sql notranslate"><div
class="highlight"><pre><span></span><span class="o">></span><span class="w">
</span><span class="k">select</span><span class="w"> </span><span
class="n">split_part</span><span class="p">(</span><span
class="n">path</span><span class="p">,</span><span class="w"> </span><span
class="s1">'/'</span><span class="p">,</span><span class="w">
</span><span class="o">-</span><span class="mi">1</span><span
class="p">)</span><span class="w"> </s [...]
+<span class="o">+</span><span
class="c1">----------+---------------------+-----------------------------------+-----------------+-------------------------------+</span>
+<span class="o">|</span><span class="w"> </span><span
class="n">folder</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="n">metadata_size_bytes</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="n">expires_in</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="n">file_size_bytes</span><span class="w"> </span><span
class="o">|</span><span class=" [...]
+<span class="o">+</span><span
class="c1">----------+---------------------+-----------------------------------+-----------------+-------------------------------+</span>
+<span class="o">|</span><span class="w"> </span><span
class="n">customer</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1592</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">customer</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1592</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">customer</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1592</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">customer</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1592</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">customer</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1592</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">customer</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1592</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">customer</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1592</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">customer</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1592</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">lineitem</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1600</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">lineitem</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1600</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">lineitem</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1600</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">lineitem</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1600</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">lineitem</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1600</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">lineitem</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1600</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">lineitem</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1600</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">|</span><span class="w"> </span><span
class="n">lineitem</span><span class="w"> </span><span class="o">|</span><span
class="w"> </span><span class="mi">1600</span><span class="w">
</span><span class="o">|</span><span class="w"> </span><span
class="mi">0</span><span class="w"> </span><span class="n">days</span><span
class="w"> </span><span class="mi">0</span><span class="w"> </span><span
class="n">hours</span><span class="w"> </span><span class="mi">0</span> [...]
+<span class="o">+</span><span
class="c1">----------+---------------------+-----------------------------------+-----------------+-------------------------------+</span>
+</pre></div>
+</div>
+<p>The columns of the returned table are:</p>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>column_name</p></th>
+<th class="head"><p>data_type</p></th>
+<th class="head"><p>Description</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p>path</p></td>
+<td><p>Utf8</p></td>
+<td><p>File path relative to the object store / filesystem root</p></td>
+</tr>
+<tr class="row-odd"><td><p>metadata_size_bytes</p></td>
+<td><p>UInt64</p></td>
+<td><p>Size of the cached metadata in memory (not its thrift encoded
form)</p></td>
+</tr>
+<tr class="row-even"><td><p>expires_in</p></td>
+<td><p>Duration(ms)</p></td>
+<td><p>Last modified time of the file</p></td>
+</tr>
+<tr class="row-odd"><td><p>metadata_list</p></td>
+<td><p>List(Struct)</p></td>
+<td><p>List of metadatas, one for each file under the path.</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
</section>
@@ -735,6 +791,7 @@ enabled.</p>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#parquet-metadata"><code class="docutils literal notranslate"><span
class="pre">parquet_metadata</span></code></a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#metadata-cache"><code class="docutils literal notranslate"><span
class="pre">metadata_cache</span></code></a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#statistics-cache"><code class="docutils literal notranslate"><span
class="pre">statistics_cache</span></code></a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link"
href="#list-files-cache"><code class="docutils literal notranslate"><span
class="pre">list_files_cache</span></code></a></li>
</ul>
</nav></div>
diff --git a/user-guide/cli/index.html b/user-guide/cli/index.html
index c9162ad9c4..023f8dd386 100644
--- a/user-guide/cli/index.html
+++ b/user-guide/cli/index.html
@@ -430,6 +430,7 @@
<li class="toctree-l2"><a class="reference internal"
href="functions.html#parquet-metadata"><code class="docutils literal
notranslate"><span class="pre">parquet_metadata</span></code></a></li>
<li class="toctree-l2"><a class="reference internal"
href="functions.html#metadata-cache"><code class="docutils literal
notranslate"><span class="pre">metadata_cache</span></code></a></li>
<li class="toctree-l2"><a class="reference internal"
href="functions.html#statistics-cache"><code class="docutils literal
notranslate"><span class="pre">statistics_cache</span></code></a></li>
+<li class="toctree-l2"><a class="reference internal"
href="functions.html#list-files-cache"><code class="docutils literal
notranslate"><span class="pre">list_files_cache</span></code></a></li>
</ul>
</li>
</ul>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
