This is an automated email from the ASF dual-hosted git repository.
dzamo pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/drill-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new ab4b80d New blog post on HTTP streaming from the REST API.
ab4b80d is described below
commit ab4b80d6c6b1c47bd7f2391f1c7a31800745dcff
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Fri Jul 9 16:10:19 2021 +0200
New blog post on HTTP streaming from the REST API.
---
README.md | 2 +-
.../09/streaming-data-from-the-rest-api/index.html | 235 +++++++++++++++++++++
blog/index.html | 5 +
feed.xml | 129 ++++++-----
index.html | 4 +-
zh/README.md | 2 +-
.../09/streaming-data-from-the-rest-api/index.html | 235 +++++++++++++++++++++
zh/blog/index.html | 5 +
zh/feed.xml | 129 ++++++-----
zh/index.html | 4 +-
10 files changed, 632 insertions(+), 118 deletions(-)
diff --git a/README.md b/README.md
index 5d87b02..e230317 100644
--- a/README.md
+++ b/README.md
@@ -144,7 +144,7 @@ The English versions of "site" pages such as index.html are
stored in the root d
## Add translated collection pages
-The English versions of "collection" pages such as the markdown under _docs/
are stored in an en/ subdirectory of the collection root. Create corresponding
translated pages in the collection under `lang-code/` in which you translate
both `title` and `parent` in the front matter but leave the `slug` the same as
the English page and set `lang` to `lang-code`.
+The English versions of "collection" pages such as the markdown under _docs/
are stored in an en/ subdirectory of the collection root. Create corresponding
translated pages in the collection under `lang-code/` in which you translate
both `title` and `parent` in the front matter but leave the `slug` the same as
the English page and set `lang` to `lang-code`. Once you've translated the
`title` of a parent page, you will need to provide files for each of its
children (which can still cont [...]
# Compiling the Website
diff --git a/blog/2021/07/09/streaming-data-from-the-rest-api/index.html
b/blog/2021/07/09/streaming-data-from-the-rest-api/index.html
new file mode 100644
index 0000000..04e3dcb
--- /dev/null
+++ b/blog/2021/07/09/streaming-data-from-the-rest-api/index.html
@@ -0,0 +1,235 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+<meta name=viewport content="width=device-width, initial-scale=1">
+
+
+<title>Streaming data from Drill REST API - Apache Drill</title>
+
+<link
href="https://maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css";
rel="stylesheet" type="text/css"/>
+<link href='https://fonts.googleapis.com/css?family=PT+Sans' rel='stylesheet'
type='text/css'/>
+<link href="/css/site.css" rel="stylesheet" type="text/css"/>
+
+<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"/>
+<link rel="icon" href="/favicon.ico" type="image/x-icon"/>
+
+<script
src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js";
language="javascript" type="text/javascript"></script>
+<script
src="https://cdnjs.cloudflare.com/ajax/libs/jquery-easing/1.3/jquery.easing.min.js";
language="javascript" type="text/javascript"></script>
+<script language="javascript" type="text/javascript"
src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/script.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/drill.js"></script>
+
+</head>
+
+
+<body onResize="resized();">
+ <div class="page-wrap">
+ <div class="bui"></div>
+
+<div id="menu" class="mw">
+<ul>
+ <li class='toc-categories'>
+ <a class="expand-toc-icon" href="javascript:void(0);"><i class="fa
fa-bars"></i></a>
+ </li>
+ <li class="logo"><a href="/"></a></li>
+ <li class='expand-menu'>
+ <a href="javascript:void(0);"><span class='menu-text'>Menu</span><span
class='expand-icon'><i class="fa fa-bars"></i></span></a>
+ </li>
+ <li class="clear-float"></li>
+ <li class="nav">
+ <a>Language</a>
+ <ul>
+
+ <li>
+ <a style="font-weight: bold;"
href="/blog/2021/07/09/streaming-data-from-the-rest-api/" >en</a>
+ </li>
+
+ <li>
+ <a
href="/zh/blog/2021/07/09/streaming-data-from-the-rest-api/" >zh</a>
+ </li>
+
+ </ul>
+ </li>
+ <li class="apache-link">
+ <a href="/apacheASF/">Apache</a>
+ </li>
+ <li class="poweredby">
+ <a href="/poweredBy">Powered By</a>
+ </li>
+ <li class="documentation-menu">
+ <a href="/docs/">Documentation</a>
+ <ul>
+
+ <li><a href="/docs/getting-started/">Getting Started</a></li>
+
+ <li><a href="/docs/architecture/">Architecture</a></li>
+
+ <li><a href="/docs/tutorials/">Tutorials</a></li>
+
+ <li><a href="/docs/drill-on-yarn/">Drill-on-YARN</a></li>
+
+ <li><a href="/docs/install-drill/">Install Drill</a></li>
+
+ <li><a href="/docs/configure-drill/">Configure Drill</a></li>
+
+ <li><a href="/docs/connect-a-data-source/">Connect a Data
Source</a></li>
+
+ <li><a href="/docs/odbc-jdbc-interfaces/">ODBC/JDBC Interfaces</a></li>
+
+ <li><a href="/docs/query-data/">Query Data</a></li>
+
+ <li><a href="/docs/performance-tuning/">Performance Tuning</a></li>
+
+ <li><a href="/docs/log-and-debug/">Log and Debug</a></li>
+
+ <li><a href="/docs/sql-reference/">SQL Reference</a></li>
+
+ <li><a href="/docs/data-sources-and-file-formats/">Data Sources and
File Formats</a></li>
+
+ <li><a href="/docs/develop-custom-functions/">Develop Custom
Functions</a></li>
+
+ <li><a href="/docs/troubleshooting/">Troubleshooting</a></li>
+
+ <li><a href="/docs/developer-information/">Developer
Information</a></li>
+
+ <li><a href="/docs/release-notes/">Release Notes</a></li>
+
+ <li><a href="/docs/sample-datasets/">Sample Datasets</a></li>
+
+ <li><a href="/docs/project-bylaws/">Project Bylaws</a></li>
+
+ <li><a href="/docs/ecosystem/">Ecosystem</a></li>
+
+ </ul>
+ </li>
+ <li class='nav'>
+ <a href="/community-resources/">Community</a>
+ <ul>
+ <li><a href="/team/">Team</a></li>
+ <li><a href="/mailinglists/">Mailing Lists</a></li>
+ <li><a href="/community-resources/">Community Resources</a></li>
+ </ul>
+ </li>
+ <li class='nav'><a href="/faq/">FAQ</a></li>
+ <li class='nav'><a href="/blog/">Blog</a></li>
+ <li class="social-menu-item"><a href="https://twitter.com/apachedrill";
title="apachedrill on twitter" target="_blank"><img
src="/images/twitter_32_26_white.png" alt="twitter logo" align="center"></a>
</li>
+ <li class="social-menu-item"><a
href="https://join.slack.com/t/apache-drill/shared_invite/enQtNTQ4MjM1MDA3MzQ2LTJlYmUxMTRkMmUwYmQ2NTllYmFmMjU4MDk0NjYwZjBmYjg0MDZmOTE2ZDg0ZjBlYmI3Yjc4Y2I2NTQyNGVlZTc";
title="Apache Drill Slack channels"
+ target="_blank"><img src="/images/slack-logo.svg" alt="Slack logo"
align="center"></a> </li>
+ <li class='search-bar'>
+ <form id="drill-search-form">
+ <input type="text" placeholder="Search Apache Drill"
id="drill-search-term" />
+ <button type="submit">
+ <i class="fa fa-search"></i>
+ </button>
+ </form>
+ </li>
+ <li class="d">
+ <a href="/download/">
+ <i class="fa fa-cloud-download"></i> Download
+ </a>
+ </li>
+</ul>
+</div>
+
+ <link href="/css/content.css" rel="stylesheet" type="text/css">
+
+<div class="post int_text">
+ <header class="post-header">
+ <div class="int_title">
+ <h1 class="post-title">Streaming data from Drill REST API</h1>
+ </div>
+ <p class="post-meta">
+
+
+
+ <strong>Author:</strong> James Turton (Committer, Apache Software
Foundation)<br />
+
+<strong>Date:</strong> Jul 9, 2021
+</p>
+ </header>
+ <div class="addthis_sharing_toolbox"></div>
+
+ <article class="post-content">
+ <p>Anyone who’s used a UNIX pipe, or even just watched something on
Netflix, is at least a little familiar with the idea of processing data in a
streaming fashion. While your data are in small volume compared to available
memory and I/O speeds, streaming is something you can afford to dispense with.
But when you cannot fit an entire dataset in RAM, or when if you have to
download an entire 4K movie before you can start playing it, then streaming
data processing can make a game chan [...]
+
+<p>With the relase of version 1.19, Drill will stream JSON query result data
over an HTTP response to the client that initiated the query using the REST
API. And if anything can easily get big compared to your available RAM or
network speed, it’s query results coming back from Drill. It’s important to
note here that JSON over HTTP is never going to be the most <em>efficient</em>
way to move big data around<sup id="fnref:1" role="doc-noteref"><a href="#fn:1"
class="footnote">1</a></sup> [...]
+
+<p>Where JSON and HTTP <em>do</em> win is universality: today it’s hard to
imagine a client hardware and software stack that doesn’t provide JSON and HTTP
out of the box with minimal effort. So it’s important that they work as well
as is possible, in spite of the alternatives that exist. The new streaming
query results delivery on the server side means that Drill’s heap memory isn’t
pressurised by having to buffer entire result sets before it starts to transmit
them over the network. [...]
+
+<p>To fully realise the benefits of streaming query result data, clients can
<em>themselves</em> operate on the HTTP response they receive in a streaming
fashion, thereby potentially starting to process records before Drill has even
finished materialising them and avoiding holding the full set in memory if they
choose. At the transport level (when there are enough results to warrant it),
the HTTP response headers from the Drill REST API will include a <code
class="language-plaintext hig [...]
+
+<p>If you set out to develop a streaming HTTP client for the Drill REST API,
do take note that the schema of the JSON query result is <em>not</em> a <a
href="https://en.wikipedia.org/wiki/JSON_streaming";>streaming JSON format</a>
like “JSON lines”. This means that you must be careful about which JSON
objects you parse entirely in a single call, particularly avoiding any parent
of the <code class="language-plaintext highlighter-rouge">rows</code> property
in the query result which would [...]
+
+<p>In closing, and for a bit of fun, here’s the log from a short IPython
session where I use sqlalchemy-drill to run <code class="language-plaintext
highlighter-rouge">SELECT *</code> on a remote 17 billion record table over a
0.5 Mbit/s link and start scanning through (a steady trickle of) rows in
seconds.</p>
+
+<pre><code class="language-ipython">In [4]: r = engine.execute('select
count(*) from dfs.ws.big_table')
+INFO:drilldbapi:received Drill query ID 1f211888-cc20-fe6f-69d1-6584c5caa2df.
+INFO:drilldbapi:opened a row data stream of 1 columns.
+
+In [5]: next(r)
+Out[5]: (17437571247,)
+
+In [6]: r = engine.execute('select * from dfs.ws.big_table')
+INFO:drilldbapi:received Drill query ID 1f211838-73df-1506-a74e-f5695f6b0ff5.
+INFO:drilldbapi:opened a row data stream of 21 columns.
+
+In [7]: while True:
+ ...: _ = next(r)
+ ...:
+ INFO:drilldbapi:streamed 10000 rows.
+ INFO:drilldbapi:streamed 20000 rows.
+ INFO:drilldbapi:streamed 30000 rows.
+ INFO:drilldbapi:streamed 40000 rows.
+</code></pre>
+
+<div class="footnotes" role="doc-endnotes">
+ <ol>
+ <li id="fn:1" role="doc-endnote">
+ <p>Some mitigation is possible. JSON representations of tabular big
data are typically extremely compressible and you can reduce the bytes sent
over the network by 10-20x by enabling HTTP response compression on a web
server like Apache placed in front of the Drill REST API. <a href="#fnref:1"
class="reversefootnote" role="doc-backlink">↩</a></p>
+ </li>
+ </ol>
+</div>
+
+ </article>
+ <div id="disqus_thread"></div>
+ <script type="text/javascript">
+ /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE
* * */
+ var disqus_shortname = 'drill'; // required: replace example with your
forum shortname
+
+ /* * * DON'T EDIT BELOW THIS LINE * * */
+ (function() {
+ var dsq = document.createElement('script'); dsq.type =
'text/javascript'; dsq.async = true;
+ dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
+ (document.getElementsByTagName('head')[0] ||
document.getElementsByTagName('body')[0]).appendChild(dsq);
+ })();
+ </script>
+ <noscript>Please enable JavaScript to view the <a
href="http://disqus.com/?ref_noscript";>comments powered by
Disqus.</a></noscript>
+
+</div>
+<script type="text/javascript"
src="https://s7.addthis.com/js/300/addthis_widget.js#pubid=ra-548b2caa33765e8d";
async="async"></script>
+
+ </div>
+ <p class="push"></p>
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2020 The Apache Software Foundation, licensed under the
Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software
Foundation. Other names appearing on the site may be trademarks of their
respective owners.<br/><br/>
+</div>
+</div>
+
+ <script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+<script type="text/javascript"
src="https://s7.addthis.com/js/300/addthis_widget.js#pubid=ra-548b2caa33765e8d";
async="async"></script>
+
+</body>
+</html>
diff --git a/blog/index.html b/blog/index.html
index ff5df94..9f136a0 100644
--- a/blog/index.html
+++ b/blog/index.html
@@ -140,6 +140,11 @@
</div>
<div class="int_text" align="left"><!-- previously: site.posts -->
+<p><a class="post-link"
href="/blog/2021/07/09/streaming-data-from-the-rest-api/">Streaming data from
Drill REST API</a><br/>
+<span class="post-date">Posted on Jul 9, 2021
+by James Turton</span>
+<br/>The release of Apache Drill 1.19 saw a major change under the hood of
Drill's REST API with the introduction of a streaming data path for query
results moving from Drill and over the network to the initiating client. The
result is better memory utilisation, less blocking and a more reliable API.</p>
+<!-- previously: site.posts -->
<p><a class="post-link" href="/blog/2021/06/10/drill-1.19-released/">Drill
1.19 Released</a><br/>
<span class="post-date">Posted on Jun 10, 2021
by Laurent Goujon</span>
diff --git a/feed.xml b/feed.xml
index 4e0f7d3..3520e0d 100644
--- a/feed.xml
+++ b/feed.xml
@@ -4,13 +4,64 @@
<title>Apache Drill - Schema-free SQL for Hadoop, NoSQL and Cloud
Storage</title>
<description>The official user documentation for Apache Drill.
</description>
- <link>/</link>
- <atom:link href="/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Thu, 08 Jul 2021 14:57:25 +0200</pubDate>
- <lastBuildDate>Thu, 08 Jul 2021 14:57:25 +0200</lastBuildDate>
+ <link>http://localhost:4000/</link>
+ <atom:link href="http://localhost:4000/feed.xml"; rel="self"
type="application/rss+xml"/>
+ <pubDate>Fri, 09 Jul 2021 15:37:50 +0200</pubDate>
+ <lastBuildDate>Fri, 09 Jul 2021 15:37:50 +0200</lastBuildDate>
<generator>Jekyll v3.9.0</generator>
<item>
+ <title>Streaming data from Drill REST API</title>
+ <description><p>Anyone who’s used a UNIX pipe, or even just
watched something on Netflix, is at least a little familiar with the idea of
processing data in a streaming fashion. While your data are in small volume
compared to available memory and I/O speeds, streaming is something you can
afford to dispense. But if you cannot fit an entire dataset in RAM, or when if
you have to download an entire 4K movie before you can start playing it, then
streaming can make a game chan [...]
+
+<p>With the relase of version 1.19, Drill will stream JSON query result
data over an HTTP response to the client that initiated the query using the
REST API. And if anything can easily get big compared to your available RAM or
network speeds, its query results coming out of Drill. It’s important to note
here that JSON over HTTP is never going to be the most efficient way to move
big data around<sup id="fnref:1"
role="doc-noteref"><a href="#fn:1&q [...]
+
+<p>Where JSON and HTTP <em>do</em> win on universality:
today it’s hard to image a client hardware and software stack that doesn’t
provide JSON and HTTP out of the box with minimal effort. So it’s important
that they work as well as is possible. The new streaming query results
materialisation on the server side means that Drill’s heap memory isn’t
pressurised by having to buffer entire result sets before starting to transmit
them over the network. Even existing REST [...]
+
+<p>To fully realise the benefits of streaming query result data, clients
can <em>themselves</em> operate on the HTTP response they receive
in a streaming fashion, thereby potentially starting to process records before
Drill has even finished materialsing them. At the transport level (when there
are enough results to warrant it), the HTTP response headers from the Drill
REST API will include a <code class="language-plaintext
highlighter-rouge">Transfer-E [...]
+
+<p>Streaming HTTP client developers should note well that the schema of
JSON query result is <em>not</em> a <a
href="https://en.wikipedia.org/wiki/JSON_streaming">streaming JSON
format</a> like “JSON lines”. This means that you should take care to
avoid parsing any entire object which is a parent of the <code
class="language-plaintext highlighter-rouge">rows</code>
property in the query result, thereby parsing the essentailly [...]
+
+<p>In closing, and for a bit of fun, here’s the log from a short IPython
session where I use sqlalchemy-drill to run <code
class="language-plaintext highlighter-rouge">SELECT *</code>
on a remote 17 billion record table over a 0.5 Mbit/s link and start receiving
(a steady trickle of) rows in seconds.</p>
+
+<pre><code class="language-ipython">In [4]: r =
engine.execute('select count(*) from dfs.ws.big_table')
+INFO:drilldbapi:received Drill query ID 1f211888-cc20-fe6f-69d1-6584c5caa2df.
+INFO:drilldbapi:opened a row data stream of 1 columns.
+
+In [5]: next(r)
+Out[5]: (17437571247,)
+
+In [6]: r = engine.execute('select * from dfs.ws.big_table')
+INFO:drilldbapi:received Drill query ID 1f211838-73df-1506-a74e-f5695f6b0ff5.
+INFO:drilldbapi:opened a row data stream of 21 columns.
+
+In [7]: while True:
+ ...: _ = next(r)
+ ...:
+ INFO:drilldbapi:streamed 10000 rows.
+ INFO:drilldbapi:streamed 20000 rows.
+ INFO:drilldbapi:streamed 30000 rows.
+ INFO:drilldbapi:streamed 40000 rows.
+</code></pre>
+
+<div class="footnotes" role="doc-endnotes">
+ <ol>
+ <li id="fn:1" role="doc-endnote">
+ <p>JSON representations of tabular big data are typically
extremely compressible and you can reduce the bytes sent over the network by
10-20x by enabling HTTP response compression on a web server like Apache placed
in front of the Drill REST API. <a href="#fnref:1"
class="reversefootnote"
role="doc-backlink">&#8617;</a></p>
+ </li>
+ </ol>
+</div>
+</description>
+ <pubDate>Fri, 09 Jul 2021 00:00:00 +0200</pubDate>
+
<link>http://localhost:4000/blog/2021/07/09/streaming-data-from-the-rest-api/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2021/07/09/streaming-data-from-the-rest-api/</guid>
+
+
+ <category>blog</category>
+
+ </item>
+
+ <item>
<title>Drill 1.19 Released</title>
<description><p>Today, we’re happy to announce the availability
of Drill 1.19.0. You can download it <a
href="https://drill.apache.org/download/">here</a>.</p>
@@ -40,8 +91,8 @@
<p>You can find a complete list of improvements and JIRAs resolved in
the 1.19.0 release <a
href="/docs/apache-drill-1-19-0-release-notes/">here</a>.</p>
</description>
<pubDate>Thu, 10 Jun 2021 00:00:00 +0200</pubDate>
- <link>/blog/2021/06/10/drill-1.19-released/</link>
- <guid isPermaLink="true">/blog/2021/06/10/drill-1.19-released/</guid>
+ <link>http://localhost:4000/blog/2021/06/10/drill-1.19-released/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2021/06/10/drill-1.19-released/</guid>
<category>blog</category>
@@ -64,8 +115,8 @@
<p>You can find a complete list of improvements and JIRAs resolved in
the 1.18.0 release <a
href="/docs/apache-drill-1-18-0-release-notes/">here</a>.</p>
</description>
<pubDate>Sat, 05 Sep 2020 00:00:00 +0200</pubDate>
- <link>/blog/2020/09/05/drill-1.18-released/</link>
- <guid isPermaLink="true">/blog/2020/09/05/drill-1.18-released/</guid>
+ <link>http://localhost:4000/blog/2020/09/05/drill-1.18-released/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2020/09/05/drill-1.18-released/</guid>
<category>blog</category>
@@ -106,8 +157,8 @@
</description>
<pubDate>Thu, 26 Dec 2019 00:00:00 +0200</pubDate>
- <link>/blog/2019/12/26/drill-1.17-released/</link>
- <guid isPermaLink="true">/blog/2019/12/26/drill-1.17-released/</guid>
+ <link>http://localhost:4000/blog/2019/12/26/drill-1.17-released/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2019/12/26/drill-1.17-released/</guid>
<category>blog</category>
@@ -119,8 +170,8 @@
<description>
</description>
<pubDate>Thu, 02 May 2019 00:00:00 +0200</pubDate>
- <link>/blog/2019/05/02/drill-user-meetup/</link>
- <guid isPermaLink="true">/blog/2019/05/02/drill-user-meetup/</guid>
+ <link>http://localhost:4000/blog/2019/05/02/drill-user-meetup/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2019/05/02/drill-user-meetup/</guid>
<category>blog</category>
@@ -155,8 +206,8 @@
</description>
<pubDate>Thu, 02 May 2019 00:00:00 +0200</pubDate>
- <link>/blog/2019/05/02/drill-1.16-released/</link>
- <guid isPermaLink="true">/blog/2019/05/02/drill-1.16-released/</guid>
+ <link>http://localhost:4000/blog/2019/05/02/drill-1.16-released/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2019/05/02/drill-1.16-released/</guid>
<category>blog</category>
@@ -188,8 +239,8 @@
</description>
<pubDate>Mon, 31 Dec 2018 00:00:00 +0200</pubDate>
- <link>/blog/2018/12/31/drill-1.15-released/</link>
- <guid isPermaLink="true">/blog/2018/12/31/drill-1.15-released/</guid>
+ <link>http://localhost:4000/blog/2018/12/31/drill-1.15-released/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2018/12/31/drill-1.15-released/</guid>
<category>blog</category>
@@ -223,8 +274,8 @@
</ul>
</description>
<pubDate>Sat, 01 Dec 2018 00:00:00 +0200</pubDate>
- <link>/blog/2018/12/01/learning-apache-drill-book/</link>
- <guid
isPermaLink="true">/blog/2018/12/01/learning-apache-drill-book/</guid>
+
<link>http://localhost:4000/blog/2018/12/01/learning-apache-drill-book/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2018/12/01/learning-apache-drill-book/</guid>
<category>blog</category>
@@ -236,8 +287,8 @@
<description>
</description>
<pubDate>Tue, 16 Oct 2018 21:18:04 +0200</pubDate>
- <link>/blog/2018/10/16/drill-user-meetup/</link>
- <guid isPermaLink="true">/blog/2018/10/16/drill-user-meetup/</guid>
+ <link>http://localhost:4000/blog/2018/10/16/drill-user-meetup/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2018/10/16/drill-user-meetup/</guid>
<category>blog</category>
@@ -249,42 +300,8 @@
<description>
</description>
<pubDate>Tue, 16 Oct 2018 21:18:04 +0200</pubDate>
- <link>/blog/2018/10/16/drill-developer-day/</link>
- <guid isPermaLink="true">/blog/2018/10/16/drill-developer-day/</guid>
-
-
- <category>blog</category>
-
- </item>
-
- <item>
- <title>Drill 1.14 Released</title>
- <description><p>Today, we’re happy to announce the availability
of Drill 1.14.0. You can download it <a
href="https://drill.apache.org/download/">here</a>.</p>
-
-<p>The release provides the following bug fixes and
improvements:</p>
-
-<h2 id="run-drill-in-a-docker-container-drill-6346">Run Drill
in a Docker Container (DRILL-6346)</h2>
-<p>Running Drill in a Docker container is the simplest way to start
using Drill; all you need is the Docker client installed on your machine. You
simply run a Docker command, and your Docker client downloads the Drill Docker
image from the apache-drill repository on Docker Hub and then brings up a
container with Apache Drill running in embedded mode. See <a
href="/docs/running-drill-on-docker/">Running Drill on
Docker</a>.</p>
-
-<h2
id="export-and-save-storage-plugin-configurations-drill-4580">Export
and Save Storage Plugin Configurations (DRILL-4580)</h2>
-<p>You can export and save your storage plugin configurations from the
Storage page in the Drill Web UI. See <a
href="/docs/configuring-storage-plugins/#exporting-storage-plugin-configurations">Exporting
Storage Plugin Configurations</a>.</p>
-
-<h2
id="manage-storage-plugin-configurations-in-a-configuration-file-drill-6494">Manage
Storage Plugin Configurations in a Configuration File (DRILL-6494)</h2>
-<p>You can manage storage plugin configurations in the Drill
configuration file, storage-plugins-override.conf. When you provide the
storage plugin configurations in the storage-plugins-override.conf file, Drill
reads the file and configures the plugins during start-up. See <a
href="https://drill.apache.org/docs/configuring-storage-plugins/#configuring-storage-plugins-with-the-storage-plugins-override.conf-file">Configuring
Storage Plugins with the storage-plugins- [...]
-
-<h2
id="query-metadata-in-various-image-formats-drill-4364">Query
Metadata in Various Image Formats (DRILL-4364)</h2>
-<p>The metadata format plugin is useful for querying a large number of
image files stored in a distributed file system. You do not have to build a
metadata repository in advance.<br />
-See <a href="/docs/image-metadata-format-plugin/">Image
Metadata Format Plugin</a>.</p>
-
-<h2
id="set-hive-properties-at-the-session-level-drill-6575">Set Hive
Properties at the Session Level (DRILL-6575)</h2>
-<p>The store.hive.conf.properties option enables you to specify Hive
properties at the session level using the SET command. See <a
href="/docs/hive-storage-plugin/#setting-hive-properties">Setting
Hive Properties</a>.</p>
-
-<p>You can find a complete list of JIRAs resolved in the 1.14.0 release
<a
href="/docs/apache-drill-1-14-0-release-notes/">here</a>.</p>
-
-</description>
- <pubDate>Sun, 05 Aug 2018 00:00:00 +0200</pubDate>
- <link>/blog/2018/08/05/drill-1.14-released/</link>
- <guid isPermaLink="true">/blog/2018/08/05/drill-1.14-released/</guid>
+ <link>http://localhost:4000/blog/2018/10/16/drill-developer-day/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2018/10/16/drill-developer-day/</guid>
<category>blog</category>
diff --git a/index.html b/index.html
index da4e9b0..82bc0b6 100644
--- a/index.html
+++ b/index.html
@@ -203,9 +203,9 @@ $(document).ready(function() {
<div class="news">News:
</div>
- <div><a href="/blog/2021/06/10/drill-1.19-released/">Drill 1.19
Released</a><br/><span>(Laurent Goujon)</span></div>
+ <div><a href="/blog/2021/07/09/streaming-data-from-the-rest-api/">Streaming
data from Drill REST API</a><br/><span>(James Turton)</span></div>
- <div><a href="/blog/2020/09/05/drill-1.18-released/">Drill 1.18
Released</a><br/><span>(Abhishek Girish)</span></div>
+ <div><a href="/blog/2021/06/10/drill-1.19-released/">Drill 1.19
Released</a><br/><span>(Laurent Goujon)</span></div>
</div>
<div class="mw introWrapper">
<table class="intro" cellpadding="0" cellspacing="0" align="center">
diff --git a/zh/README.md b/zh/README.md
index 5d87b02..e230317 100644
--- a/zh/README.md
+++ b/zh/README.md
@@ -144,7 +144,7 @@ The English versions of "site" pages such as index.html are
stored in the root d
## Add translated collection pages
-The English versions of "collection" pages such as the markdown under _docs/
are stored in an en/ subdirectory of the collection root. Create corresponding
translated pages in the collection under `lang-code/` in which you translate
both `title` and `parent` in the front matter but leave the `slug` the same as
the English page and set `lang` to `lang-code`.
+The English versions of "collection" pages such as the markdown under _docs/
are stored in an en/ subdirectory of the collection root. Create corresponding
translated pages in the collection under `lang-code/` in which you translate
both `title` and `parent` in the front matter but leave the `slug` the same as
the English page and set `lang` to `lang-code`. Once you've translated the
`title` of a parent page, you will need to provide files for each of its
children (which can still cont [...]
# Compiling the Website
diff --git a/zh/blog/2021/07/09/streaming-data-from-the-rest-api/index.html
b/zh/blog/2021/07/09/streaming-data-from-the-rest-api/index.html
new file mode 100644
index 0000000..209378d
--- /dev/null
+++ b/zh/blog/2021/07/09/streaming-data-from-the-rest-api/index.html
@@ -0,0 +1,235 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+
+<meta charset="UTF-8">
+<meta name=viewport content="width=device-width, initial-scale=1">
+
+
+<title>Streaming data from Drill REST API - Apache Drill</title>
+
+<link
href="https://maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css";
rel="stylesheet" type="text/css"/>
+<link href='https://fonts.googleapis.com/css?family=PT+Sans' rel='stylesheet'
type='text/css'/>
+<link href="/css/site.css" rel="stylesheet" type="text/css"/>
+
+<link rel="shortcut icon" href="/zh/favicon.ico" type="image/x-icon"/>
+<link rel="icon" href="/zh/favicon.ico" type="image/x-icon"/>
+
+<script
src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js";
language="javascript" type="text/javascript"></script>
+<script
src="https://cdnjs.cloudflare.com/ajax/libs/jquery-easing/1.3/jquery.easing.min.js";
language="javascript" type="text/javascript"></script>
+<script language="javascript" type="text/javascript"
src="/js/modernizr.custom.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/script.js"></script>
+<script language="javascript" type="text/javascript"
src="/js/drill.js"></script>
+
+</head>
+
+
+<body onResize="resized();">
+ <div class="page-wrap">
+ <div class="bui"></div>
+
+<div id="menu" class="mw">
+<ul>
+ <li class='toc-categories'>
+ <a class="expand-toc-icon" href="javascript:void(0);"><i class="fa
fa-bars"></i></a>
+ </li>
+ <li class="logo"><a href="/zh/"></a></li>
+ <li class='expand-menu'>
+ <a href="javascript:void(0);"><span class='menu-text'>Menu</span><span
class='expand-icon'><i class="fa fa-bars"></i></span></a>
+ </li>
+ <li class="clear-float"></li>
+ <li class="nav">
+ <a>Language</a>
+ <ul>
+
+ <li>
+ <a
href="/blog/2021/07/09/streaming-data-from-the-rest-api/" >en</a>
+ </li>
+
+ <li>
+ <a style="font-weight: bold;"
href="/zh/blog/2021/07/09/streaming-data-from-the-rest-api/" >zh</a>
+ </li>
+
+ </ul>
+ </li>
+ <li class="apache-link">
+ <a href="/zh/apacheASF/">Apache</a>
+ </li>
+ <li class="poweredby">
+ <a href="/zh/poweredBy">Powered By</a>
+ </li>
+ <li class="documentation-menu">
+ <a href="/zh/docs/">Documentation</a>
+ <ul>
+
+ <li><a href="/zh/docs/getting-started/">新手开始</a></li>
+
+ <li><a href="/zh/docs/architecture/">Architecture</a></li>
+
+ <li><a href="/zh/docs/tutorials/">Tutorials</a></li>
+
+ <li><a href="/zh/docs/drill-on-yarn/">Drill-on-YARN</a></li>
+
+ <li><a href="/zh/docs/install-drill/">Install Drill</a></li>
+
+ <li><a href="/zh/docs/configure-drill/">Configure Drill</a></li>
+
+ <li><a href="/zh/docs/connect-a-data-source/">Connect a Data
Source</a></li>
+
+ <li><a href="/zh/docs/odbc-jdbc-interfaces/">ODBC/JDBC
Interfaces</a></li>
+
+ <li><a href="/zh/docs/query-data/">Query Data</a></li>
+
+ <li><a href="/zh/docs/performance-tuning/">Performance Tuning</a></li>
+
+ <li><a href="/zh/docs/log-and-debug/">Log and Debug</a></li>
+
+ <li><a href="/zh/docs/sql-reference/">SQL Reference</a></li>
+
+ <li><a href="/zh/docs/data-sources-and-file-formats/">Data Sources and
File Formats</a></li>
+
+ <li><a href="/zh/docs/develop-custom-functions/">Develop Custom
Functions</a></li>
+
+ <li><a href="/zh/docs/troubleshooting/">Troubleshooting</a></li>
+
+ <li><a href="/zh/docs/developer-information/">Developer
Information</a></li>
+
+ <li><a href="/zh/docs/release-notes/">Release Notes</a></li>
+
+ <li><a href="/zh/docs/sample-datasets/">Sample Datasets</a></li>
+
+ <li><a href="/zh/docs/project-bylaws/">Project Bylaws</a></li>
+
+ <li><a href="/zh/docs/ecosystem/">Ecosystem</a></li>
+
+ </ul>
+ </li>
+ <li class='nav'>
+ <a href="/zh/community-resources/">Community</a>
+ <ul>
+ <li><a href="/zh/team/">Team</a></li>
+ <li><a href="/zh/mailinglists/">Mailing Lists</a></li>
+ <li><a href="/zh/community-resources/">Community Resources</a></li>
+ </ul>
+ </li>
+ <li class='nav'><a href="/zh/faq/">FAQ</a></li>
+ <li class='nav'><a href="/zh/blog/">Blog</a></li>
+ <li class="social-menu-item"><a href="https://twitter.com/apachedrill";
title="apachedrill on twitter" target="_blank"><img
src="/images/twitter_32_26_white.png" alt="twitter logo" align="center"></a>
</li>
+ <li class="social-menu-item"><a
href="https://join.slack.com/t/apache-drill/shared_invite/enQtNTQ4MjM1MDA3MzQ2LTJlYmUxMTRkMmUwYmQ2NTllYmFmMjU4MDk0NjYwZjBmYjg0MDZmOTE2ZDg0ZjBlYmI3Yjc4Y2I2NTQyNGVlZTc";
title="Apache Drill Slack channels"
+ target="_blank"><img src="/images/slack-logo.svg" alt="Slack logo"
align="center"></a> </li>
+ <li class='search-bar'>
+ <form id="drill-search-form">
+ <input type="text" placeholder="Search Apache Drill"
id="drill-search-term" />
+ <button type="submit">
+ <i class="fa fa-search"></i>
+ </button>
+ </form>
+ </li>
+ <li class="d">
+ <a href="/zh/download/">
+ <i class="fa fa-cloud-download"></i> Download
+ </a>
+ </li>
+</ul>
+</div>
+
+ <link href="/css/content.css" rel="stylesheet" type="text/css">
+
+<div class="post int_text">
+ <header class="post-header">
+ <div class="int_title">
+ <h1 class="post-title">Streaming data from Drill REST API</h1>
+ </div>
+ <p class="post-meta">
+
+
+
+ <strong>Author:</strong> James Turton (Committer, Apache Software
Foundation)<br />
+
+<strong>Date:</strong> Jul 9, 2021
+</p>
+ </header>
+ <div class="addthis_sharing_toolbox"></div>
+
+ <article class="post-content">
+ <p>Anyone who’s used a UNIX pipe, or even just watched something on
Netflix, is at least a little familiar with the idea of processing data in a
streaming fashion. While your data are in small volume compared to available
memory and I/O speeds, streaming is something you can afford to dispense with.
But when you cannot fit an entire dataset in RAM, or when if you have to
download an entire 4K movie before you can start playing it, then streaming
data processing can make a game chan [...]
+
+<p>With the relase of version 1.19, Drill will stream JSON query result data
over an HTTP response to the client that initiated the query using the REST
API. And if anything can easily get big compared to your available RAM or
network speed, it’s query results coming back from Drill. It’s important to
note here that JSON over HTTP is never going to be the most <em>efficient</em>
way to move big data around<sup id="fnref:1" role="doc-noteref"><a href="#fn:1"
class="footnote">1</a></sup> [...]
+
+<p>Where JSON and HTTP <em>do</em> win is universality: today it’s hard to
imagine a client hardware and software stack that doesn’t provide JSON and HTTP
out of the box with minimal effort. So it’s important that they work as well
as is possible, in spite of the alternatives that exist. The new streaming
query results delivery on the server side means that Drill’s heap memory isn’t
pressurised by having to buffer entire result sets before it starts to transmit
them over the network. [...]
+
+<p>To fully realise the benefits of streaming query result data, clients can
<em>themselves</em> operate on the HTTP response they receive in a streaming
fashion, thereby potentially starting to process records before Drill has even
finished materialising them and avoiding holding the full set in memory if they
choose. At the transport level (when there are enough results to warrant it),
the HTTP response headers from the Drill REST API will include a <code
class="language-plaintext hig [...]
+
+<p>If you set out to develop a streaming HTTP client for the Drill REST API,
do take note that the schema of the JSON query result is <em>not</em> a <a
href="https://en.wikipedia.org/wiki/JSON_streaming";>streaming JSON format</a>
like “JSON lines”. This means that you must be careful about which JSON
objects you parse entirely in a single call, particularly avoiding any parent
of the <code class="language-plaintext highlighter-rouge">rows</code> property
in the query result which would [...]
+
+<p>In closing, and for a bit of fun, here’s the log from a short IPython
session where I use sqlalchemy-drill to run <code class="language-plaintext
highlighter-rouge">SELECT *</code> on a remote 17 billion record table over a
0.5 Mbit/s link and start scanning through (a steady trickle of) rows in
seconds.</p>
+
+<pre><code class="language-ipython">In [4]: r = engine.execute('select
count(*) from dfs.ws.big_table')
+INFO:drilldbapi:received Drill query ID 1f211888-cc20-fe6f-69d1-6584c5caa2df.
+INFO:drilldbapi:opened a row data stream of 1 columns.
+
+In [5]: next(r)
+Out[5]: (17437571247,)
+
+In [6]: r = engine.execute('select * from dfs.ws.big_table')
+INFO:drilldbapi:received Drill query ID 1f211838-73df-1506-a74e-f5695f6b0ff5.
+INFO:drilldbapi:opened a row data stream of 21 columns.
+
+In [7]: while True:
+ ...: _ = next(r)
+ ...:
+ INFO:drilldbapi:streamed 10000 rows.
+ INFO:drilldbapi:streamed 20000 rows.
+ INFO:drilldbapi:streamed 30000 rows.
+ INFO:drilldbapi:streamed 40000 rows.
+</code></pre>
+
+<div class="footnotes" role="doc-endnotes">
+ <ol>
+ <li id="fn:1" role="doc-endnote">
+ <p>Some mitigation is possible. JSON representations of tabular big
data are typically extremely compressible and you can reduce the bytes sent
over the network by 10-20x by enabling HTTP response compression on a web
server like Apache placed in front of the Drill REST API. <a href="#fnref:1"
class="reversefootnote" role="doc-backlink">↩</a></p>
+ </li>
+ </ol>
+</div>
+
+ </article>
+ <div id="disqus_thread"></div>
+ <script type="text/javascript">
+ /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE
* * */
+ var disqus_shortname = 'drill'; // required: replace example with your
forum shortname
+
+ /* * * DON'T EDIT BELOW THIS LINE * * */
+ (function() {
+ var dsq = document.createElement('script'); dsq.type =
'text/javascript'; dsq.async = true;
+ dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
+ (document.getElementsByTagName('head')[0] ||
document.getElementsByTagName('body')[0]).appendChild(dsq);
+ })();
+ </script>
+ <noscript>Please enable JavaScript to view the <a
href="http://disqus.com/?ref_noscript";>comments powered by
Disqus.</a></noscript>
+
+</div>
+<script type="text/javascript"
src="https://s7.addthis.com/js/300/addthis_widget.js#pubid=ra-548b2caa33765e8d";
async="async"></script>
+
+ </div>
+ <p class="push"></p>
+<div id="footer" class="mw">
+<div class="wrapper">
+Copyright © 2012-2020 The Apache Software Foundation, licensed under the
Apache License, Version 2.0.<br>
+Apache and the Apache feather logo are trademarks of The Apache Software
Foundation. Other names appearing on the site may be trademarks of their
respective owners.<br/><br/>
+</div>
+</div>
+
+ <script>
+(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+ga('create', 'UA-53379651-1', 'auto');
+ga('send', 'pageview');
+</script>
+<script type="text/javascript"
src="https://s7.addthis.com/js/300/addthis_widget.js#pubid=ra-548b2caa33765e8d";
async="async"></script>
+
+</body>
+</html>
diff --git a/zh/blog/index.html b/zh/blog/index.html
index cd8ab0a..ff02bf0 100644
--- a/zh/blog/index.html
+++ b/zh/blog/index.html
@@ -140,6 +140,11 @@
</div>
<div class="int_text" align="left"><!-- previously: site.posts -->
+<p><a class="post-link"
href="/zh/blog/2021/07/09/streaming-data-from-the-rest-api/">Streaming data
from Drill REST API</a><br/>
+<span class="post-date">Posted on Jul 9, 2021
+by James Turton</span>
+<br/>The release of Apache Drill 1.19 saw a major change under the hood of
Drill's REST API with the introduction of a streaming data path for query
results moving from Drill and over the network to the initiating client. The
result is better memory utilisation, less blocking and a more reliable API.</p>
+<!-- previously: site.posts -->
<p><a class="post-link" href="/zh/blog/2021/06/10/drill-1.19-released/">Drill
1.19 Released</a><br/>
<span class="post-date">Posted on Jun 10, 2021
by Laurent Goujon</span>
diff --git a/zh/feed.xml b/zh/feed.xml
index 645d54e..90de618 100644
--- a/zh/feed.xml
+++ b/zh/feed.xml
@@ -4,13 +4,64 @@
<title>Apache Drill - Schema-free SQL for Hadoop, NoSQL and Cloud
Storage</title>
<description>The official user documentation for Apache Drill.
</description>
- <link>/</link>
- <atom:link href="/zh/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Thu, 08 Jul 2021 14:57:25 +0200</pubDate>
- <lastBuildDate>Thu, 08 Jul 2021 14:57:25 +0200</lastBuildDate>
+ <link>http://localhost:4000/</link>
+ <atom:link href="http://localhost:4000/zh/feed.xml"; rel="self"
type="application/rss+xml"/>
+ <pubDate>Fri, 09 Jul 2021 15:37:50 +0200</pubDate>
+ <lastBuildDate>Fri, 09 Jul 2021 15:37:50 +0200</lastBuildDate>
<generator>Jekyll v3.9.0</generator>
<item>
+ <title>Streaming data from Drill REST API</title>
+ <description><p>Anyone who’s used a UNIX pipe, or even just
watched something on Netflix, is at least a little familiar with the idea of
processing data in a streaming fashion. While your data are in small volume
compared to available memory and I/O speeds, streaming is something you can
afford to dispense. But if you cannot fit an entire dataset in RAM, or when if
you have to download an entire 4K movie before you can start playing it, then
streaming can make a game chan [...]
+
+<p>With the relase of version 1.19, Drill will stream JSON query result
data over an HTTP response to the client that initiated the query using the
REST API. And if anything can easily get big compared to your available RAM or
network speeds, its query results coming out of Drill. It’s important to note
here that JSON over HTTP is never going to be the most efficient way to move
big data around<sup id="fnref:1"
role="doc-noteref"><a href="#fn:1&q [...]
+
+<p>Where JSON and HTTP <em>do</em> win on universality:
today it’s hard to image a client hardware and software stack that doesn’t
provide JSON and HTTP out of the box with minimal effort. So it’s important
that they work as well as is possible. The new streaming query results
materialisation on the server side means that Drill’s heap memory isn’t
pressurised by having to buffer entire result sets before starting to transmit
them over the network. Even existing REST [...]
+
+<p>To fully realise the benefits of streaming query result data, clients
can <em>themselves</em> operate on the HTTP response they receive
in a streaming fashion, thereby potentially starting to process records before
Drill has even finished materialsing them. At the transport level (when there
are enough results to warrant it), the HTTP response headers from the Drill
REST API will include a <code class="language-plaintext
highlighter-rouge">Transfer-E [...]
+
+<p>Streaming HTTP client developers should note well that the schema of
JSON query result is <em>not</em> a <a
href="https://en.wikipedia.org/wiki/JSON_streaming">streaming JSON
format</a> like “JSON lines”. This means that you should take care to
avoid parsing any entire object which is a parent of the <code
class="language-plaintext highlighter-rouge">rows</code>
property in the query result, thereby parsing the essentailly [...]
+
+<p>In closing, and for a bit of fun, here’s the log from a short IPython
session where I use sqlalchemy-drill to run <code
class="language-plaintext highlighter-rouge">SELECT *</code>
on a remote 17 billion record table over a 0.5 Mbit/s link and start receiving
(a steady trickle of) rows in seconds.</p>
+
+<pre><code class="language-ipython">In [4]: r =
engine.execute('select count(*) from dfs.ws.big_table')
+INFO:drilldbapi:received Drill query ID 1f211888-cc20-fe6f-69d1-6584c5caa2df.
+INFO:drilldbapi:opened a row data stream of 1 columns.
+
+In [5]: next(r)
+Out[5]: (17437571247,)
+
+In [6]: r = engine.execute('select * from dfs.ws.big_table')
+INFO:drilldbapi:received Drill query ID 1f211838-73df-1506-a74e-f5695f6b0ff5.
+INFO:drilldbapi:opened a row data stream of 21 columns.
+
+In [7]: while True:
+ ...: _ = next(r)
+ ...:
+ INFO:drilldbapi:streamed 10000 rows.
+ INFO:drilldbapi:streamed 20000 rows.
+ INFO:drilldbapi:streamed 30000 rows.
+ INFO:drilldbapi:streamed 40000 rows.
+</code></pre>
+
+<div class="footnotes" role="doc-endnotes">
+ <ol>
+ <li id="fn:1" role="doc-endnote">
+ <p>JSON representations of tabular big data are typically
extremely compressible and you can reduce the bytes sent over the network by
10-20x by enabling HTTP response compression on a web server like Apache placed
in front of the Drill REST API. <a href="#fnref:1"
class="reversefootnote"
role="doc-backlink">&#8617;</a></p>
+ </li>
+ </ol>
+</div>
+</description>
+ <pubDate>Fri, 09 Jul 2021 00:00:00 +0200</pubDate>
+
<link>http://localhost:4000/blog/2021/07/09/streaming-data-from-the-rest-api/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2021/07/09/streaming-data-from-the-rest-api/</guid>
+
+
+ <category>blog</category>
+
+ </item>
+
+ <item>
<title>Drill 1.19 Released</title>
<description><p>今天, we’re happy to announce the availability of
Drill 1.19.0. You can download it <a
href="https://drill.apache.org/download/">here</a>.</p>
@@ -40,8 +91,8 @@
<p>You can find a complete list of improvements and JIRAs resolved in
the 1.19.0 release <a
href="/docs/apache-drill-1-19-0-release-notes/">here</a>.</p>
</description>
<pubDate>Thu, 10 Jun 2021 00:00:00 +0200</pubDate>
- <link>/blog/2021/06/10/drill-1.19-released/</link>
- <guid isPermaLink="true">/blog/2021/06/10/drill-1.19-released/</guid>
+ <link>http://localhost:4000/blog/2021/06/10/drill-1.19-released/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2021/06/10/drill-1.19-released/</guid>
<category>blog</category>
@@ -64,8 +115,8 @@
<p>You can find a complete list of improvements and JIRAs resolved in
the 1.18.0 release <a
href="/docs/apache-drill-1-18-0-release-notes/">here</a>.</p>
</description>
<pubDate>Sat, 05 Sep 2020 00:00:00 +0200</pubDate>
- <link>/blog/2020/09/05/drill-1.18-released/</link>
- <guid isPermaLink="true">/blog/2020/09/05/drill-1.18-released/</guid>
+ <link>http://localhost:4000/blog/2020/09/05/drill-1.18-released/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2020/09/05/drill-1.18-released/</guid>
<category>blog</category>
@@ -106,8 +157,8 @@
</description>
<pubDate>Thu, 26 Dec 2019 00:00:00 +0200</pubDate>
- <link>/blog/2019/12/26/drill-1.17-released/</link>
- <guid isPermaLink="true">/blog/2019/12/26/drill-1.17-released/</guid>
+ <link>http://localhost:4000/blog/2019/12/26/drill-1.17-released/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2019/12/26/drill-1.17-released/</guid>
<category>blog</category>
@@ -119,8 +170,8 @@
<description>
</description>
<pubDate>Thu, 02 May 2019 00:00:00 +0200</pubDate>
- <link>/blog/2019/05/02/drill-user-meetup/</link>
- <guid isPermaLink="true">/blog/2019/05/02/drill-user-meetup/</guid>
+ <link>http://localhost:4000/blog/2019/05/02/drill-user-meetup/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2019/05/02/drill-user-meetup/</guid>
<category>blog</category>
@@ -155,8 +206,8 @@
</description>
<pubDate>Thu, 02 May 2019 00:00:00 +0200</pubDate>
- <link>/blog/2019/05/02/drill-1.16-released/</link>
- <guid isPermaLink="true">/blog/2019/05/02/drill-1.16-released/</guid>
+ <link>http://localhost:4000/blog/2019/05/02/drill-1.16-released/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2019/05/02/drill-1.16-released/</guid>
<category>blog</category>
@@ -188,8 +239,8 @@
</description>
<pubDate>Mon, 31 Dec 2018 00:00:00 +0200</pubDate>
- <link>/blog/2018/12/31/drill-1.15-released/</link>
- <guid isPermaLink="true">/blog/2018/12/31/drill-1.15-released/</guid>
+ <link>http://localhost:4000/blog/2018/12/31/drill-1.15-released/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2018/12/31/drill-1.15-released/</guid>
<category>blog</category>
@@ -223,8 +274,8 @@
</ul>
</description>
<pubDate>Sat, 01 Dec 2018 00:00:00 +0200</pubDate>
- <link>/blog/2018/12/01/learning-apache-drill-book/</link>
- <guid
isPermaLink="true">/blog/2018/12/01/learning-apache-drill-book/</guid>
+
<link>http://localhost:4000/blog/2018/12/01/learning-apache-drill-book/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2018/12/01/learning-apache-drill-book/</guid>
<category>blog</category>
@@ -236,8 +287,8 @@
<description>
</description>
<pubDate>Tue, 16 Oct 2018 21:18:04 +0200</pubDate>
- <link>/blog/2018/10/16/drill-user-meetup/</link>
- <guid isPermaLink="true">/blog/2018/10/16/drill-user-meetup/</guid>
+ <link>http://localhost:4000/blog/2018/10/16/drill-user-meetup/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2018/10/16/drill-user-meetup/</guid>
<category>blog</category>
@@ -249,42 +300,8 @@
<description>
</description>
<pubDate>Tue, 16 Oct 2018 21:18:04 +0200</pubDate>
- <link>/blog/2018/10/16/drill-developer-day/</link>
- <guid isPermaLink="true">/blog/2018/10/16/drill-developer-day/</guid>
-
-
- <category>blog</category>
-
- </item>
-
- <item>
- <title>Drill 1.14 Released</title>
- <description><p>Today, we’re happy to announce the availability
of Drill 1.14.0. You can download it <a
href="https://drill.apache.org/download/">here</a>.</p>
-
-<p>The release provides the following bug fixes and
improvements:</p>
-
-<h2 id="run-drill-in-a-docker-container-drill-6346">Run Drill
in a Docker Container (DRILL-6346)</h2>
-<p>Running Drill in a Docker container is the simplest way to start
using Drill; all you need is the Docker client installed on your machine. You
simply run a Docker command, and your Docker client downloads the Drill Docker
image from the apache-drill repository on Docker Hub and then brings up a
container with Apache Drill running in embedded mode. See <a
href="/docs/running-drill-on-docker/">Running Drill on
Docker</a>.</p>
-
-<h2
id="export-and-save-storage-plugin-configurations-drill-4580">Export
and Save Storage Plugin Configurations (DRILL-4580)</h2>
-<p>You can export and save your storage plugin configurations from the
Storage page in the Drill Web UI. See <a
href="/docs/configuring-storage-plugins/#exporting-storage-plugin-configurations">Exporting
Storage Plugin Configurations</a>.</p>
-
-<h2
id="manage-storage-plugin-configurations-in-a-configuration-file-drill-6494">Manage
Storage Plugin Configurations in a Configuration File (DRILL-6494)</h2>
-<p>You can manage storage plugin configurations in the Drill
configuration file, storage-plugins-override.conf. When you provide the
storage plugin configurations in the storage-plugins-override.conf file, Drill
reads the file and configures the plugins during start-up. See <a
href="https://drill.apache.org/docs/configuring-storage-plugins/#configuring-storage-plugins-with-the-storage-plugins-override.conf-file">Configuring
Storage Plugins with the storage-plugins- [...]
-
-<h2
id="query-metadata-in-various-image-formats-drill-4364">Query
Metadata in Various Image Formats (DRILL-4364)</h2>
-<p>The metadata format plugin is useful for querying a large number of
image files stored in a distributed file system. You do not have to build a
metadata repository in advance.<br />
-See <a href="/docs/image-metadata-format-plugin/">Image
Metadata Format Plugin</a>.</p>
-
-<h2
id="set-hive-properties-at-the-session-level-drill-6575">Set Hive
Properties at the Session Level (DRILL-6575)</h2>
-<p>The store.hive.conf.properties option enables you to specify Hive
properties at the session level using the SET command. See <a
href="/docs/hive-storage-plugin/#setting-hive-properties">Setting
Hive Properties</a>.</p>
-
-<p>You can find a complete list of JIRAs resolved in the 1.14.0 release
<a
href="/docs/apache-drill-1-14-0-release-notes/">here</a>.</p>
-
-</description>
- <pubDate>Sun, 05 Aug 2018 00:00:00 +0200</pubDate>
- <link>/blog/2018/08/05/drill-1.14-released/</link>
- <guid isPermaLink="true">/blog/2018/08/05/drill-1.14-released/</guid>
+ <link>http://localhost:4000/blog/2018/10/16/drill-developer-day/</link>
+ <guid
isPermaLink="true">http://localhost:4000/blog/2018/10/16/drill-developer-day/</guid>
<category>blog</category>
diff --git a/zh/index.html b/zh/index.html
index d23a962..51bc664 100644
--- a/zh/index.html
+++ b/zh/index.html
@@ -202,9 +202,9 @@
<div class="news">News:
</div>
- <div><a href="/zh/blog/2021/06/10/drill-1.19-released/">Drill 1.19
Released</a><br/><span>(Laurent Goujon)</span></div>
+ <div><a
href="/zh/blog/2021/07/09/streaming-data-from-the-rest-api/">Streaming data
from Drill REST API</a><br/><span>(James Turton)</span></div>
- <div><a href="/zh/blog/2020/09/05/drill-1.18-released/">Drill 1.18
Released</a><br/><span>(Abhishek Girish)</span></div>
+ <div><a href="/zh/blog/2021/06/10/drill-1.19-released/">Drill 1.19
Released</a><br/><span>(Laurent Goujon)</span></div>
</div>
<div class="mw introWrapper">
<table class="intro" cellpadding="0" cellspacing="0" align="center">
