http://git-wip-us.apache.org/repos/asf/spark-website/blob/d2bcf185/site/docs/2.1.0/structured-streaming-kafka-integration.html ---------------------------------------------------------------------- diff --git a/site/docs/2.1.0/structured-streaming-kafka-integration.html b/site/docs/2.1.0/structured-streaming-kafka-integration.html index 5ca9259..7d2254f 100644 --- a/site/docs/2.1.0/structured-streaming-kafka-integration.html +++ b/site/docs/2.1.0/structured-streaming-kafka-integration.html @@ -144,7 +144,7 @@ application. See the <a href="#deploying">Deploying</a> subsection below.</p> <div class="codetabs"> <div data-lang="scala"> - <div class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// Subscribe to 1 topic</span> + <figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span></span><span class="c1">// Subscribe to 1 topic</span> <span class="k">val</span> <span class="n">ds1</span> <span class="k">=</span> <span class="n">spark</span> <span class="o">.</span><span class="n">readStream</span> <span class="o">.</span><span class="n">format</span><span class="o">(</span><span class="s">"kafka"</span><span class="o">)</span> @@ -172,12 +172,12 @@ application. See the <a href="#deploying">Deploying</a> subsection below.</p> <span class="o">.</span><span class="n">option</span><span class="o">(</span><span class="s">"subscribePattern"</span><span class="o">,</span> <span class="s">"topic.*"</span><span class="o">)</span> <span class="o">.</span><span class="n">load</span><span class="o">()</span> <span class="n">ds3</span><span class="o">.</span><span class="n">selectExpr</span><span class="o">(</span><span class="s">"CAST(key AS STRING)"</span><span class="o">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="o">)</span> - <span class="o">.</span><span class="n">as</span><span class="o">[(</span><span class="kt">String</span>, <span class="kt">String</span><span class="o">)]</span></code></pre></div> + <span class="o">.</span><span class="n">as</span><span class="o">[(</span><span class="kt">String</span>, <span class="kt">String</span><span class="o">)]</span></code></pre></figure> </div> <div data-lang="java"> - <div class="highlight"><pre><code class="language-java" data-lang="java"><span class="c1">// Subscribe to 1 topic</span> + <figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="c1">// Subscribe to 1 topic</span> <span class="n">Dataset</span><span class="o"><</span><span class="n">Row</span><span class="o">></span> <span class="n">ds1</span> <span class="o">=</span> <span class="n">spark</span> <span class="o">.</span><span class="na">readStream</span><span class="o">()</span> <span class="o">.</span><span class="na">format</span><span class="o">(</span><span class="s">"kafka"</span><span class="o">)</span> @@ -202,43 +202,43 @@ application. See the <a href="#deploying">Deploying</a> subsection below.</p> <span class="o">.</span><span class="na">option</span><span class="o">(</span><span class="s">"kafka.bootstrap.servers"</span><span class="o">,</span> <span class="s">"host1:port1,host2:port2"</span><span class="o">)</span> <span class="o">.</span><span class="na">option</span><span class="o">(</span><span class="s">"subscribePattern"</span><span class="o">,</span> <span class="s">"topic.*"</span><span class="o">)</span> <span class="o">.</span><span class="na">load</span><span class="o">()</span> -<span class="n">ds3</span><span class="o">.</span><span class="na">selectExpr</span><span class="o">(</span><span class="s">"CAST(key AS STRING)"</span><span class="o">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="o">)</span></code></pre></div> +<span class="n">ds3</span><span class="o">.</span><span class="na">selectExpr</span><span class="o">(</span><span class="s">"CAST(key AS STRING)"</span><span class="o">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="o">)</span></code></pre></figure> </div> <div data-lang="python"> - <div class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># Subscribe to 1 topic</span> + <figure class="highlight"><pre><code class="language-python" data-lang="python"><span></span><span class="c1"># Subscribe to 1 topic</span> <span class="n">ds1</span> <span class="o">=</span> <span class="n">spark</span> <span class="o">.</span><span class="n">readStream</span><span class="p">()</span> - <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s">"kafka"</span><span class="p">)</span> - <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s">"host1:port1,host2:port2"</span><span class="p">)</span> - <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"subscribe"</span><span class="p">,</span> <span class="s">"topic1"</span><span class="p">)</span> + <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s2">"kafka"</span><span class="p">)</span> + <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s2">"host1:port1,host2:port2"</span><span class="p">)</span> + <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"subscribe"</span><span class="p">,</span> <span class="s2">"topic1"</span><span class="p">)</span> <span class="o">.</span><span class="n">load</span><span class="p">()</span> -<span class="n">ds1</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="p">)</span> +<span class="n">ds1</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s2">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s2">"CAST(value AS STRING)"</span><span class="p">)</span> -<span class="c"># Subscribe to multiple topics</span> +<span class="c1"># Subscribe to multiple topics</span> <span class="n">ds2</span> <span class="o">=</span> <span class="n">spark</span> <span class="o">.</span><span class="n">readStream</span> - <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s">"kafka"</span><span class="p">)</span> - <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s">"host1:port1,host2:port2"</span><span class="p">)</span> - <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"subscribe"</span><span class="p">,</span> <span class="s">"topic1,topic2"</span><span class="p">)</span> + <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s2">"kafka"</span><span class="p">)</span> + <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s2">"host1:port1,host2:port2"</span><span class="p">)</span> + <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"subscribe"</span><span class="p">,</span> <span class="s2">"topic1,topic2"</span><span class="p">)</span> <span class="o">.</span><span class="n">load</span><span class="p">()</span> -<span class="n">ds2</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="p">)</span> +<span class="n">ds2</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s2">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s2">"CAST(value AS STRING)"</span><span class="p">)</span> -<span class="c"># Subscribe to a pattern</span> +<span class="c1"># Subscribe to a pattern</span> <span class="n">ds3</span> <span class="o">=</span> <span class="n">spark</span> <span class="o">.</span><span class="n">readStream</span><span class="p">()</span> - <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s">"kafka"</span><span class="p">)</span> - <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s">"host1:port1,host2:port2"</span><span class="p">)</span> - <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s">"subscribePattern"</span><span class="p">,</span> <span class="s">"topic.*"</span><span class="p">)</span> + <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="s2">"kafka"</span><span class="p">)</span> + <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"kafka.bootstrap.servers"</span><span class="p">,</span> <span class="s2">"host1:port1,host2:port2"</span><span class="p">)</span> + <span class="o">.</span><span class="n">option</span><span class="p">(</span><span class="s2">"subscribePattern"</span><span class="p">,</span> <span class="s2">"topic.*"</span><span class="p">)</span> <span class="o">.</span><span class="n">load</span><span class="p">()</span> -<span class="n">ds3</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s">"CAST(value AS STRING)"</span><span class="p">)</span></code></pre></div> +<span class="n">ds3</span><span class="o">.</span><span class="n">selectExpr</span><span class="p">(</span><span class="s2">"CAST(key AS STRING)"</span><span class="p">,</span> <span class="s2">"CAST(value AS STRING)"</span><span class="p">)</span></code></pre></figure> </div> </div> -<p>Each row in the source has the following schema:</p> -<table class="table"> +<p>Each row in the source has the following schema: +<table class="table"></p> <tr><th>Column</th><th>Type</th></tr> <tr> <td>key</td> @@ -268,7 +268,7 @@ application. See the <a href="#deploying">Deploying</a> subsection below.</p> <td>timestampType</td> <td>int</td> </tr> -</table> +<p></table></p> <p>The following options must be set for the Kafka source.</p>
--------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
