http://git-wip-us.apache.org/repos/asf/arrow-site/blob/4d4a3202/docs/python/getting_involved.html ---------------------------------------------------------------------- diff --git a/docs/python/getting_involved.html b/docs/python/getting_involved.html index efa1953..5cf11e6 100644 --- a/docs/python/getting_involved.html +++ b/docs/python/getting_involved.html @@ -72,7 +72,8 @@ <li class="toctree-l1"><a class="reference internal" href="memory.html">Memory and IO Interfaces</a></li> <li class="toctree-l1"><a class="reference internal" href="data.html">In-Memory Data Model</a></li> <li class="toctree-l1"><a class="reference internal" href="ipc.html">IPC: Fast Streaming and Serialization</a></li> -<li class="toctree-l1"><a class="reference internal" href="filesystems.html">Filesystem Interfaces</a></li> +<li class="toctree-l1"><a class="reference internal" href="filesystems.html">File System Interfaces</a></li> +<li class="toctree-l1"><a class="reference internal" href="plasma.html">The Plasma In-Memory Object Store</a></li> <li class="toctree-l1"><a class="reference internal" href="pandas.html">Using PyArrow with pandas</a></li> <li class="toctree-l1"><a class="reference internal" href="parquet.html">Reading and Writing the Apache Parquet Format</a></li> <li class="toctree-l1"><a class="reference internal" href="api.html">API Reference</a></li>
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/4d4a3202/docs/python/index.html ---------------------------------------------------------------------- diff --git a/docs/python/index.html b/docs/python/index.html index 079d907..bb5ecf7 100644 --- a/docs/python/index.html +++ b/docs/python/index.html @@ -72,7 +72,8 @@ <li class="toctree-l1"><a class="reference internal" href="memory.html">Memory and IO Interfaces</a></li> <li class="toctree-l1"><a class="reference internal" href="data.html">In-Memory Data Model</a></li> <li class="toctree-l1"><a class="reference internal" href="ipc.html">IPC: Fast Streaming and Serialization</a></li> -<li class="toctree-l1"><a class="reference internal" href="filesystems.html">Filesystem Interfaces</a></li> +<li class="toctree-l1"><a class="reference internal" href="filesystems.html">File System Interfaces</a></li> +<li class="toctree-l1"><a class="reference internal" href="plasma.html">The Plasma In-Memory Object Store</a></li> <li class="toctree-l1"><a class="reference internal" href="pandas.html">Using PyArrow with pandas</a></li> <li class="toctree-l1"><a class="reference internal" href="parquet.html">Reading and Writing the Apache Parquet Format</a></li> <li class="toctree-l1"><a class="reference internal" href="api.html">API Reference</a></li> @@ -166,7 +167,6 @@ structures.</p> <li class="toctree-l2"><a class="reference internal" href="memory.html#memory-pools">Memory Pools</a></li> <li class="toctree-l2"><a class="reference internal" href="memory.html#on-disk-and-memory-mapped-files">On-Disk and Memory Mapped Files</a></li> <li class="toctree-l2"><a class="reference internal" href="memory.html#in-memory-reading-and-writing">In-Memory Reading and Writing</a></li> -<li class="toctree-l2"><a class="reference internal" href="memory.html#hadoop-filesystem">Hadoop Filesystem</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="data.html">In-Memory Data Model</a><ul> @@ -184,10 +184,15 @@ structures.</p> <li class="toctree-l2"><a class="reference internal" href="ipc.html#reading-from-stream-and-file-format-for-pandas">Reading from Stream and File Format for pandas</a></li> </ul> </li> -<li class="toctree-l1"><a class="reference internal" href="filesystems.html">Filesystem Interfaces</a><ul> +<li class="toctree-l1"><a class="reference internal" href="filesystems.html">File System Interfaces</a><ul> <li class="toctree-l2"><a class="reference internal" href="filesystems.html#hadoop-file-system-hdfs">Hadoop File System (HDFS)</a></li> </ul> </li> +<li class="toctree-l1"><a class="reference internal" href="plasma.html">The Plasma In-Memory Object Store</a><ul> +<li class="toctree-l2"><a class="reference internal" href="plasma.html#the-plasma-api">The Plasma API</a></li> +<li class="toctree-l2"><a class="reference internal" href="plasma.html#using-arrow-and-pandas-with-plasma">Using Arrow and Pandas with Plasma</a></li> +</ul> +</li> <li class="toctree-l1"><a class="reference internal" href="pandas.html">Using PyArrow with pandas</a><ul> <li class="toctree-l2"><a class="reference internal" href="pandas.html#dataframes">DataFrames</a></li> <li class="toctree-l2"><a class="reference internal" href="pandas.html#series">Series</a></li> @@ -206,13 +211,15 @@ structures.</p> <li class="toctree-l1"><a class="reference internal" href="api.html">API Reference</a><ul> <li class="toctree-l2"><a class="reference internal" href="api.html#type-and-schema-factory-functions">Type and Schema Factory Functions</a></li> <li class="toctree-l2"><a class="reference internal" href="api.html#scalar-value-types">Scalar Value Types</a></li> -<li class="toctree-l2"><a class="reference internal" href="api.html#array-types-and-constructors">Array Types and Constructors</a></li> +<li class="toctree-l2"><a class="reference internal" href="api.html#array-types">Array Types</a></li> <li class="toctree-l2"><a class="reference internal" href="api.html#tables-and-record-batches">Tables and Record Batches</a></li> <li class="toctree-l2"><a class="reference internal" href="api.html#tensor-type-and-functions">Tensor type and Functions</a></li> <li class="toctree-l2"><a class="reference internal" href="api.html#input-output-and-shared-memory">Input / Output and Shared Memory</a></li> +<li class="toctree-l2"><a class="reference internal" href="api.html#file-systems">File Systems</a></li> <li class="toctree-l2"><a class="reference internal" href="api.html#interprocess-communication-and-messaging">Interprocess Communication and Messaging</a></li> <li class="toctree-l2"><a class="reference internal" href="api.html#memory-pools">Memory Pools</a></li> <li class="toctree-l2"><a class="reference internal" href="api.html#type-classes">Type Classes</a></li> +<li class="toctree-l2"><a class="reference internal" href="api.html#in-memory-object-store">In-Memory Object Store</a></li> <li class="toctree-l2"><a class="reference internal" href="api.html#apache-parquet">Apache Parquet</a></li> </ul> </li> http://git-wip-us.apache.org/repos/asf/arrow-site/blob/4d4a3202/docs/python/install.html ---------------------------------------------------------------------- diff --git a/docs/python/install.html b/docs/python/install.html index 663133f..9918a3a 100644 --- a/docs/python/install.html +++ b/docs/python/install.html @@ -73,7 +73,8 @@ <li class="toctree-l1"><a class="reference internal" href="memory.html">Memory and IO Interfaces</a></li> <li class="toctree-l1"><a class="reference internal" href="data.html">In-Memory Data Model</a></li> <li class="toctree-l1"><a class="reference internal" href="ipc.html">IPC: Fast Streaming and Serialization</a></li> -<li class="toctree-l1"><a class="reference internal" href="filesystems.html">Filesystem Interfaces</a></li> +<li class="toctree-l1"><a class="reference internal" href="filesystems.html">File System Interfaces</a></li> +<li class="toctree-l1"><a class="reference internal" href="plasma.html">The Plasma In-Memory Object Store</a></li> <li class="toctree-l1"><a class="reference internal" href="pandas.html">Using PyArrow with pandas</a></li> <li class="toctree-l1"><a class="reference internal" href="parquet.html">Reading and Writing the Apache Parquet Format</a></li> <li class="toctree-l1"><a class="reference internal" href="api.html">API Reference</a></li> @@ -106,7 +107,7 @@ <li> - <a href="index.html" title="Previous Chapter: Apache Arrow (Python)"><span class="glyphicon glyphicon-chevron-left visible-sm"></span><span class="hidden-sm hidden-tablet">« Apache Arrow (Python)</span> + <a href="index.html" title="Previous Chapter: Apache Arrow (Python)"><span class="glyphicon glyphicon-chevron-left visible-sm"></span><span class="hidden-sm hidden-tablet">« Apache Arrow ...</span> </a> </li> <li> http://git-wip-us.apache.org/repos/asf/arrow-site/blob/4d4a3202/docs/python/ipc.html ---------------------------------------------------------------------- diff --git a/docs/python/ipc.html b/docs/python/ipc.html index 39641e1..68d95cd 100644 --- a/docs/python/ipc.html +++ b/docs/python/ipc.html @@ -30,7 +30,7 @@ <script type="text/javascript" src="_static/bootstrap-sphinx.js"></script> <link rel="index" title="Index" href="genindex.html" /> <link rel="search" title="Search" href="search.html" /> - <link rel="next" title="Filesystem Interfaces" href="filesystems.html" /> + <link rel="next" title="File System Interfaces" href="filesystems.html" /> <link rel="prev" title="In-Memory Data Model" href="data.html" /> <meta charset='utf-8'> <meta http-equiv='X-UA-Compatible' content='IE=edge,chrome=1'> @@ -73,7 +73,8 @@ <li class="toctree-l1"><a class="reference internal" href="memory.html">Memory and IO Interfaces</a></li> <li class="toctree-l1"><a class="reference internal" href="data.html">In-Memory Data Model</a></li> <li class="toctree-l1 current"><a class="current reference internal" href="#">IPC: Fast Streaming and Serialization</a></li> -<li class="toctree-l1"><a class="reference internal" href="filesystems.html">Filesystem Interfaces</a></li> +<li class="toctree-l1"><a class="reference internal" href="filesystems.html">File System Interfaces</a></li> +<li class="toctree-l1"><a class="reference internal" href="plasma.html">The Plasma In-Memory Object Store</a></li> <li class="toctree-l1"><a class="reference internal" href="pandas.html">Using PyArrow with pandas</a></li> <li class="toctree-l1"><a class="reference internal" href="parquet.html">Reading and Writing the Apache Parquet Format</a></li> <li class="toctree-l1"><a class="reference internal" href="api.html">API Reference</a></li> @@ -106,11 +107,11 @@ <li> - <a href="data.html" title="Previous Chapter: In-Memory Data Model"><span class="glyphicon glyphicon-chevron-left visible-sm"></span><span class="hidden-sm hidden-tablet">« In-Memory Data Model</span> + <a href="data.html" title="Previous Chapter: In-Memory Data Model"><span class="glyphicon glyphicon-chevron-left visible-sm"></span><span class="hidden-sm hidden-tablet">« In-Memory Dat...</span> </a> </li> <li> - <a href="filesystems.html" title="Next Chapter: Filesystem Interfaces"><span class="glyphicon glyphicon-chevron-right visible-sm"></span><span class="hidden-sm hidden-tablet">Filesystem Interfaces »</span> + <a href="filesystems.html" title="Next Chapter: File System Interfaces"><span class="glyphicon glyphicon-chevron-right visible-sm"></span><span class="hidden-sm hidden-tablet">File System I... »</span> </a> </li> @@ -200,7 +201,7 @@ particular stream. Now we can do:</p> <span class="gp">In [10]: </span><span class="n">buf</span> <span class="o">=</span> <span class="n">sink</span><span class="o">.</span><span class="n">get_result</span><span class="p">()</span> <span class="gp">In [11]: </span><span class="n">buf</span><span class="o">.</span><span class="n">size</span> -<span class="gh">Out[11]: </span><span class="go">3820</span> +<span class="gh">Out[11]: </span><span class="go">2388</span> </pre></div> </div> <p>Now <code class="docutils literal"><span class="pre">buf</span></code> contains the complete stream as an in-memory byte buffer. We can @@ -247,7 +248,7 @@ batches are also zero-copy and do not allocate any new memory on read.</p> <span class="gp">In [21]: </span><span class="n">buf</span> <span class="o">=</span> <span class="n">sink</span><span class="o">.</span><span class="n">get_result</span><span class="p">()</span> <span class="gp">In [22]: </span><span class="n">buf</span><span class="o">.</span><span class="n">size</span> -<span class="gh">Out[22]: </span><span class="go">7914</span> +<span class="gh">Out[22]: </span><span class="go">5042</span> </pre></div> </div> <p>The difference between <a class="reference internal" href="generated/pyarrow.RecordBatchFileReader.html#pyarrow.RecordBatchFileReader" title="pyarrow.RecordBatchFileReader"><code class="xref py py-class docutils literal"><span class="pre">RecordBatchFileReader</span></code></a> and
