This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new 238b682 chore: Add documentation publishing infrastructure (#314)
238b682 is described below
commit 238b6820c3733239b97b45cd378d1d0c697abeb1
Author: Andy Grove <[email protected]>
AuthorDate: Wed Apr 24 10:11:46 2024 -0600
chore: Add documentation publishing infrastructure (#314)
* Add starting point for documentation site
* remove branding guideline
* Remove old logos
* remove logos
* add ASF license headers
* Add ASF links
---
.github/workflows/docs.yaml | 81 ++++++++++++++
README.md | 19 +++-
docs/.gitignore | 21 ++++
docs/Makefile | 38 +++++++
docs/README.md | 68 ++++++++++++
docs/build.sh | 26 +++++
docs/make.bat | 52 +++++++++
docs/requirements.txt | 22 ++++
docs/source/_static/images/2x_bgwhite_original.png | Bin 0 -> 60451 bytes
docs/source/_static/images/original.png | Bin 0 -> 26337 bytes
docs/source/_static/images/original.svg | 49 +++++++++
docs/source/_static/images/original2x.png | Bin 0 -> 66517 bytes
docs/source/_static/theme_overrides.css | 86 +++++++++++++++
docs/source/_templates/docs-sidebar.html | 38 +++++++
docs/source/_templates/layout.html | 45 ++++++++
docs/source/conf.py | 121 +++++++++++++++++++++
docs/source/index.rst | 58 ++++++++++
17 files changed, 719 insertions(+), 5 deletions(-)
diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml
new file mode 100644
index 0000000..40ddece
--- /dev/null
+++ b/.github/workflows/docs.yaml
@@ -0,0 +1,81 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - .asf.yaml
+ - .github/workflows/docs.yaml
+ - docs/**
+
+name: Deploy DataFusion Comet site
+
+jobs:
+ build-docs:
+ name: Build docs
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout docs sources
+ uses: actions/checkout@v4
+
+ - name: Checkout asf-site branch
+ uses: actions/checkout@v4
+ with:
+ ref: asf-site
+ path: asf-site
+
+ - name: Setup Python
+ uses: actions/setup-python@v5
+ with:
+ python-version: "3.10"
+
+ - name: Install dependencies
+ run: |
+ set -x
+ python3 -m venv venv
+ source venv/bin/activate
+ pip install -r docs/requirements.txt
+
+ - name: Build docs
+ run: |
+ set -x
+ source venv/bin/activate
+ cd docs
+ ./build.sh
+
+ - name: Copy & push the generated HTML
+ run: |
+ set -x
+ cd asf-site/
+ rsync \
+ -a \
+ --delete \
+ --exclude '/.git/' \
+ ../docs/build/html/ \
+ ./
+ cp ../.asf.yaml .
+ touch .nojekyll
+ git status --porcelain
+ if [ "$(git status --porcelain)" != "" ]; then
+ git config user.name "github-actions[bot]"
+ git config user.email
"github-actions[bot]@users.noreply.github.com"
+ git add --all
+ git commit -m 'Publish built docs triggered by ${{ github.sha }}'
+ git push || git push --force
+ fi
\ No newline at end of file
diff --git a/README.md b/README.md
index 299b390..7d020eb 100644
--- a/README.md
+++ b/README.md
@@ -19,7 +19,7 @@ under the License.
# Apache DataFusion Comet
-Comet is an Apache Spark plugin that uses [Apache
DataFusion](https://datafusion.apache.org/datafusion/)
+Apache DataFusion Comet is an Apache Spark plugin that uses [Apache
DataFusion](https://datafusion.apache.org/)
as native runtime to achieve improvement in terms of query efficiency and
query runtime.
Comet runs Spark SQL queries using the native DataFusion runtime, which is
@@ -28,6 +28,7 @@ typically faster and more resource efficient than JVM based
runtimes.
<a href="doc/comet-overview.png"><img src="doc/comet-system-diagram.png"
align="center" width="500" ></a>
Comet aims to support:
+
- a native Parquet implementation, including both reader and writer
- full implementation of Spark operators, including
Filter/Project/Aggregation/Join/Exchange etc.
@@ -71,18 +72,22 @@ Linux, Apple OSX (Intel and M1)
Make sure the requirements above are met and software installed on your machine
### Clone repo
+
```commandline
git clone https://github.com/apache/datafusion-comet.git
```
### Specify the Spark version and build the Comet
+
Spark 3.4 used for the example.
+
```
cd datafusion-comet
make release PROFILES="-Pspark-3.4"
```
### Run Spark with Comet enabled
+
Make sure `SPARK_HOME` points to the same Spark version as Comet has built for.
```
@@ -93,16 +98,19 @@ $SPARK_HOME/bin/spark-shell --jars
spark/target/comet-spark-spark3.4_2.12-0.1.0-
--conf spark.comet.exec.all.enabled=true
```
-### Verify Comet enabled for Spark SQL query
+### Verify Comet enabled for Spark SQL query
Create a test Parquet source
+
```scala
scala> (0 until 10).toDF("a").write.mode("overwrite").parquet("/tmp/test")
```
-Query the data from the test source and check:
+Query the data from the test source and check:
+
- INFO message shows the native Comet library has been initialized.
- The query plan reflects Comet operators being used for this query instead of
Spark ones
+
```scala
scala> spark.read.parquet("/tmp/test").createOrReplaceTempView("t1")
scala> spark.sql("select * from t1 where a > 5").explain
@@ -110,8 +118,8 @@ INFO src/lib.rs: Comet native library initialized
== Physical Plan ==
*(1) ColumnarToRow
+- CometFilter [a#14], (isnotnull(a#14) AND (a#14 > 5))
- +- CometScan parquet [a#14] Batched: true, DataFilters:
[isnotnull(a#14), (a#14 > 5)],
- Format: CometParquet, Location: InMemoryFileIndex(1
paths)[file:/tmp/test], PartitionFilters: [],
+ +- CometScan parquet [a#14] Batched: true, DataFilters:
[isnotnull(a#14), (a#14 > 5)],
+ Format: CometParquet, Location: InMemoryFileIndex(1
paths)[file:/tmp/test], PartitionFilters: [],
PushedFilters: [IsNotNull(a), GreaterThan(a,5)], ReadSchema:
struct<a:int>
```
@@ -136,6 +144,7 @@ Comet doesn't have official release yet so currently the
only way to test it is
Some cluster managers may require additional configuration, see
https://spark.apache.org/docs/latest/cluster-overview.html
To enable columnar shuffle which supports all partitioning and basic complex
types, one more config is required:
+
```
--conf spark.comet.columnar.shuffle.enabled=true
```
diff --git a/docs/.gitignore b/docs/.gitignore
new file mode 100644
index 0000000..e2a54c0
--- /dev/null
+++ b/docs/.gitignore
@@ -0,0 +1,21 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+build
+temp
+venv/
+.python-version
diff --git a/docs/Makefile b/docs/Makefile
new file mode 100644
index 0000000..6bce199
--- /dev/null
+++ b/docs/Makefile
@@ -0,0 +1,38 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS ?=
+SPHINXBUILD ?= sphinx-build
+SOURCEDIR = source
+BUILDDIR = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+ @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+ @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
diff --git a/docs/README.md b/docs/README.md
new file mode 100644
index 0000000..ca33315
--- /dev/null
+++ b/docs/README.md
@@ -0,0 +1,68 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Apache DataFusion Comet Documentation
+
+This folder contains the source content for the Apache DataFusion Comet
documentation site. This content is published
+to https://datafusion.apache.org/comet when any changes are merged into the
main branch.
+
+## Dependencies
+
+It's recommended to install build dependencies and build the documentation
+inside a Python virtualenv.
+
+- Python
+- `pip install -r requirements.txt`
+
+## Build & Preview
+
+Run the provided script to build the HTML pages.
+
+```bash
+./build.sh
+```
+
+The HTML will be generated into a `build` directory.
+
+Preview the site on Linux by running this command.
+
+```bash
+firefox build/html/index.html
+```
+
+## Making Changes
+
+To make changes to the docs, simply make a Pull Request with your
+proposed changes as normal. When the PR is merged the docs will be
+automatically updated.
+
+## Release Process
+
+This documentation is hosted at https://datafusion.apache.org/comet/
+
+When the PR is merged to the `main` branch of the `datafusion-comet`
+repository, a [github
workflow](https://github.com/apache/datafusion-comet/blob/main/.github/workflows/docs.yaml)
which:
+
+1. Builds the html content
+2. Pushes the html content to the
[`asf-site`](https://github.com/apache/datafusion-comet/tree/asf-site) branch
in this repository.
+
+The Apache Software Foundation provides https://datafusion.apache.org/,
+which serves content based on the configuration in
+[.asf.yaml](https://github.com/apache/datafusion-comet/blob/main/.asf.yaml),
+which specifies the target as https://datafusion.apache.org/comet/.
diff --git a/docs/build.sh b/docs/build.sh
new file mode 100755
index 0000000..ff30cf8
--- /dev/null
+++ b/docs/build.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+set -e
+rm -rf build 2> /dev/null
+rm -rf temp 2> /dev/null
+mkdir temp
+cp -rf source/* temp/
+make SOURCEDIR=`pwd`/temp html
diff --git a/docs/make.bat b/docs/make.bat
new file mode 100644
index 0000000..ded5b4a
--- /dev/null
+++ b/docs/make.bat
@@ -0,0 +1,52 @@
+@rem Licensed to the Apache Software Foundation (ASF) under one
+@rem or more contributor license agreements. See the NOTICE file
+@rem distributed with this work for additional information
+@rem regarding copyright ownership. The ASF licenses this file
+@rem to you under the Apache License, Version 2.0 (the
+@rem "License"); you may not use this file except in compliance
+@rem with the License. You may obtain a copy of the License at
+@rem
+@rem http://www.apache.org/licenses/LICENSE-2.0
+@rem
+@rem Unless required by applicable law or agreed to in writing,
+@rem software distributed under the License is distributed on an
+@rem "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+@rem KIND, either express or implied. See the License for the
+@rem specific language governing permissions and limitations
+@rem under the License.
+
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+ set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+ echo.
+ echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+ echo.installed, then set the SPHINXBUILD environment variable to point
+ echo.to the full path of the 'sphinx-build' executable. Alternatively
you
+ echo.may add the Sphinx directory to PATH.
+ echo.
+ echo.If you don't have Sphinx installed, grab it from
+ echo.http://sphinx-doc.org/
+ exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd
diff --git a/docs/requirements.txt b/docs/requirements.txt
new file mode 100644
index 0000000..24546d5
--- /dev/null
+++ b/docs/requirements.txt
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+sphinx
+pydata-sphinx-theme==0.8.0
+myst-parser
+maturin
+jinja2
diff --git a/docs/source/_static/images/2x_bgwhite_original.png
b/docs/source/_static/images/2x_bgwhite_original.png
new file mode 100644
index 0000000..abb5fca
Binary files /dev/null and b/docs/source/_static/images/2x_bgwhite_original.png
differ
diff --git a/docs/source/_static/images/original.png
b/docs/source/_static/images/original.png
new file mode 100644
index 0000000..687f946
Binary files /dev/null and b/docs/source/_static/images/original.png differ
diff --git a/docs/source/_static/images/original.svg
b/docs/source/_static/images/original.svg
new file mode 100644
index 0000000..4b4f141
--- /dev/null
+++ b/docs/source/_static/images/original.svg
@@ -0,0 +1,49 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<svg width="801" height="168" viewBox="0 0 801 168" fill="none"
xmlns="http://www.w3.org/2000/svg";>
+<g clip-path="url(#clip0_1_181)">
+<path fill-rule="evenodd" clip-rule="evenodd" d="M76.1297 168C88.4597 168
99.6097 158.25 107.58 142.55C127.23 144.6 143.09 139.99 149.76 128.51C156.29
117.25 152.65 101.68 141.58 86.09C152.36 70.67 155.85 55.32 149.39 44.19C143.01
33.19 128.19 28.5 109.69 29.92C101.65 11.62 89.5997 0 76.1297 0C62.6597 0
50.4997 11.73 42.4497 30.18C24.3497 28.97 9.87966 33.68 3.59966 44.51C-2.81034
55.56 0.589656 70.78 11.1897 86.09C0.299656 101.57 -3.24034 117.01 3.23966
128.18C9.80966 139.5 25.3097 144. [...]
+<path d="M204.34 45.97H192.83L190.47 52.2H185.2L196.07 23.88H201.47L212.54
52.2H206.78L204.34 45.97ZM202.74 41.74L198.54 30.99L194.46
41.74H202.73H202.74Z" fill="#1F2937"/>
+<path d="M235.55 23.88H246.82C249.59 23.88 251.9 24.73 253.73 26.44C255.57
28.14 256.49 30.38 256.49 33.15C256.49 35.92 255.57 38.2 253.73 39.9C251.89
41.6 249.59 42.46 246.82 42.46H240.95V52.21H235.56V23.89L235.55 23.88ZM240.94
37.26H246.85C248 37.26 248.96 36.87 249.73 36.1C250.5 35.3 250.89 34.32 250.89
33.14C250.89 31.96 250.5 31 249.73 30.22C248.98 29.45 248.02 29.06 246.85
29.06H240.94V37.25V37.26Z" fill="#1F2937"/>
+<path d="M293.67 45.97H282.16L279.8 52.2H274.53L285.4 23.88H290.79L301.86
52.2H296.1L293.66 45.97H293.67ZM292.07 41.74L287.87 30.99L283.79
41.74H292.06H292.07Z" fill="#1F2937"/>
+<path d="M335.99 23C338.57 23 340.92 23.68 343.02 25.04C345.15 26.37 346.84
28.18 348.1 30.47L343.1 32.87C341.37 29.57 339 27.92 335.99 27.92C333.62 27.92
331.65 28.91 330.07 30.88C328.53 32.82 327.75 35.22 327.75 38.07C327.75 40.92
328.52 43.32 330.07 45.26C331.62 47.18 333.59 48.14 335.99 48.14C337.45 48.14
338.81 47.73 340.07 46.9C341.32 46.05 342.32 44.93 343.07 43.54L347.99
45.9C346.79 48.14 345.13 49.89 342.99 51.17C340.86 52.42 338.53 53.05 336
53.05C331.95 53.05 328.63 51.65 326. [...]
+<path d="M393.49
52.2H388.09V40.61H377.1V52.2H371.71V23.88H377.1V35.43H388.09V23.88H393.49V52.2Z"
fill="#1F2937"/>
+<path d="M435.17
29.07H423.9V35.46H432.41V40.65H423.9V47H435.17V52.19H418.51V23.87H435.17V29.06V29.07Z"
fill="#1F2937"/>
+<path d="M185 76.1H207.25C218.63 76.1 227.58 79.3 234.1 85.69C240.69 92.02
243.98 100.14 243.98 110.04C243.98 119.94 240.69 128.1 234.1 134.49C227.51
140.88 218.53 144.08 207.15 144.08H184.99V76.1H185ZM199.96
90.49V129.7H206.67C209.55 129.7 212.2 129.38 214.63 128.74C217.12 128.1 219.43
127.08 221.54 125.67C223.71 124.26 225.41 122.25 226.62 119.63C227.83 116.95
228.44 113.75 228.44 110.04C228.44 106.33 227.83 103.17 226.62 100.55C225.4
97.93 223.71 95.92 221.54 94.51C219.43 93.1 217.16 [...]
+<path d="M291.82 130.85H264.96L259.97 144.08H245.39L271.38 76.1H286.34L312.91
144.08H296.89L291.81 130.85H291.82ZM287.4 119.35L278.29 95.76L269.37
119.35H287.4Z" fill="#1F2937"/>
+<path d="M362.7 90.49H342.94V144.09H327.98V90.49H308.51V76.11H362.7V90.49Z"
fill="#1F2937"/>
+<path d="M404.77 130.85H377.92L372.93 144.08H358.35L384.34 76.1H399.3L425.87
144.08H409.85L404.77 130.85ZM400.36 119.35L391.25 95.76L382.33 119.35H400.36Z"
fill="#1F2937"/>
+<path d="M474.73
90.49H448.45V102.95H468.4V117.33H448.45V144.08H433.49V76.1H474.73V90.48V90.49Z"
fill="#1F2937"/>
+<path d="M541.71 76.1V118.96C541.65 127.27 538.96 133.85 533.65 138.71C528.41
143.57 521.53 146 513.03 146C504.53 146 497.78 143.57 492.41 138.71C487.04
133.85 484.39 127.27 484.45 118.96V76.1H499.32V119.05C499.32 122.88 500.6
125.95 503.16 128.25C505.78 130.49 509.08 131.61 513.04 131.61C517 131.61
520.27 130.49 522.82 128.25C525.38 125.95 526.69 122.88 526.75
119.05V76.1H541.71Z" fill="#1F2937"/>
+<path d="M578.69 73.99C585.72 73.99 591.96 75.59 597.39 78.78L590.77
91.24C585.98 89.19 581.43 88.17 577.15 88.17C570.95 88.17 567.85 90.25 567.85
94.4C567.85 96.45 569.03 98.17 571.4 99.58C573.83 100.92 577.28 102.39 581.76
103.99C586.3 105.52 589.72 106.96 592.02 108.3C598.22 111.88 601.32 117.25
601.32 124.41C601.32 131.12 598.86 136.4 593.93 140.23C589.01 144.07 582.87
145.98 575.52 145.98C571.36 145.98 567.05 145.21 562.57 143.68C558.09 142.15
554.61 140.39 552.12 138.41L560.18 126. [...]
+<path d="M626.79 144.08H611.83V76.1H626.79V144.08Z" fill="#1F2937"/>
+<path d="M647.04 84.16C653.88 77.38 662.48 74 672.84 74C683.2 74 691.73 77.39
698.45 84.16C705.23 90.87 708.62 99.4 708.62 109.76C708.62 120.12 705.33 128.68
698.74 135.45C692.15 142.16 683.52 145.68 672.84 146C662.48 146 653.88 142.58
647.04 135.74C640.26 128.84 636.87 120.21 636.87 109.85C636.87 99.49 640.26
90.93 647.04 84.15V84.16ZM672.84 87.61C666.64 87.61 661.68 89.69 657.97
93.84C654.26 97.99 652.41 103.3 652.41 109.76C652.41 116.22 654.26 121.62
657.97 125.96C661.74 130.24 666.7 [...]
+<path d="M775 144.08H759.65L732.7 101.99V144.08H718.7V76.1H732.99L761
118.57V76.1H775V144.08Z" fill="#1F2937"/>
+<path d="M788.01
78.18H784.62V87.44H782.46V78.18H779.13V76.1H788.01V78.18ZM800.56
87.45H798.48V80.17L795.5 85.79H794.12L791.14 80.17L791.12
87.45H789.1V76.11H791.13L794.87 83.04L798.45 76.11H800.51L800.53 87.45H800.56Z"
fill="#1F2937"/>
+</g>
+<defs>
+<linearGradient id="paint0_linear_1_181" x1="17.0897" y1="117.74" x2="162.73"
y2="36.78" gradientUnits="userSpaceOnUse">
+<stop stop-color="#EF4137"/>
+<stop offset="1" stop-color="#FBB042"/>
+</linearGradient>
+<clipPath id="clip0_1_181">
+<rect width="800.56" height="168" fill="white"/>
+</clipPath>
+</defs>
+</svg>
diff --git a/docs/source/_static/images/original2x.png
b/docs/source/_static/images/original2x.png
new file mode 100644
index 0000000..a740210
Binary files /dev/null and b/docs/source/_static/images/original2x.png differ
diff --git a/docs/source/_static/theme_overrides.css
b/docs/source/_static/theme_overrides.css
new file mode 100644
index 0000000..3b1b86d
--- /dev/null
+++ b/docs/source/_static/theme_overrides.css
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+
+/* Customizing with theme CSS variables */
+
+:root {
+ --pst-color-active-navigation: 215, 70, 51;
+ --pst-color-link-hover: 215, 70, 51;
+ --pst-color-headerlink: 215, 70, 51;
+ /* Use normal text color (like h3, ..) instead of primary color */
+ --pst-color-h1: var(--color-text-base);
+ --pst-color-h2: var(--color-text-base);
+ /* Use softer blue from bootstrap's default info color */
+ --pst-color-info: 23, 162, 184;
+ --pst-header-height: 0px;
+}
+
+code {
+ color: rgb(215, 70, 51);
+}
+
+.footer {
+ text-align: center;
+}
+
+/* Ensure the logo is properly displayed */
+
+.navbar-brand {
+ height: auto;
+ width: auto;
+ padding: 0 2em;
+}
+
+/* This is the bootstrap CSS style for "table-striped". Since the theme does
+not yet provide an easy way to configure this globally, it easier to simply
+include this snippet here than updating each table in all rst files to
+add ":class: table-striped" */
+
+.table tbody tr:nth-of-type(odd) {
+ background-color: rgba(0, 0, 0, 0.05);
+}
+
+
+/* Limit the max height of the sidebar navigation section. Because in our
+customized template, there is more content above the navigation, i.e.
+larger logo: if we don't decrease the max-height, it will overlap with
+the footer.
+Details: 8rem for search box etc*/
+
+@media (min-width:720px) {
+ @supports (position:-webkit-sticky) or (position:sticky) {
+ .bd-links {
+ max-height: calc(100vh - 8rem)
+ }
+ }
+}
+
+
+/* Fix table text wrapping in RTD theme,
+ * see https://rackerlabs.github.io/docs-rackspace/tools/rtd-tables.html
+ */
+
+@media screen {
+ table.docutils td {
+ /* !important prevents the common CSS stylesheets from overriding
+ this as on RTD they are loaded after this stylesheet */
+ white-space: normal !important;
+ }
+}
diff --git a/docs/source/_templates/docs-sidebar.html
b/docs/source/_templates/docs-sidebar.html
new file mode 100644
index 0000000..b328129
--- /dev/null
+++ b/docs/source/_templates/docs-sidebar.html
@@ -0,0 +1,38 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+<form class="bd-search d-flex align-items-center" action="{{ pathto('search')
}}" method="get">
+ <i class="icon fas fa-search"></i>
+ <input type="search" class="form-control" name="q" id="search-input"
placeholder="{{ theme_search_bar_text }}" aria-label="{{ theme_search_bar_text
}}" autocomplete="off" >
+</form>
+
+<nav class="bd-links" id="bd-docs-nav" aria-label="Main navigation">
+ <div class="bd-toc-item active">
+ {% if "python/api" in pagename or "python/generated" in pagename %}
+ {{ generate_nav_html("sidebar", startdepth=0, maxdepth=3, collapse=False,
includehidden=True, titles_only=True) }}
+ {% else %}
+ {{ generate_nav_html("sidebar", startdepth=0, maxdepth=4, collapse=False,
includehidden=True, titles_only=True) }}
+ {% endif %}
+ </div>
+
+ <a class="navbar-brand" href="{{ pathto(master_doc) }}">
+ <img src="{{ pathto('_static/images/2x_bgwhite_original.png', 1) }}"
class="logo" alt="logo">
+ </a>
+</nav>
+
diff --git a/docs/source/_templates/layout.html
b/docs/source/_templates/layout.html
new file mode 100644
index 0000000..50405cd
--- /dev/null
+++ b/docs/source/_templates/layout.html
@@ -0,0 +1,45 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+{% extends "pydata_sphinx_theme/layout.html" %}
+
+{# Silence the navbar #}
+{% block docs_navbar %}
+{% endblock %}
+
+<!--
+ Custom footer
+-->
+{% block footer %}
+<!-- Based on pydata_sphinx_theme/footer.html -->
+<footer class="footer mt-5 mt-md-0">
+ <div class="container">
+ {% for footer_item in theme_footer_items %}
+ <div class="footer-item">
+ {% include footer_item %}
+ </div>
+ {% endfor %}
+ <div class="footer-item">
+ <p>Apache DataFusion, Apache DataFusion Comet, Apache, the Apache
feather logo, and the Apache DataFusion project logo</p>
+ <p>are either registered trademarks or trademarks of The Apache Software
Foundation in the United States and other countries.</p>
+ </div>
+ </div>
+</footer>
+
+{% endblock %}
diff --git a/docs/source/conf.py b/docs/source/conf.py
new file mode 100644
index 0000000..e7ea625
--- /dev/null
+++ b/docs/source/conf.py
@@ -0,0 +1,121 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+
+# -- Project information -----------------------------------------------------
+
+project = 'Apache DataFusion Comet'
+copyright = '2023-2024, Apache Software Foundation'
+author = 'Apache Software Foundation'
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+ 'sphinx.ext.autodoc',
+ 'sphinx.ext.autosummary',
+ 'sphinx.ext.doctest',
+ 'sphinx.ext.ifconfig',
+ 'sphinx.ext.mathjax',
+ 'sphinx.ext.viewcode',
+ 'sphinx.ext.napoleon',
+ 'myst_parser',
+]
+
+source_suffix = {
+ '.rst': 'restructuredtext',
+ '.md': 'markdown',
+}
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = []
+
+# Show members for classes in .. autosummary
+autodoc_default_options = {
+ "members": None,
+ "undoc-members": None,
+ "show-inheritance": None,
+ "inherited-members": None,
+}
+
+autosummary_generate = True
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages. See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'pydata_sphinx_theme'
+
+html_theme_options = {
+ "use_edit_page_button": True,
+}
+
+html_context = {
+ "github_user": "apache",
+ "github_repo": "datafusion-comet",
+ "github_version": "main",
+ "doc_path": "docs/source",
+}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+html_logo = "_static/images/2x_bgwhite_original.png"
+
+html_css_files = [
+ "theme_overrides.css"
+]
+
+html_js_files = [
+ ("https://buttons.github.io/buttons.js";, {'async': 'true', 'defer':
'true'}),
+]
+
+html_sidebars = {
+ "**": ["docs-sidebar.html"],
+}
+
+# tell myst_parser to auto-generate anchor links for headers h1, h2, h3
+myst_heading_anchors = 3
+
+# enable nice rendering of checkboxes for the task lists
+myst_enable_extensions = ["colon_fence", "deflist", "tasklist"]
diff --git a/docs/source/index.rst b/docs/source/index.rst
new file mode 100644
index 0000000..2cff77f
--- /dev/null
+++ b/docs/source/index.rst
@@ -0,0 +1,58 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. image:: _static/images/2x_bgwhite_original.png
+ :alt: DataFusion Logo
+
+=======================
+Apache DataFusion Comet
+=======================
+
+.. Code from https://buttons.github.io/
+.. raw:: html
+
+ <p>
+ <!-- Place this tag where you want the button to render. -->
+ <a class="github-button" href="https://github.com/apache/datafusion-comet";
data-size="large" data-show-count="true" aria-label="Star
apache/datafusion-comet on GitHub">Star</a>
+ <!-- Place this tag where you want the button to render. -->
+ <a class="github-button"
href="https://github.com/apache/datafusion-comet/fork"; data-size="large"
data-show-count="true" aria-label="Fork apache/datafusion-comet on
GitHub">Fork</a>
+ </p>
+
+Apache DataFusion Comet is an Apache Spark plugin that uses Apache DataFusion
+as a native runtime to achieve improvement in terms of query efficiency and
query runtime.
+
+This documentation site is currently being developed. The most up-to-date
documentation can be found in the
+GitHub repository at https://github.com/apache/datafusion-comet.
+
+.. _toc.links:
+.. toctree::
+ :maxdepth: 1
+ :caption: Project Links
+
+ Github and Issue Tracker <https://github.com/apache/datafusion-comet>
+
+.. _toc.asf-links:
+.. toctree::
+ :maxdepth: 1
+ :caption: ASF Links
+
+ Apache Software Foundation <https://apache.org>
+ License <https://www.apache.org/licenses/>
+ Donate <https://www.apache.org/foundation/sponsorship.html>
+ Thanks <https://www.apache.org/foundation/thanks.html>
+ Security <https://www.apache.org/security/>
+ Code of conduct <https://www.apache.org/foundation/policies/conduct.html>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
