This is an automated email from the ASF dual-hosted git repository.
timbrown pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-xtable.git
The following commit(s) were added to refs/heads/main by this push:
new a2a93d8e added docker file (#1)
a2a93d8e is described below
commit a2a93d8e574c3b92b83cb18aead730d9b4625498
Author: Guilherme Torres Castro <[email protected]>
AuthorDate: Wed Aug 7 15:33:49 2024 -0300
added docker file (#1)
Added dockerfile
---
Dockerfile | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
README.md | 14 ++++++++++++++
2 files changed, 72 insertions(+)
diff --git a/Dockerfile b/Dockerfile
new file mode 100644
index 00000000..c749651a
--- /dev/null
+++ b/Dockerfile
@@ -0,0 +1,58 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# syntax=docker/dockerfile:1
+
+FROM maven:3.9.8-eclipse-temurin-11-focal as package
+
+WORKDIR /build
+
+COPY ./ ./
+RUN --mount=type=cache,target=/root/.m2 \
+ MAVEN_OPTS=-Dorg.slf4j.simpleLogger.defaultLogLevel=warn mvn -B package
-DskipTests
+RUN mv xtable-utilities/target/xtable-utilities-$(mvn help:evaluate
-Dexpression=project.version -q -DforceStdout)-bundled.jar target/app.jar
+
+FROM eclipse-temurin:17-jre-jammy AS final
+
+# Create a non-privileged user that the app will run under.
+# See https://docs.docker.com/go/dockerfile-user-best-practices/
+ARG UID=10001
+RUN adduser \
+ --disabled-password \
+ --gecos "" \
+ --home "/nonexistent" \
+ --shell "/sbin/nologin" \
+ --no-create-home \
+ --uid "${UID}" \
+ appuser
+USER appuser
+
+# Copy the executable from the "package" stage.
+COPY --from=package build/target/app.jar ./app.jar
+
+# It's necessary to added opens to run spark on java >= 17
+#
https://stackoverflow.com/questions/73465937/apache-spark-3-3-0-breaks-on-java-17-with-cannot-access-class-sun-nio-ch-direct
+ENTRYPOINT [
+ "java", \
+ "--add-opens=java.base/sun.nio.hb=ALL-UNNAMED", \
+ "--add-opens=java.base/sun.nio.ch=ALL-UNNAMED", \
+ "--add-opens=java.base/java.nio=ALL-UNNAMED", \
+ "--add-opens=java.base/java.lang.invoke=ALL-UNNAMED", \
+ "--add-opens=java.base/java.util=ALL-UNNAMED", \
+ "--add-opens=java.base/java.lang=ALL-UNNAMED", \
+ "-jar", \
+ "./app.jar" \
+ ]
diff --git a/README.md b/README.md
index a150af49..5c32cdb5 100644
--- a/README.md
+++ b/README.md
@@ -117,6 +117,20 @@ The custom hadoop configurations can be passed in with the
`--hadoopConfig [cust
The config in custom hadoop config file will override the default hadoop
configurations. For an example
of a custom hadoop config file, see
[hadoop.xml](https://xtable.apache.org/docs/fabric#step-2-translate-source-table-to-delta-lake-format-using-apache-xtable-incubating).
+# Running using docker
+
+1. Build the docker image using `docker builld . -t xtable`
+2. Mount the config files on the container and run the container:
+
+```
+docker run \
+ -v ./xtable/config.yml:/xtable/config.yml \
+ -v ./xtable/core-site.xml:/xtable/core-site.xml \
+ -v ./xtable/catalog.yml:/xtable/catalog.yml \
+ xtable \
+ --datasetConfig /xtable/config.yml --hadoopConfig /xtable/core-site.xml
--icebergCatalogConfig xtable/catalog.yml
+```
+
# Contributing
## Setup
For setting up the repo on IntelliJ, open the project and change the java
version to Java11 in File->ProjectStructure
