[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691125#comment-16691125 ] ASF GitHub Bot commented on PIO-86: --- marevol commented on issue #462: [PIO-86] Docker support URL: https://github.com/apache/predictionio/pull/462#issuecomment-439735918 No update at the moment... This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Assignee: Shinsuke Sugaya >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690106#comment-16690106 ] ASF GitHub Bot commented on PIO-86: --- ahmettahasakar commented on issue #462: [PIO-86] Docker support URL: https://github.com/apache/predictionio/pull/462#issuecomment-439553663 @marevol Is there any update on the kubernetes deploy documents? And is there any documentation about how to do it with multiple replicas? Thank you This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Assignee: Shinsuke Sugaya >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654849#comment-16654849 ] Shinsuke Sugaya commented on PIO-86: Yes. Please file a new issue. I'll close this issue. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Assignee: Shinsuke Sugaya >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651844#comment-16651844 ] Wei Chen commented on PIO-86: - I can work on it. Should we create another issue? > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Assignee: Shinsuke Sugaya >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651360#comment-16651360 ] Shinsuke Sugaya commented on PIO-86: Yes, we need to update them. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Assignee: Shinsuke Sugaya >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651157#comment-16651157 ] Wei Chen commented on PIO-86: - btw, [~shinsuke], should we also point our livedoc and README's docker installation to the docker you pushed in #PR-462? Currently, our livedoc and README only point docker installation to community contribution. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Assignee: Shinsuke Sugaya >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[
https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644566#comment-16644566
]
ASF GitHub Bot commented on PIO-86:
---
marevol closed pull request #462: [PIO-86] Docker support
URL: https://github.com/apache/predictionio/pull/462
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docker/README.md b/docker/README.md
new file mode 100644
index 0..f1bc5a08f
--- /dev/null
+++ b/docker/README.md
@@ -0,0 +1,249 @@
+
+
+Apache PredictionIO Docker
+==
+
+## Overview
+
+PredictionIO Docker provides Docker image for use in development and
production environment.
+
+
+## Usage
+
+### Run PredictionIO with Selectable docker-compose Files
+
+You can choose storages for event/meta/model to select docker-compose.yml.
+
+```
+docker-compose -f docker-compose.yml -f ... up
+```
+
+Supported storages are as below:
+
+| Type | Storage |
+|:-:|:-|
+| Event | Postgresql, MySQL, Elasticsearch |
+| Meta | Postgresql, MySQL, Elasticsearch |
+| Model | Postgresql, MySQL, LocalFS |
+
+If you run PredictionIO with Postgresql, run as below:
+
+```
+docker-compose -f docker-compose.yml \
+ -f pgsql/docker-compose.base.yml \
+ -f pgsql/docker-compose.meta.yml \
+ -f pgsql/docker-compose.event.yml \
+ -f pgsql/docker-compose.model.yml \
+ up
+```
+
+To use localfs as model storage, change as below:
+
+```
+docker-compose -f docker-compose.yml \
+ -f pgsql/docker-compose.base.yml \
+ -f pgsql/docker-compose.meta.yml \
+ -f pgsql/docker-compose.event.yml \
+ -f localfs/docker-compose.model.yml \
+ up
+```
+
+## Tutorial
+
+In this demo, we will show you how to build a recommendation template.
+
+### Run PredictionIO environment
+
+The following command starts PredictionIO with an event server.
+PredictionIO docker image mounts ./templates directory to /templates.
+
+```
+$ docker-compose -f docker-compose.yml \
+-f pgsql/docker-compose.base.yml \
+-f pgsql/docker-compose.meta.yml \
+-f pgsql/docker-compose.event.yml \
+-f pgsql/docker-compose.model.yml \
+up
+```
+
+We provide `pio-docker` command as an utility for `pio` command.
+`pio-docker` invokes `pio` command in PredictionIO container.
+
+```
+$ export PATH=`pwd`/bin:$PATH
+$ pio-docker status
+...
+[INFO] [Management$] Your system is all ready to go.
+```
+
+### Download Recommendation Template
+
+This demo uses
[predictionio-template-recommender](https://github.com/apache/predictionio-template-recommender).
+
+```
+$ cd templates
+$ git clone https://github.com/apache/predictionio-template-recommender.git
MyRecommendation
+$ cd MyRecommendation
+```
+
+### Register Application
+
+You need to register this application to PredictionIO:
+
+```
+$ pio-docker app new MyApp1
+[INFO] [App$] Initialized Event Store for this app ID: 1.
+[INFO] [Pio$] Created a new app:
+[INFO] [Pio$] Name: MyApp1
+[INFO] [Pio$] ID: 1
+[INFO] [Pio$] Access Key:
i-zc4EleEM577EJhx3CzQhZZ0NnjBKKdSbp3MiR5JDb2zdTKKzH9nF6KLqjlMnvl
+```
+
+Since an access key is required in subsequent steps, set it to ACCESS_KEY.
+
+```
+$ ACCESS_KEY=i-zc4EleEM577EJhx3CzQhZZ0NnjBKKdSbp3MiR5JDb2zdTKKzH9nF6KLqjlMnvl
+```
+
+`engine.json` contains an application name, so replace `INVALID_APP_NAME` with
`MyApp1`.
+
+```
+...
+"datasource": {
+ "params" : {
+"appName": "MyApp1"
+ }
+},
+...
+```
+
+### Import Data
+
+To import training data to Event server for PredictionIO, this template
provides an import tool.
+The tool depends on PredictionIO Python SDK and install as below:
+
+```
+$ pip install predictionio
+```
+and then import data:
+```
+$ curl
https://raw.githubusercontent.com/apache/spark/master/data/mllib/sample_movielens_data.txt
--create-dirs -o data/sample_movielens_data.txt
+$ python data/import_eventserver.py --access_key $ACCESS_KEY
+```
+
+### Build Template
+
+This is Scala based template.
+So, you need to build this template by `pio` command.
+
+```
+$ pio-docker build --verbose
+```
+
+### Train and Create Model
+
+To train a recommendation model, run `train` sub-command:
+
+```
+$ pio-docker train
+```
+
+### Deploy Model
+
+If a recommendation model is created successfully, deploy it to Prediction
server for PredictionIO.
+
+```
+$ pio-docker deploy
+
+```
+You can check predictions as below:
+```
+$ curl -H "Content-Type: application/json" \
+-d '{ "user": "1", "num": 4 }' http://localhost:8000/queries.json
+```
+
+## Advanced Topics
+
+### Run with Elasticsearch
+
+For Elasticsearch, Meta and Event storage are available.
+To start PredictionIO with Elasticsearch,
+
+```
+docker-compose -f docker-compose.yml \
+
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637006#comment-16637006 ] ASF GitHub Bot commented on PIO-86: --- Github user takezoe commented on the issue: https://github.com/apache/predictionio/pull/462 > Does --build-arg option solve it? Using `--build-arg` is nice! But I mean it would be nice if we can create a docker image on my working copy before commit and push something. But this is my opinion. Don't mind if you don't need it. > Just in case, I moved MySQL download process into pio_run. Good idea. This solution can be applied to setup of native distributions. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635008#comment-16635008 ] ASF GitHub Bot commented on PIO-86: --- Github user marevol commented on the issue: https://github.com/apache/predictionio/pull/462 Updated: - Set v0.13.0 as a default build tag - Use --build-arg option to change a tag/branch I'll merge this PR if there are no other concerns. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629468#comment-16629468 ] ASF GitHub Bot commented on PIO-86: --- Github user marevol commented on the issue: https://github.com/apache/predictionio/pull/462 It's okay to merge this PR. For a release procedure, I'll file a new issue on JIRA and update PMC.md. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629229#comment-16629229 ] ASF GitHub Bot commented on PIO-86: --- Github user dszeto commented on the issue: https://github.com/apache/predictionio/pull/462 @marevol Is there further updates to this PR? Is this ready for merging? > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620012#comment-16620012 ] ASF GitHub Bot commented on PIO-86: --- Github user marevol commented on the issue: https://github.com/apache/predictionio/pull/462 > your plan is that creating whole PredictionIO enviornment indcluding Spark cluster and storage cluster on the Kubernetes cluster Yes, and also you can use external clusters to specify it in docker-compose.[meta|event].yml. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619953#comment-16619953 ] ASF GitHub Bot commented on PIO-86: --- Github user takezoe commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218643244 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git +WORKDIR /opt/src/predictionio +RUN git checkout refs/tags/$PIO_TAG +RUN bash ./make-distribution.sh \ + -Dscala.version=$SCALA_VERSION \ + -Dspark.version=$SPARK_VERSION \ + -Dhadoop.version=$HADOOP_VERSION \ + -Delasticsearch.version=$ELASTICSEARCH_VERSION \ + --with-deb && \ +dpkg -i ./assembly/target/predictionio_*.deb && \ +rm -rf /opt/src/predictionio/* --- End diff -- I missed that `pio build` does Scala build in the container (and it requires ivy cache!). While `docker build` takes very long time, it's better than taking same time at `pio build` as you say. It's OK now, but it's nice if we can consider how to reduce the build time of the docker image in the future. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619906#comment-16619906 ] ASF GitHub Bot commented on PIO-86: --- Github user dszeto commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218631476 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git +WORKDIR /opt/src/predictionio +RUN git checkout refs/tags/$PIO_TAG +RUN bash ./make-distribution.sh \ + -Dscala.version=$SCALA_VERSION \ + -Dspark.version=$SPARK_VERSION \ + -Dhadoop.version=$HADOOP_VERSION \ + -Delasticsearch.version=$ELASTICSEARCH_VERSION \ + --with-deb && \ +dpkg -i ./assembly/target/predictionio_*.deb && \ +rm -rf /opt/src/predictionio/* --- End diff -- It’s probably okay if we give the container more CPU and memory. The Ivy cache is a nice touch. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619905#comment-16619905 ] ASF GitHub Bot commented on PIO-86: --- Github user dszeto commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218631217 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git --- End diff -- Agree. I intend to use this as the base for CI, replacing the current test image. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619888#comment-16619888 ] ASF GitHub Bot commented on PIO-86: --- Github user marevol commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218627394 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git +WORKDIR /opt/src/predictionio +RUN git checkout refs/tags/$PIO_TAG +RUN bash ./make-distribution.sh \ + -Dscala.version=$SCALA_VERSION \ + -Dspark.version=$SPARK_VERSION \ + -Dhadoop.version=$HADOOP_VERSION \ + -Delasticsearch.version=$ELASTICSEARCH_VERSION \ + --with-deb && \ +dpkg -i ./assembly/target/predictionio_*.deb && \ +rm -rf /opt/src/predictionio/* --- End diff -- To create ivy cache, I put it at image build time. `pio build` will take a long time(over 30min...) without the cache when running a container. So, I do not think that it's acceptable to users... Any other ideas are welcomed :) > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619782#comment-16619782 ] ASF GitHub Bot commented on PIO-86: --- Github user takezoe commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218608459 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git --- End diff -- I think building the container from the current working copy would help development of PredictionIO as well. When we need to build a specific version of container, we can do it by checkout the tag. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619778#comment-16619778 ] ASF GitHub Bot commented on PIO-86: --- Github user takezoe commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218607723 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git +WORKDIR /opt/src/predictionio +RUN git checkout refs/tags/$PIO_TAG +RUN bash ./make-distribution.sh \ + -Dscala.version=$SCALA_VERSION \ + -Dspark.version=$SPARK_VERSION \ + -Dhadoop.version=$HADOOP_VERSION \ + -Delasticsearch.version=$ELASTICSEARCH_VERSION \ + --with-deb && \ +dpkg -i ./assembly/target/predictionio_*.deb && \ +rm -rf /opt/src/predictionio/* --- End diff -- Building PredictionIO inside of the container is too slow. How about build outside of the container and just copy a binary to the container? > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619422#comment-16619422 ] ASF GitHub Bot commented on PIO-86: --- Github user dszeto commented on the issue: https://github.com/apache/predictionio/pull/462 One way we can proceed is to mark the `Dockerfile` and the image it generates as official. Helper scripts and compose files can be marked as official reference. Good work @marevol ! > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618853#comment-16618853 ] ASF GitHub Bot commented on PIO-86: --- Github user marevol commented on the issue: https://github.com/apache/predictionio/pull/462 Thank you for your feedback. > However I think docker-compose is just for quick start for users I have a plan to support Kubernetes after this PR. To do that, I consider that Kompose generates K8s' yml files for production use. So, I think that this multiple docker compose files are for production. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16601790#comment-16601790 ] Shinsuke Sugaya commented on PIO-86: I'm working on Docker support. A prototype I created is https://github.com/jpioug/predictionio-docker This docker image works with other images, such as Postgresql, by using Docker Compose. You can select images for Meta/Event/Model Storages and will also use Spark Cluster. I'll create PR to move it to PredictionIO. Please let me know if there are any concerns. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
