[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618853#comment-16618853 ] ASF GitHub Bot commented on PIO-86: --- Github user marevol commented on the issue: https://github.com/apache/predictionio/pull/462 Thank you for your feedback. > However I think docker-compose is just for quick start for users I have a plan to support Kubernetes after this PR. To do that, I consider that Kompose generates K8s' yml files for production use. So, I think that this multiple docker compose files are for production. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619055#comment-16619055 ] ASF GitHub Bot commented on PIO-86: --- Github user takezoe commented on the issue: https://github.com/apache/predictionio/pull/462 Ah, your plan is that creating whole PredictionIO enviornment indcluding Spark cluster and storage cluster on the Kubernetes cluster, right? That was beyond my imagination, really exciting! If so, it would be nice if we can support HDFS, HBase and various managed services on the cloud as well. I have some questions about Dockerfile, so I will make comments later. But basically agree with your Kubernetes support plan. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619422#comment-16619422 ] ASF GitHub Bot commented on PIO-86: --- Github user dszeto commented on the issue: https://github.com/apache/predictionio/pull/462 One way we can proceed is to mark the `Dockerfile` and the image it generates as official. Helper scripts and compose files can be marked as official reference. Good work @marevol ! > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619778#comment-16619778 ] ASF GitHub Bot commented on PIO-86: --- Github user takezoe commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218607723 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git +WORKDIR /opt/src/predictionio +RUN git checkout refs/tags/$PIO_TAG +RUN bash ./make-distribution.sh \ + -Dscala.version=$SCALA_VERSION \ + -Dspark.version=$SPARK_VERSION \ + -Dhadoop.version=$HADOOP_VERSION \ + -Delasticsearch.version=$ELASTICSEARCH_VERSION \ + --with-deb && \ +dpkg -i ./assembly/target/predictionio_*.deb && \ +rm -rf /opt/src/predictionio/* --- End diff -- Building PredictionIO inside of the container is too slow. How about build outside of the container and just copy a binary to the container? > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619782#comment-16619782 ] ASF GitHub Bot commented on PIO-86: --- Github user takezoe commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218608459 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git --- End diff -- I think building the container from the current working copy would help development of PredictionIO as well. When we need to build a specific version of container, we can do it by checkout the tag. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619888#comment-16619888 ] ASF GitHub Bot commented on PIO-86: --- Github user marevol commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218627394 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git +WORKDIR /opt/src/predictionio +RUN git checkout refs/tags/$PIO_TAG +RUN bash ./make-distribution.sh \ + -Dscala.version=$SCALA_VERSION \ + -Dspark.version=$SPARK_VERSION \ + -Dhadoop.version=$HADOOP_VERSION \ + -Delasticsearch.version=$ELASTICSEARCH_VERSION \ + --with-deb && \ +dpkg -i ./assembly/target/predictionio_*.deb && \ +rm -rf /opt/src/predictionio/* --- End diff -- To create ivy cache, I put it at image build time. `pio build` will take a long time(over 30min...) without the cache when running a container. So, I do not think that it's acceptable to users... Any other ideas are welcomed :) > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619905#comment-16619905 ] ASF GitHub Bot commented on PIO-86: --- Github user dszeto commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218631217 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git --- End diff -- Agree. I intend to use this as the base for CI, replacing the current test image. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] predictionio pull request #462: [PIO-86] Docker support
Github user dszeto commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218631217 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git --- End diff -- Agree. I intend to use this as the base for CI, replacing the current test image. ---
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619906#comment-16619906 ] ASF GitHub Bot commented on PIO-86: --- Github user dszeto commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218631476 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git +WORKDIR /opt/src/predictionio +RUN git checkout refs/tags/$PIO_TAG +RUN bash ./make-distribution.sh \ + -Dscala.version=$SCALA_VERSION \ + -Dspark.version=$SPARK_VERSION \ + -Dhadoop.version=$HADOOP_VERSION \ + -Delasticsearch.version=$ELASTICSEARCH_VERSION \ + --with-deb && \ +dpkg -i ./assembly/target/predictionio_*.deb && \ +rm -rf /opt/src/predictionio/* --- End diff -- It’s probably okay if we give the container more CPU and memory. The Ivy cache is a nice touch. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619953#comment-16619953 ] ASF GitHub Bot commented on PIO-86: --- Github user takezoe commented on a diff in the pull request: https://github.com/apache/predictionio/pull/462#discussion_r218643244 --- Diff: docker/pio/Dockerfile --- @@ -0,0 +1,71 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM openjdk:8 + +ENV PIO_TAG=v0.12.1 +ENV SCALA_VERSION=2.11.12 +ENV SPARK_VERSION=2.2.2 +ENV HADOOP_VERSION=2.7.7 +ENV ELASTICSEARCH_VERSION=5.5.3 +ENV PGSQL_VERSION=42.2.4 +ENV MYSQL_VERSION=8.0.12 +ENV PIO_HOME=/usr/share/predictionio + +RUN apt-get update && \ +apt-get install -y dpkg-dev fakeroot && \ +apt-get clean && \ +rm -rf /var/lib/apt/lists/* + +WORKDIR /opt/src +RUN git clone https://github.com/apache/predictionio.git +WORKDIR /opt/src/predictionio +RUN git checkout refs/tags/$PIO_TAG +RUN bash ./make-distribution.sh \ + -Dscala.version=$SCALA_VERSION \ + -Dspark.version=$SPARK_VERSION \ + -Dhadoop.version=$HADOOP_VERSION \ + -Delasticsearch.version=$ELASTICSEARCH_VERSION \ + --with-deb && \ +dpkg -i ./assembly/target/predictionio_*.deb && \ +rm -rf /opt/src/predictionio/* --- End diff -- I missed that `pio build` does Scala build in the container (and it requires ivy cache!). While `docker build` takes very long time, it's better than taking same time at `pio build` as you say. It's OK now, but it's nice if we can consider how to reduce the build time of the docker image in the future. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] Apache PredictionIO 0.13.0 Release (RC1)
The vote has closed and passed. We have 6 +1 binding votes from: - Naoki Takezoe - Donald Szeto - Matthew Tovbin - Shinsuke Sugaya - Takako Shimamoto - Chan Lee We have no -1 votes. Thank you all for voting. I will proceed to releasing the candidate. Regards, Takako On Sat, Sep 15, 2018 at 11:03 AM Chan Lee wrote: > > +1 binding. > > Checked: > - source distribution working > - binary distribution working > > > On Thu, Sep 13, 2018 at 11:47 PM takako shimamoto > wrote: > > > +1 binding > > > > I checked: > > - Binaries working > > - Test the official template examples (scala-parallel-classification) > > using Maven artifacts > > > > > > On Fri, Sep 14, 2018 at 2:32 PM Shinsuke Sugaya > > wrote: > > > > > > +1 > > > > > > I checked it. Thank you for your work! > > > > > > Regards, > > > shinsuke > > > > > > > > > 2018年9月12日(水) 20:17 takako shimamoto : > > > > > > > > This is the vote for 0.13.0 of Apache PredictionIO. > > > > > > > > The vote will run for at least 72 hours and will close on Sep 15th, > > 2018. > > > > > > > > The release candidate artifacts can be downloaded here: > > > > > > https://dist.apache.org/repos/dist/dev/predictionio/apache-predictionio-0.13.0-rc1/ > > > > > > > > Test results of RC1 can be found here: > > > > https://travis-ci.org/apache/predictionio/builds/427541341 > > > > > > > > Maven artifacts are built from the release candidate artifacts above, > > > > and are provided as convenience for testing with engine templates. The > > > > Maven artifacts are provided at the Maven staging repo here: > > > > > > https://repository.apache.org/content/repositories/orgapachepredictionio-1024/ > > > > > > > > All JIRAs completed for this release are tagged with 'FixVersion = > > > > 0.13.0'. You can view them here: > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12340605&projectId=12320420 > > > > > > > > The artifacts have been signed with Key : 9F4FBC3D > > > > > > > > Please vote accordingly: > > > > > > > > [ ] +1, accept RC as the official 0.13.0 release > > > > [ ] -1, do not accept RC as the official 0.13.0 release because... > >
[jira] [Commented] (PIO-86) Maintain a working docker container
[ https://issues.apache.org/jira/browse/PIO-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620012#comment-16620012 ] ASF GitHub Bot commented on PIO-86: --- Github user marevol commented on the issue: https://github.com/apache/predictionio/pull/462 > your plan is that creating whole PredictionIO enviornment indcluding Spark cluster and storage cluster on the Kubernetes cluster Yes, and also you can use external clusters to specify it in docker-compose.[meta|event].yml. > Maintain a working docker container > --- > > Key: PIO-86 > URL: https://issues.apache.org/jira/browse/PIO-86 > Project: PredictionIO > Issue Type: Wish >Reporter: Sara Asher >Priority: Major > Labels: needs-discussion > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
