[jira] [Created] (PIO-209) Upgrade default build versions
Takako Shimamoto created PIO-209: Summary: Upgrade default build versions Key: PIO-209 URL: https://issues.apache.org/jira/browse/PIO-209 Project: PredictionIO Issue Type: Improvement Components: Build Affects Versions: 0.14.0 Reporter: Takako Shimamoto Most users use pre-built binary distribution and would like to run on new versions as much as possible. before: Spark 2.1.3 Elasticsearch 5.6.9 Hbase 1.2.6 after: Spark 2.4.3 Elasticsearch 6.8.0 Hbase 1.4.10 Also, Elasticsearch 5 support is deprecated because of EOL (https://www.elastic.co/support/eol). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PIO-181) Drop Elasticsearch 1.x support
[ https://issues.apache.org/jira/browse/PIO-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto resolved PIO-181. -- Resolution: Fixed Assignee: Takako Shimamoto Fix Version/s: 0.15.0 > Drop Elasticsearch 1.x support > -- > > Key: PIO-181 > URL: https://issues.apache.org/jira/browse/PIO-181 > Project: PredictionIO > Issue Type: Task > Components: Build, Core, Documentation >Affects Versions: 0.14.0 >Reporter: Takako Shimamoto >Assignee: Takako Shimamoto >Priority: Major > Fix For: 0.15.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Support for Elasticsearch 1.x will be removed after 0.14.0 release. > See PIO-175 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PIO-207) Upgrade Docker image to PIO 0.14.0
Takako Shimamoto created PIO-207: Summary: Upgrade Docker image to PIO 0.14.0 Key: PIO-207 URL: https://issues.apache.org/jira/browse/PIO-207 Project: PredictionIO Issue Type: Task Affects Versions: 0.14.0 Reporter: Takako Shimamoto -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PIO-187) Livedoc and Develop-Readme Docker Doc Update
[ https://issues.apache.org/jira/browse/PIO-187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto resolved PIO-187. -- Resolution: Fixed Target Version/s: Live Documentation (was: Future, Live Documentation) > Livedoc and Develop-Readme Docker Doc Update > > > Key: PIO-187 > URL: https://issues.apache.org/jira/browse/PIO-187 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Affects Versions: Future, Live Documentation >Reporter: Wei Chen >Priority: Minor > Fix For: Live Documentation > > > Docker support had been provided through > [#462|https://github.com/apache/predictionio/pull/462] > Based on the discussion with [~shinsuke] in PIO-86 > We create another issue to track the modification of livedoc and develop > readme. > In this issue: > 1. Create a new Docker installation page in livedoc > 2. Modify the Docker installation link in livedoc and develop readme -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-187) Livedoc and Develop-Readme Docker Doc Update
[ https://issues.apache.org/jira/browse/PIO-187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-187: - Fix Version/s: (was: Future) > Livedoc and Develop-Readme Docker Doc Update > > > Key: PIO-187 > URL: https://issues.apache.org/jira/browse/PIO-187 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Affects Versions: Future, Live Documentation >Reporter: Wei Chen >Priority: Minor > Fix For: Live Documentation > > > Docker support had been provided through > [#462|https://github.com/apache/predictionio/pull/462] > Based on the discussion with [~shinsuke] in PIO-86 > We create another issue to track the modification of livedoc and develop > readme. > In this issue: > 1. Create a new Docker installation page in livedoc > 2. Modify the Docker installation link in livedoc and develop readme -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (PIO-187) Livedoc and Develop-Readme Docker Doc Update
[ https://issues.apache.org/jira/browse/PIO-187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto reopened PIO-187: -- Assignee: (was: Wei Chen) > Livedoc and Develop-Readme Docker Doc Update > > > Key: PIO-187 > URL: https://issues.apache.org/jira/browse/PIO-187 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Affects Versions: Future, Live Documentation >Reporter: Wei Chen >Priority: Minor > Fix For: Future, Live Documentation > > > Docker support had been provided through > [#462|https://github.com/apache/predictionio/pull/462] > Based on the discussion with [~shinsuke] in PIO-86 > We create another issue to track the modification of livedoc and develop > readme. > In this issue: > 1. Create a new Docker installation page in livedoc > 2. Modify the Docker installation link in livedoc and develop readme -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PIO-186) Github description update spray to akka-http
[ https://issues.apache.org/jira/browse/PIO-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto resolved PIO-186. -- Resolution: Fixed > Github description update spray to akka-http > > > Key: PIO-186 > URL: https://issues.apache.org/jira/browse/PIO-186 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Reporter: Wei Chen >Priority: Major > > In our Github page: https://github.com/apache/predictionio > In the project description (the line on top of the status bar) > We see: > *PredictionIO, a machine learning server for developers and ML engineers. > Built on Apache Spark, HBase and Spray.* > which should be updated from *Spray* to *Akka-Http* > *PredictionIO, a machine learning server for developers and ML engineers. > Built on Apache Spark, HBase, and Akka-Http.* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-186) Github description update spray to akka-http
[ https://issues.apache.org/jira/browse/PIO-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773789#comment-16773789 ] Takako Shimamoto commented on PIO-186: -- Done https://issues.apache.org/jira/browse/INFRA-17880 > Github description update spray to akka-http > > > Key: PIO-186 > URL: https://issues.apache.org/jira/browse/PIO-186 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Reporter: Wei Chen >Priority: Major > > In our Github page: https://github.com/apache/predictionio > In the project description (the line on top of the status bar) > We see: > *PredictionIO, a machine learning server for developers and ML engineers. > Built on Apache Spark, HBase and Spray.* > which should be updated from *Spray* to *Akka-Http* > *PredictionIO, a machine learning server for developers and ML engineers. > Built on Apache Spark, HBase, and Akka-Http.* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-170) Upgrade sbt to 1.x
[ https://issues.apache.org/jira/browse/PIO-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-170: - Fix Version/s: 0.14.0 > Upgrade sbt to 1.x > -- > > Key: PIO-170 > URL: https://issues.apache.org/jira/browse/PIO-170 > Project: PredictionIO > Issue Type: Task > Components: Core >Affects Versions: 0.13.0 >Reporter: Naoki Takezoe >Assignee: Naoki Takezoe >Priority: Major > Fix For: 0.14.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-194) S3 Model Data Storage should allow more flexible ways for specifying AWS credentials
[ https://issues.apache.org/jira/browse/PIO-194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-194: - Fix Version/s: 0.14.0 > S3 Model Data Storage should allow more flexible ways for specifying AWS > credentials > > > Key: PIO-194 > URL: https://issues.apache.org/jira/browse/PIO-194 > Project: PredictionIO > Issue Type: Improvement > Components: Core >Affects Versions: 0.13.0 >Reporter: Chris Wewerka >Assignee: Chris Wewerka >Priority: Major > Fix For: 0.14.0 > > > S3 Storage for Model metadata storage in AWS S3 currently uses > ProfileCredentialsProvider with the default profile. Switching to > DefaultAWSCredentialsProviderChain which also includes > ProfileCredentialsProvider as one step to get credentials,is a more flexible > solution. > It also allows e.g. for AWS Instance Profile Roles and thus is more flexible > and does not force developers to include credentials on their PredictionIO > machines. > https://github.com/apache/predictionio/pull/496 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-172) Migration guide for ES 6.x changes
[ https://issues.apache.org/jira/browse/PIO-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-172: - Summary: Migration guide for ES 6.x changes (was: Migration tool for ES 6.x changes) > Migration guide for ES 6.x changes > -- > > Key: PIO-172 > URL: https://issues.apache.org/jira/browse/PIO-172 > Project: PredictionIO > Issue Type: New Feature > Components: Core >Reporter: Donald Szeto >Assignee: Takako Shimamoto >Priority: Major > Fix For: 0.14.0 > > Time Spent: 20m > Remaining Estimate: 0h > > With PIO-168, the metadata model of ES 6.x will change to reflect > [changes|https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html] > that will become effective with ES 7.x and onward. A migration tool should > be provided for users who want to upgrade to the latest version of PIO with > ES, and keeping metadata intact. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-186) Github description update spray to akka-http
[ https://issues.apache.org/jira/browse/PIO-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-186: - Priority: Major (was: Minor) > Github description update spray to akka-http > > > Key: PIO-186 > URL: https://issues.apache.org/jira/browse/PIO-186 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Reporter: Wei Chen >Priority: Major > > In our Github page: https://github.com/apache/predictionio > In the project description (the line on top of the status bar) > We see: > *PredictionIO, a machine learning server for developers and ML engineers. > Built on Apache Spark, HBase and Spray.* > which should be updated from *Spray* to *Akka-Http* > *PredictionIO, a machine learning server for developers and ML engineers. > Built on Apache Spark, HBase, and Akka-Http.* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-186) Github description update spray to akka-http
[ https://issues.apache.org/jira/browse/PIO-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772909#comment-16772909 ] Takako Shimamoto commented on PIO-186: -- I will create an INFRA ticket. I think it'd be better if library names aren't included in GitHub description like this, "PredictionIO, a machine learning server for developers and ML engineers. https://predictionio.apache.org; > Github description update spray to akka-http > > > Key: PIO-186 > URL: https://issues.apache.org/jira/browse/PIO-186 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Reporter: Wei Chen >Priority: Minor > > In our Github page: https://github.com/apache/predictionio > In the project description (the line on top of the status bar) > We see: > *PredictionIO, a machine learning server for developers and ML engineers. > Built on Apache Spark, HBase and Spray.* > which should be updated from *Spray* to *Akka-Http* > *PredictionIO, a machine learning server for developers and ML engineers. > Built on Apache Spark, HBase, and Akka-Http.* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PIO-175) Deprecation of Elasticsearch 1.x support
[ https://issues.apache.org/jira/browse/PIO-175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto resolved PIO-175. -- Resolution: Fixed Fix Version/s: 0.14.0 > Deprecation of Elasticsearch 1.x support > > > Key: PIO-175 > URL: https://issues.apache.org/jira/browse/PIO-175 > Project: PredictionIO > Issue Type: Task > Components: Documentation >Affects Versions: 0.13.0 >Reporter: Naoki Takezoe >Assignee: Takako Shimamoto >Priority: Major > Fix For: 0.14.0 > > > Mark Elasticsearch 1.x support as deprecated and announce the future deletion > in 0.14.0. > It will be deleted actually in 0.15.0 from our codebase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PIO-168) Elasticsearch 6.x support
[ https://issues.apache.org/jira/browse/PIO-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto resolved PIO-168. -- Resolution: Fixed > Elasticsearch 6.x support > - > > Key: PIO-168 > URL: https://issues.apache.org/jira/browse/PIO-168 > Project: PredictionIO > Issue Type: New Feature > Components: Core >Affects Versions: 0.13.0 >Reporter: Donald Szeto >Assignee: Alexander Merritt >Priority: Major > Fix For: 0.14.0 > > > This is a JIRA ticket for tracking the pull request from [~emergentorder] > that adds support of Elasticsearch 6.x. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PIO-172) Migration tool for ES 6.x changes
[ https://issues.apache.org/jira/browse/PIO-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto resolved PIO-172. -- Resolution: Fixed Assignee: Takako Shimamoto (was: saurabh gulati) Fix Version/s: 0.14.0 > Migration tool for ES 6.x changes > - > > Key: PIO-172 > URL: https://issues.apache.org/jira/browse/PIO-172 > Project: PredictionIO > Issue Type: New Feature > Components: Core >Reporter: Donald Szeto >Assignee: Takako Shimamoto >Priority: Major > Fix For: 0.14.0 > > Time Spent: 10m > Remaining Estimate: 0h > > With PIO-168, the metadata model of ES 6.x will change to reflect > [changes|https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html] > that will become effective with ES 7.x and onward. A migration tool should > be provided for users who want to upgrade to the latest version of PIO with > ES, and keeping metadata intact. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PIO-195) Improve readability and grammar of documentation
[ https://issues.apache.org/jira/browse/PIO-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto resolved PIO-195. -- Resolution: Fixed Fix Version/s: 0.14.0 > Improve readability and grammar of documentation > > > Key: PIO-195 > URL: https://issues.apache.org/jira/browse/PIO-195 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Affects Versions: 0.13.0 >Reporter: Takako Shimamoto >Priority: Major > Fix For: 0.14.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PIO-205) Update Dockerfile to reflect new Spark version
[
https://issues.apache.org/jira/browse/PIO-205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takako Shimamoto resolved PIO-205.
--
Resolution: Fixed
Fix Version/s: (was: Future)
0.14.0
> Update Dockerfile to reflect new Spark version
> --
>
> Key: PIO-205
> URL: https://issues.apache.org/jira/browse/PIO-205
> Project: PredictionIO
> Issue Type: Bug
> Components: Build
>Reporter: Faris bin Faris
>Priority: Minor
> Fix For: 0.14.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> new release of Spark(2.2.3) making the old mirror(2.2.2) expired and throws
> an error when building.
>
>
> {code:java}
> RUN curl -o /opt/src/spark-$SPARK_VERSION.tgz
> http://www-us.apache.org/dist/spark/spark-$SPARK_VERSION/spark-$SPARK_VERSION-bin-hadoop2.7.tgz
> && tar zxvf /opt/src/spark-$SPARK_VERSION.tgz && echo "SPARK_HOME="`pwd`/`ls
> -d spark*` >> /etc/predictionio/pio-env.sh && rm -rf /opt/src ---> Running in
> b4603a9775ce
> gzip: stdin: not in gzip format
> tar: Child returned status 1
> tar: Error is not recoverable: exiting now{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (PIO-199) Spark 2.4 (Scala 2.11) support
[ https://issues.apache.org/jira/browse/PIO-199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto resolved PIO-199. -- Resolution: Fixed Fix Version/s: 0.14.0 > Spark 2.4 (Scala 2.11) support > -- > > Key: PIO-199 > URL: https://issues.apache.org/jira/browse/PIO-199 > Project: PredictionIO > Issue Type: Improvement >Reporter: Takako Shimamoto >Assignee: Takako Shimamoto >Priority: Major > Fix For: 0.14.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Though Spark 2.4 has added Scala 2.12 support, it is marked as experimental > on the Spark release notes, so the next version of PIO will only add Spark > 2.4 (Scala 2.11) support. > https://spark.apache.org/releases/spark-release-2-4-0.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (PIO-199) Spark 2.4 (Scala 2.11) support
[ https://issues.apache.org/jira/browse/PIO-199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto reassigned PIO-199: Assignee: Takako Shimamoto > Spark 2.4 (Scala 2.11) support > -- > > Key: PIO-199 > URL: https://issues.apache.org/jira/browse/PIO-199 > Project: PredictionIO > Issue Type: Improvement >Reporter: Takako Shimamoto >Assignee: Takako Shimamoto >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Though Spark 2.4 has added Scala 2.12 support, it is marked as experimental > on the Spark release notes, so the next version of PIO will only add Spark > 2.4 (Scala 2.11) support. > https://spark.apache.org/releases/spark-release-2-4-0.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PIO-206) Spark 2.3.2 to 2.3.3
Takako Shimamoto created PIO-206: Summary: Spark 2.3.2 to 2.3.3 Key: PIO-206 URL: https://issues.apache.org/jira/browse/PIO-206 Project: PredictionIO Issue Type: Task Components: Build Reporter: Takako Shimamoto Spark 2.3.3 will be released shortly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (PIO-204) pio deploy --engine-instance-id doesn't work
[
https://issues.apache.org/jira/browse/PIO-204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takako Shimamoto closed PIO-204.
Resolution: Duplicate
The bug is already fixed via
[#507|https://github.com/apache/predictionio/pull/507] as a part of PIO-203.
> pio deploy --engine-instance-id doesn't work
>
>
> Key: PIO-204
> URL: https://issues.apache.org/jira/browse/PIO-204
> Project: PredictionIO
> Issue Type: Sub-task
>Affects Versions: 0.14.0
>Reporter: Takako Shimamoto
>Priority: Major
>
> {code:sh}
> bash-4.4$ curl -XGET 'http://localhost:9600/'
> {
> "name" : "09B4kBg",
> "cluster_name" : "predictionio6",
> "cluster_uuid" : "S6Dljs0MTPqg7Ivj_plewg",
> "version" : {
> "number" : "6.5.4",
> "build_flavor" : "default",
> "build_type" : "tar",
> "build_hash" : "d2ef93d",
> "build_date" : "2018-12-17T21:17:40.758843Z",
> "build_snapshot" : false,
> "lucene_version" : "7.5.0",
> "minimum_wire_compatibility_version" : "5.6.0",
> "minimum_index_compatibility_version" : "5.0.0"
> },
> "tagline" : "You Know, for Search"
> }
> bash-4.4$ curl -H "Content-Type: application/json" -XGET
> http://localhost:9600/pio_meta_engine_instances/_search -d '
> {
> "query" : {
> "match_all" : {}
> }
> }'
> {"took":188,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":1,"max_score":1.0,"hits":[{"_index":"pio_meta_engine_instances","_type":"engine_instances","_id":"AWhQ9y_8C4sx-GpWCc6N","_score":1.0,"_source":{"id":"AWhQ9y_8C4sx-GpWCc6N","status":"COMPLETED","startTime":"2019-01-15T10:03:22.275Z","endTime":"2019-01-15T10:03:37.235Z","engineId":"org.example.recommendation.RecommendationEngine","engineVersion":"6a2841579d2c44559cafedbf97d19cb57b37eec2","engineVariant":"default","engineFactory":"org.example.recommendation.RecommendationEngine","batch":"","env":{"PIO_ENV_LOADED":"1","PIO_STORAGE_REPOSITORIES_METADATA_NAME":"pio_meta","PIO_FS_BASEDIR":"/home/pio/.pio_store","PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS":"localhost","PIO_HOME":"/predictionio","PIO_FS_ENGINESDIR":"/home/pio/.pio_store/engines","PIO_STORAGE_SOURCES_LOCALFS_PATH":"/home/pio/.pio_store/models","PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE":"elasticsearch","PIO_STORAGE_REPOSITORIES_METADATA_SOURCE":"ELASTICSEARCH","PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE":"LOCALFS","PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME":"pio_event","PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME":"predictionio","PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME":"/vendors/elasticsearch","PIO_FS_TMPDIR":"/home/pio/.pio_store/tmp","PIO_STORAGE_REPOSITORIES_MODELDATA_NAME":"pio_model","PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE":"ELASTICSEARCH","PIO_CONF_DIR":"/predictionio/conf","PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS":"9200","PIO_STORAGE_SOURCES_LOCALFS_TYPE":"localfs"},"sparkConf":{"spark.executor.extraClassPath":"."},"dataSourceParams":"{\"\":{\"appName\":\"MyApp1\"}}","preparatorParams":"{\"\":{}}","algorithmsParams":"[{\"als\":{\"rank\":10,\"numIterations\":20,\"lambda\":0.01,\"seed\":3}}]","servingParams":"{\"\":{}}"}}]}}
> bash-4.4$ pio deploy --engine-instance-id AWhQ9y_8C4sx-GpWCc6N
> [WARN] [RestClient] request [PUT http://localhost:9600/pio_meta] returned 1
> warnings: [299 Elasticsearch-6.5.4-d2ef93d "the default number of shards will
> change from [5] to [1] in 7.0.0; if you wish to continue using the default of
> [5] shards, you must manage this on the create index request or with an index
> template" "Tue, 15 Jan 2019 10:24:14 GMT"]
> [WARN] [RestClient] request [PUT
> http://localhost:9600/pio_meta/_mapping/engine_instances] returned 1
> warnings: [299 Elasticsearch-6.5.4-d2ef93d "[_all] is deprecated in 6.0+ and
> will be removed in 7.0. As a replacement, you can use [copy_to] on mapping
> fields to create your own catch all field." "Tue, 15 Jan 2019 10:24:14 GMT"]
> [ERROR] [Engine$] Invalid engine instance ID AWhQ9y_8C4sx-GpWCc6N. Aborting.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (PIO-204) pio deploy --engine-instance-id doesn't work
Takako Shimamoto created PIO-204:
Summary: pio deploy --engine-instance-id doesn't work
Key: PIO-204
URL: https://issues.apache.org/jira/browse/PIO-204
Project: PredictionIO
Issue Type: Sub-task
Affects Versions: 0.14.0
Reporter: Takako Shimamoto
{code:sh}
bash-4.4$ curl -XGET 'http://localhost:9600/'
{
"name" : "09B4kBg",
"cluster_name" : "predictionio6",
"cluster_uuid" : "S6Dljs0MTPqg7Ivj_plewg",
"version" : {
"number" : "6.5.4",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "d2ef93d",
"build_date" : "2018-12-17T21:17:40.758843Z",
"build_snapshot" : false,
"lucene_version" : "7.5.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
bash-4.4$ curl -H "Content-Type: application/json" -XGET
http://localhost:9600/pio_meta_engine_instances/_search -d '
{
"query" : {
"match_all" : {}
}
}'
{"took":188,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":1,"max_score":1.0,"hits":[{"_index":"pio_meta_engine_instances","_type":"engine_instances","_id":"AWhQ9y_8C4sx-GpWCc6N","_score":1.0,"_source":{"id":"AWhQ9y_8C4sx-GpWCc6N","status":"COMPLETED","startTime":"2019-01-15T10:03:22.275Z","endTime":"2019-01-15T10:03:37.235Z","engineId":"org.example.recommendation.RecommendationEngine","engineVersion":"6a2841579d2c44559cafedbf97d19cb57b37eec2","engineVariant":"default","engineFactory":"org.example.recommendation.RecommendationEngine","batch":"","env":{"PIO_ENV_LOADED":"1","PIO_STORAGE_REPOSITORIES_METADATA_NAME":"pio_meta","PIO_FS_BASEDIR":"/home/pio/.pio_store","PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS":"localhost","PIO_HOME":"/predictionio","PIO_FS_ENGINESDIR":"/home/pio/.pio_store/engines","PIO_STORAGE_SOURCES_LOCALFS_PATH":"/home/pio/.pio_store/models","PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE":"elasticsearch","PIO_STORAGE_REPOSITORIES_METADATA_SOURCE":"ELASTICSEARCH","PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE":"LOCALFS","PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME":"pio_event","PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME":"predictionio","PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME":"/vendors/elasticsearch","PIO_FS_TMPDIR":"/home/pio/.pio_store/tmp","PIO_STORAGE_REPOSITORIES_MODELDATA_NAME":"pio_model","PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE":"ELASTICSEARCH","PIO_CONF_DIR":"/predictionio/conf","PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS":"9200","PIO_STORAGE_SOURCES_LOCALFS_TYPE":"localfs"},"sparkConf":{"spark.executor.extraClassPath":"."},"dataSourceParams":"{\"\":{\"appName\":\"MyApp1\"}}","preparatorParams":"{\"\":{}}","algorithmsParams":"[{\"als\":{\"rank\":10,\"numIterations\":20,\"lambda\":0.01,\"seed\":3}}]","servingParams":"{\"\":{}}"}}]}}
bash-4.4$ pio deploy --engine-instance-id AWhQ9y_8C4sx-GpWCc6N
[WARN] [RestClient] request [PUT http://localhost:9600/pio_meta] returned 1
warnings: [299 Elasticsearch-6.5.4-d2ef93d "the default number of shards will
change from [5] to [1] in 7.0.0; if you wish to continue using the default of
[5] shards, you must manage this on the create index request or with an index
template" "Tue, 15 Jan 2019 10:24:14 GMT"]
[WARN] [RestClient] request [PUT
http://localhost:9600/pio_meta/_mapping/engine_instances] returned 1 warnings:
[299 Elasticsearch-6.5.4-d2ef93d "[_all] is deprecated in 6.0+ and will be
removed in 7.0. As a replacement, you can use [copy_to] on mapping fields to
create your own catch all field." "Tue, 15 Jan 2019 10:24:14 GMT"]
[ERROR] [Engine$] Invalid engine instance ID AWhQ9y_8C4sx-GpWCc6N. Aborting.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PIO-203) pio status warnings
[
https://issues.apache.org/jira/browse/PIO-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takako Shimamoto updated PIO-203:
-
Description:
* just after reindex
{code:sh}
bash-4.4$ curl -XGET 'http://localhost:9600/'
{
"name" : "SQ7XrzJ",
"cluster_name" : "predictionio6",
"cluster_uuid" : "aGsKCsBQS1SNOx_eQgF5xw",
"version" : {
"number" : "6.5.4",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "d2ef93d",
"build_date" : "2018-12-17T21:17:40.758843Z",
"build_snapshot" : false,
"lucene_version" : "7.5.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
bash-4.4$ curl -XGET 'http://localhost:9600/_cat/indices?v'
health status index uuid pri rep
docs.count docs.deleted store.size pri.store.size
yellow open pio_meta_accesskeys aGsGXbKoTRSmFlMVx-Pw_Q 5 1
10 5.1kb 5.1kb
yellow open pio_meta_sequencesW2U5YMuAQLKBXpjF-cEh0A 5 1
10 4.3kb 4.3kb
yellow open pio_meta_apps GTzwXqe5T4SbR8StVjt7Lg 5 1
10 4.5kb 4.5kb
yellow open pio_event_1 onM3TD4HSJiEDESV6So7iQ 5 1
15010297.5kb297.5kb
yellow open pio_meta_engine_instances cv939Z13TJWz3LA6X-QjYQ 5 1
10 28.2kb 28.2kb
{code}
* check the status
{code:sh}
bash-4.4$ pio status
[INFO] [Management$] Inspecting PredictionIO...
[INFO] [Management$] PredictionIO 0.14.0-SNAPSHOT is installed at
/PredictionIO-0.14.0-SNAPSHOT
[INFO] [Management$] Inspecting Apache Spark...
[INFO] [Management$] Apache Spark is installed at
/PredictionIO-0.14.0-SNAPSHOT/vendors/spark-2.3.2-bin-hadoop2.7
[INFO] [Management$] Apache Spark 2.3.2 detected (meets minimum requirement of
2.0.2)
[INFO] [Management$] Inspecting storage backend connections...
[INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
[WARN] [RestClient] request [PUT http://localhost:9600/pio_meta] returned 1
warnings: [299 Elasticsearch-6.5.4-d2ef93d "the default number of shards will
change from [5] to [1] in 7.0.0; if you wish to continue using the default of
[5] shards, you must manage this on the create index request or with an index
template" "Tue, 15 Jan 2019 07:47:40 GMT"]
[WARN] [RestClient] request [PUT
http://localhost:9600/pio_meta/_mapping/engine_instances] returned 1 warnings:
[299 Elasticsearch-6.5.4-d2ef93d "[_all] is deprecated in 6.0+ and will be
removed in 7.0. As a replacement, you can use [copy_to] on mapping fields to
create your own catch all field." "Tue, 15 Jan 2019 07:47:40 GMT"]
[WARN] [RestClient] request [PUT http://localhost:9600/null_sequences] returned
1 warnings: [299 Elasticsearch-6.5.4-d2ef93d "the default number of shards will
change from [5] to [1] in 7.0.0; if you wish to continue using the default of
[5] shards, you must manage this on the create index request or with an index
template" "Tue, 15 Jan 2019 07:47:40 GMT"]
[WARN] [RestClient] request [PUT
http://localhost:9600/null_sequences/_mapping/sequences] returned 1 warnings:
[299 Elasticsearch-6.5.4-d2ef93d "[_all] is deprecated in 6.0+ and will be
removed in 7.0. As a replacement, you can use [copy_to] on mapping fields to
create your own catch all field." "Tue, 15 Jan 2019 07:47:41 GMT"]
[WARN] [RestClient] request [PUT
http://localhost:9600/pio_meta_evaluation_instances] returned 1 warnings: [299
Elasticsearch-6.5.4-d2ef93d "the default number of shards will change from [5]
to [1] in 7.0.0; if you wish to continue using the default of [5] shards, you
must manage this on the create index request or with an index template" "Tue,
15 Jan 2019 07:47:41 GMT"]
[WARN] [RestClient] request [PUT
http://localhost:9600/pio_meta_evaluation_instances/_mapping/evaluation_instances]
returned 1 warnings: [299 Elasticsearch-6.5.4-d2ef93d "[_all] is deprecated in
6.0+ and will be removed in 7.0. As a replacement, you can use [copy_to] on
mapping fields to create your own catch all field." "Tue, 15 Jan 2019 07:47:41
GMT"]
[WARN] [RestClient] request [PUT http://localhost:9600/pio_meta_apps_sequences]
returned 1 warnings: [299 Elasticsearch-6.5.4-d2ef93d "the default number of
shards will change from [5] to [1] in 7.0.0; if you wish to continue using the
default of [5] shards, you must manage this on the create index request or with
an index template" "Tue, 15 Jan 2019 07:47:41 GMT"]
[WARN] [RestClient] request [PUT
http://localhost:9600/pio_meta_apps_sequences/_mapping/sequences] returned 1
warnings: [299 Elasticsearch-6.5.4-d2ef93d "[_all] is deprecated in 6.0+ and
will be removed in 7.0. As a replacement, you can use [copy_to] on mapping
fields to create your own catch all field." "Tue, 15 Jan 2019
[jira] [Commented] (PIO-172) Migration tool for ES 6.x changes
[ https://issues.apache.org/jira/browse/PIO-172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740108#comment-16740108 ] Takako Shimamoto commented on PIO-172: -- Users can choose Elasticsearch for EVENTDATA. I guess it's almost impossible to provide a migration tool because an Elasticsearch cluster which is made up of a number of nodes in production probably has various configuration settings and complex cluster topologies. I think it's not that easy to resolve this change with the tool. As far as I'm concerned, it's realistic to provide just a guide which explains how to migrate current indexes by reindexing. like this, https://www.elastic.co/guide/en/elasticsearch/reference/6.0/reindex-upgrade-inplace.html I'm checking the reindex process now. > Migration tool for ES 6.x changes > - > > Key: PIO-172 > URL: https://issues.apache.org/jira/browse/PIO-172 > Project: PredictionIO > Issue Type: New Feature > Components: Core >Reporter: Donald Szeto >Assignee: saurabh gulati >Priority: Major > > With PIO-168, the metadata model of ES 6.x will change to reflect > [changes|https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html] > that will become effective with ES 7.x and onward. A migration tool should > be provided for users who want to upgrade to the latest version of PIO with > ES, and keeping metadata intact. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-199) Spark 2.4 (Scala 2.11) support
[ https://issues.apache.org/jira/browse/PIO-199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723747#comment-16723747 ] Takako Shimamoto commented on PIO-199: -- Since Spark 2.4, Spark has upgraded json4s to 3.5.3. As a result, json4s doesn't depend on scala-lang's scalap; that is, it isn't added dependencies on scala-compiler and scala-reflect as well. https://issues.apache.org/jira/browse/SPARK-23043 [https://github.com/json4s/json4s/pull/180] > Spark 2.4 (Scala 2.11) support > -- > > Key: PIO-199 > URL: https://issues.apache.org/jira/browse/PIO-199 > Project: PredictionIO > Issue Type: Improvement >Reporter: Takako Shimamoto >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Though Spark 2.4 has added Scala 2.12 support, it is marked as experimental > on the Spark release notes, so the next version of PIO will only add Spark > 2.4 (Scala 2.11) support. > https://spark.apache.org/releases/spark-release-2-4-0.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PIO-199) Spark 2.4 (Scala 2.11) support
Takako Shimamoto created PIO-199: Summary: Spark 2.4 (Scala 2.11) support Key: PIO-199 URL: https://issues.apache.org/jira/browse/PIO-199 Project: PredictionIO Issue Type: Improvement Reporter: Takako Shimamoto Though Spark 2.4 has added Scala 2.12 support, it is marked as experimental on the Spark release notes, so the next version of PIO will only add Spark 2.4 (Scala 2.11) support. https://spark.apache.org/releases/spark-release-2-4-0.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (PIO-196) Use external PySpark environment variables in Jupyter Docker image
[ https://issues.apache.org/jira/browse/PIO-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto resolved PIO-196. -- Resolution: Fixed > Use external PySpark environment variables in Jupyter Docker image > -- > > Key: PIO-196 > URL: https://issues.apache.org/jira/browse/PIO-196 > Project: PredictionIO > Issue Type: Improvement >Reporter: Shinsuke Sugaya >Assignee: Shinsuke Sugaya >Priority: Minor > Fix For: 0.14.0 > > > - Set PYSPARK_DRIVER_PYTHON* values at runtime > - Ignore a token for Jupyter -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-196) Use external PySpark environment variables in Jupyter Docker image
[ https://issues.apache.org/jira/browse/PIO-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-196: - Fix Version/s: 0.14.0 > Use external PySpark environment variables in Jupyter Docker image > -- > > Key: PIO-196 > URL: https://issues.apache.org/jira/browse/PIO-196 > Project: PredictionIO > Issue Type: Improvement >Reporter: Shinsuke Sugaya >Assignee: Shinsuke Sugaya >Priority: Minor > Fix For: 0.14.0 > > > - Set PYSPARK_DRIVER_PYTHON* values at runtime > - Ignore a token for Jupyter -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-192) Enhance PySpark support
[
https://issues.apache.org/jira/browse/PIO-192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takako Shimamoto updated PIO-192:
-
Description:
h3. Summary
Enhance the pypio, which is the Python API for PIO.
h3. Goals
The limitations of the current Python support always force developers to have
access to sbt. This enhancement will get rid of the build phase.
h3. Description
A Python engine has nothing to need. Developers can use the pypio module with
jupyter notebook and Python code.
First, import the necessary modules.
{code:python}
import pypio
{code}
Once the module in imported, the first step is to initialize the pypio module.
{code:python}
pypio.init()
{code}
Next, find data from the event store.
{code:python}
event_df = pypio.find_events('BHPApp')
{code}
And then, save the model.
{code:python}
# model is a PipelineModel, which is produced after a Pipeline’s fit() method
runs
pipeline = Pipeline(...)
model = pipeline.fit(train_df)
engine_instance_id = pypio.save_model(model, ["prediction"])
{code}
h4. Run & Deploy
h5. Run Jupyter
{code:sh}
pio-shell --with-pyspark
{code}
h5. Run on Spark
{code:sh}
pio train --main-py-file .py
{code}
h5. Deploy App
{code:sh}
pio deploy --engine-instance-id
{code}
was:
h3. Summary
Enhance the pypio, which is the Python API for PIO.
h3. Goals
The limitations of the current Python support always force developers to have
access to sbt. This enhancement will get rid of the build phase.
h3. Description
A Python engine has nothing to need. Developers can use the pypio module with
jupyter notebook and Python code.
First, import the necessary modules.
{code:python}
import pypio
{code}
Once the module in imported, the first step is to initialize the pypio module.
{code:python}
# not create App
pypio.init()
# create App (pio app new BHPApp)
pypio.init('BHPApp')
{code}
Next, find data from the event store.
{code:python}
event_df = pypio.find_events('BHPApp')
{code}
And then, save the model.
{code:python}
# model is a PipelineModel, which is produced after a Pipeline’s fit() method
runs
pipeline = Pipeline(...)
model = pipeline.fit(train_df)
engine_instance_id = pypio.save_model(model, ["prediction"])
{code}
h4. Run & Deploy
h5. Run Jupyter
{code:sh}
pio-shell --with-pyspark
{code}
h5. Run on Spark
{code:sh}
pio train --main-py-file .py
{code}
h5. Deploy App
{code:sh}
pio deploy --engine-instance-id
{code}
> Enhance PySpark support
> ---
>
> Key: PIO-192
> URL: https://issues.apache.org/jira/browse/PIO-192
> Project: PredictionIO
> Issue Type: Improvement
> Components: Core
>Affects Versions: 0.13.0
>Reporter: Takako Shimamoto
>Assignee: Takako Shimamoto
>Priority: Major
>
> h3. Summary
> Enhance the pypio, which is the Python API for PIO.
> h3. Goals
> The limitations of the current Python support always force developers to have
> access to sbt. This enhancement will get rid of the build phase.
> h3. Description
> A Python engine has nothing to need. Developers can use the pypio module with
> jupyter notebook and Python code.
> First, import the necessary modules.
> {code:python}
> import pypio
> {code}
> Once the module in imported, the first step is to initialize the pypio module.
> {code:python}
> pypio.init()
> {code}
> Next, find data from the event store.
> {code:python}
> event_df = pypio.find_events('BHPApp')
> {code}
> And then, save the model.
> {code:python}
> # model is a PipelineModel, which is produced after a Pipeline’s fit() method
> runs
> pipeline = Pipeline(...)
> model = pipeline.fit(train_df)
> engine_instance_id = pypio.save_model(model, ["prediction"])
> {code}
> h4. Run & Deploy
> h5. Run Jupyter
> {code:sh}
> pio-shell --with-pyspark
> {code}
> h5. Run on Spark
> {code:sh}
> pio train --main-py-file .py
> {code}
> h5. Deploy App
> {code:sh}
> pio deploy --engine-instance-id
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PIO-192) Enhance PySpark support
[
https://issues.apache.org/jira/browse/PIO-192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takako Shimamoto updated PIO-192:
-
Description:
h3. Summary
Enhance the pypio, which is the Python API for PIO.
h3. Goals
The limitations of the current Python support always force developers to have
access to sbt. This enhancement will get rid of the build phase.
h3. Description
A Python engine has nothing to need. Developers can use the pypio module with
jupyter notebook and Python code.
First, import the necessary modules.
{code:python}
import pypio
{code}
Once the module in imported, the first step is to initialize the pypio module.
{code:python}
# not create App
pypio.init()
# create App (pio app new BHPApp)
pypio.init('BHPApp')
{code}
Next, find data from the event store.
{code:python}
event_df = pypio.find_events('BHPApp')
{code}
And then, save the model.
{code:python}
# model is a PipelineModel, which is produced after a Pipeline’s fit() method
runs
pipeline = Pipeline(...)
model = pipeline.fit(train_df)
engine_instance_id = pypio.save_model(model, ["prediction"])
{code}
h4. Run & Deploy
h5. Run Jupyter
{code:sh}
pio-shell --with-pyspark
{code}
h5. Run on Spark
{code:sh}
pio train --main-py-file .py
{code}
h5. Deploy App
{code:sh}
pio deploy --engine-instance-id
{code}
was:
h3. Summary
Enhance the pypio, which is the Python API for PIO.
h3. Goals
The limitations of the current Python support always force developers to have
access to sbt. This enhancement will get rid of the build phase.
h3. Description
A Python engine has nothing to need. Developers can use the pypio module with
jupyter notebook and Python code.
First, import the necessary modules.
{code:python}
import pypio
{code}
Once the module in imported, the first step is to initialize the pypio module.
{code:python}
# not create App
pypio.init()
# create App (pio app new BHPApp)
pypio.init('BHPApp')
{code}
Next, find data from the event store.
{code:python}
event_df = pypio.find_events('BHPApp')
{code}
And then, save the model.
{code:python}
# model is a PipelineModel, which is produced after a Pipeline’s fit() method
runs
pipeline = Pipeline(...)
model = pipeline.fit(train_df)
engine_instance_id = pypio.save_model(model, ["prediction"])
{code}
> Enhance PySpark support
> ---
>
> Key: PIO-192
> URL: https://issues.apache.org/jira/browse/PIO-192
> Project: PredictionIO
> Issue Type: Improvement
> Components: Core
>Affects Versions: 0.13.0
>Reporter: Takako Shimamoto
>Assignee: Takako Shimamoto
>Priority: Major
>
> h3. Summary
> Enhance the pypio, which is the Python API for PIO.
> h3. Goals
> The limitations of the current Python support always force developers to have
> access to sbt. This enhancement will get rid of the build phase.
> h3. Description
> A Python engine has nothing to need. Developers can use the pypio module with
> jupyter notebook and Python code.
> First, import the necessary modules.
> {code:python}
> import pypio
> {code}
> Once the module in imported, the first step is to initialize the pypio module.
> {code:python}
> # not create App
> pypio.init()
> # create App (pio app new BHPApp)
> pypio.init('BHPApp')
> {code}
> Next, find data from the event store.
> {code:python}
> event_df = pypio.find_events('BHPApp')
> {code}
> And then, save the model.
> {code:python}
> # model is a PipelineModel, which is produced after a Pipeline’s fit() method
> runs
> pipeline = Pipeline(...)
> model = pipeline.fit(train_df)
> engine_instance_id = pypio.save_model(model, ["prediction"])
> {code}
> h4. Run & Deploy
> h5. Run Jupyter
> {code:sh}
> pio-shell --with-pyspark
> {code}
> h5. Run on Spark
> {code:sh}
> pio train --main-py-file .py
> {code}
> h5. Deploy App
> {code:sh}
> pio deploy --engine-instance-id
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PIO-192) Enhance PySpark support
[
https://issues.apache.org/jira/browse/PIO-192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takako Shimamoto updated PIO-192:
-
Description:
h3. Summary
Enhance the pypio, which is the Python API for PIO.
h3. Goals
The limitations of the current Python support always force developers to have
access to sbt. This enhancement will get rid of the build phase.
h3. Description
A Python engine has nothing to need. Developers can use the pypio module with
jupyter notebook and Python code.
First, import the necessary modules.
{code:python}
import pypio
{code}
Once the module in imported, the first step is to initialize the pypio module.
{code:python}
# not create App
pypio.init()
# create App (pio app new BHPApp)
pypio.init('BHPApp')
{code}
Next, find data from the event store.
{code:python}
event_df = pypio.find_events('BHPApp')
{code}
And then, save the model.
{code:python}
# model is a PipelineModel, which is produced after a Pipeline’s fit() method
runs
pipeline = Pipeline(...)
model = pipeline.fit(train_df)
engine_instance_id = pypio.save_model(model, ["prediction"])
{code}
was:
h3. Summary
Enhance the pypio, which is the Python API for PIO.
h3. Goals
The limitations of the current Python support always force developers to have
access to sbt. This enhancement will get rid of the build phase.
h3. Description
A Python engine template requires 3 files:
* Python code to specify for the --main-py-file option
* template.json
{code:json}
{"pio": {"version": { "min": "0.14.0-SNAPSHOT" }}}
{code}
* engine.json
{code:json}
{
"id": "default",
"description": "Default settings",
"engineFactory": "org.apache.predictionio.e2.engine.PythonEngine",
"algorithms": [
{
"name": "default",
"params": {
"name": "BHPApp"
}
}
],
"serving": {
"params": {
"columns": ["prediction"]
}
}
}
{code}
h4. pypio module
Developers can use the pypio module with jupyter notebook and Python code.
First, import the necessary modules.
{code:python}
from pypio import pypio
{code}
Once the module in imported, the first step is to initialize the pypio module.
{code:python}
pypio.init()
{code}
Next, find data from the event store.
{code:python}
event_df = pypio.find('BHPApp')
{code}
And then, save the model.
{code:python}
# model is a PipelineModel, which is produced after a Pipeline’s fit() method
runs
pipeline = Pipeline(...)
model = pipeline.fit(train_df)
pypio.save(model)
{code}
> Enhance PySpark support
> ---
>
> Key: PIO-192
> URL: https://issues.apache.org/jira/browse/PIO-192
> Project: PredictionIO
> Issue Type: Improvement
> Components: Core
>Affects Versions: 0.13.0
>Reporter: Takako Shimamoto
>Assignee: Takako Shimamoto
>Priority: Major
>
> h3. Summary
> Enhance the pypio, which is the Python API for PIO.
> h3. Goals
> The limitations of the current Python support always force developers to have
> access to sbt. This enhancement will get rid of the build phase.
> h3. Description
> A Python engine has nothing to need. Developers can use the pypio module with
> jupyter notebook and Python code.
> First, import the necessary modules.
> {code:python}
> import pypio
> {code}
> Once the module in imported, the first step is to initialize the pypio module.
> {code:python}
> # not create App
> pypio.init()
> # create App (pio app new BHPApp)
> pypio.init('BHPApp')
> {code}
> Next, find data from the event store.
> {code:python}
> event_df = pypio.find_events('BHPApp')
> {code}
> And then, save the model.
> {code:python}
> # model is a PipelineModel, which is produced after a Pipeline’s fit() method
> runs
> pipeline = Pipeline(...)
> model = pipeline.fit(train_df)
> engine_instance_id = pypio.save_model(model, ["prediction"])
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (PIO-195) Improve readability and grammar of documentation
Takako Shimamoto created PIO-195: Summary: Improve readability and grammar of documentation Key: PIO-195 URL: https://issues.apache.org/jira/browse/PIO-195 Project: PredictionIO Issue Type: Improvement Components: Documentation Affects Versions: 0.13.0 Reporter: Takako Shimamoto -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-192) Enhance PySpark support
[
https://issues.apache.org/jira/browse/PIO-192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takako Shimamoto updated PIO-192:
-
Description:
h3. Summary
Enhance the pypio, which is the Python API for PIO.
h3. Goals
The limitations of the current Python support always force developers to have
access to sbt. This enhancement will get rid of the build phase.
h3. Description
A Python engine template requires 3 files:
* Python code to specify for the --main-py-file option
* template.json
{code:json}
{"pio": {"version": { "min": "0.14.0-SNAPSHOT" }}}
{code}
* engine.json
{code:json}
{
"id": "default",
"description": "Default settings",
"engineFactory": "org.apache.predictionio.e2.engine.PythonEngine",
"algorithms": [
{
"name": "default",
"params": {
"name": "BHPApp"
}
}
],
"serving": {
"params": {
"columns": ["prediction"]
}
}
}
{code}
h4. pypio module
Developers can use the pypio module with jupyter notebook and Python code.
First, import the necessary modules.
{code:python}
from pypio import pypio
{code}
Once the module in imported, the first step is to initialize the pypio module.
{code:python}
pypio.init()
{code}
Next, find data from the event store.
{code:python}
event_df = pypio.find('BHPApp')
{code}
And then, save the model.
{code:python}
# model is a PipelineModel, which is produced after a Pipeline’s fit() method
runs
pipeline = Pipeline(...)
model = pipeline.fit(train_df)
pypio.save(model)
{code}
was:
h3. Summary
Enhance the pypio, which is the Python API for PIO.
h3. Goals
The limitations of the current Python support always force developers to have
access to sbt. This enhancement will get rid of the build phase.
h3. Description
A Python engine template requires 3 files:
* Python code to specify for the --main-py-file option
* template.json
{code:json}
{"pio": {"version": { "min": "0.14.0-SNAPSHOT" }}}
{code}
* engine.json
{code:json}
{
"id": "default",
"description": "Default settings",
"engineFactory": "org.apache.predictionio.e2.engine.PythonEngine",
"algorithms": [
{
"name": "default",
"params": {
"name": "BHPApp"
}
}
],
"serving": {
"params": {
"columns": ["prediction"]
}
}
}
{code}
h4. pypio module
Developers can use the pypio module with jupyter notebook and Python code.
First, import the necessary modules.
{code:python}
from pypio import pypio
{code}
Once the module in imported, the first step is to initialize the pypio module.
{code:python}
pypio.init()
{code}
Next, find data from the event store.
{code:python}
event_df = pypio.find('BHPApp')
{code}
And then, save the model.
{code:python}
# model is a PipelineModel, which is produced after a Pipeline’s fit() method
runs
model = ...
pypio.save(model)
{code}
> Enhance PySpark support
> ---
>
> Key: PIO-192
> URL: https://issues.apache.org/jira/browse/PIO-192
> Project: PredictionIO
> Issue Type: Improvement
> Components: Core
>Affects Versions: 0.13.0
>Reporter: Takako Shimamoto
>Assignee: Takako Shimamoto
>Priority: Major
>
> h3. Summary
> Enhance the pypio, which is the Python API for PIO.
> h3. Goals
> The limitations of the current Python support always force developers to have
> access to sbt. This enhancement will get rid of the build phase.
> h3. Description
> A Python engine template requires 3 files:
> * Python code to specify for the --main-py-file option
> * template.json
> {code:json}
> {"pio": {"version": { "min": "0.14.0-SNAPSHOT" }}}
> {code}
> * engine.json
> {code:json}
> {
> "id": "default",
> "description": "Default settings",
> "engineFactory": "org.apache.predictionio.e2.engine.PythonEngine",
> "algorithms": [
> {
> "name": "default",
> "params": {
> "name": "BHPApp"
> }
> }
> ],
> "serving": {
> "params": {
> "columns": ["prediction"]
> }
> }
> }
> {code}
> h4. pypio module
> Developers can use the pypio module with jupyter notebook and Python code.
> First, import the necessary modules.
> {code:python}
> from pypio import pypio
> {code}
> Once the module in imported, the first step is to initialize the pypio module.
> {code:python}
> pypio.init()
> {code}
> Next, find data from the event store.
> {code:python}
> event_df = pypio.find('BHPApp')
> {code}
> And then, save the model.
> {code:python}
> # model is a PipelineModel, which is produced after a Pipeline’s fit() method
> runs
> pipeline = Pipeline(...)
> model = pipeline.fit(train_df)
> pypio.save(model)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PIO-192) Enhance PySpark support
[
https://issues.apache.org/jira/browse/PIO-192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takako Shimamoto updated PIO-192:
-
Description:
h3. Summary
Enhance the pypio, which is the Python API for PIO.
h3. Goals
The limitations of the current Python support always force developers to have
access to sbt. This enhancement will get rid of the build phase.
h3. Description
A Python engine template requires 3 files:
* Python code to specify for the --main-py-file option
* template.json
{code:json}
{"pio": {"version": { "min": "0.14.0-SNAPSHOT" }}}
{code}
* engine.json
{code:json}
{
"id": "default",
"description": "Default settings",
"engineFactory": "org.apache.predictionio.e2.engine.PythonEngine",
"algorithms": [
{
"name": "default",
"params": {
"name": "BHPApp"
}
}
],
"serving": {
"params": {
"columns": ["prediction"]
}
}
}
{code}
h4. pypio module
Developers can use the pypio module with jupyter notebook and Python code.
First, import the necessary modules.
{code:python}
from pypio import pypio
{code}
Once the module in imported, the first step is to initialize the pypio module.
{code:python}
pypio.init()
{code}
Next, find data from the event store.
{code:python}
event_df = pypio.find('BHPApp')
{code}
And then, save the model.
{code:python}
# model is a PipelineModel, which is produced after a Pipeline’s fit() method
runs
model = ...
pypio.save(model)
{code}
was:
h3. Summary
Enhance the pypio, which is the Python API for PIO.
h3. Goals
The limitations of the current Python support always force developers to have
access to sbt. This enhancement will get rid of the build phase.
h3. Description
A Python engine template requires 3 files:
* Python code to specify for the --main-py-file option
* template.json
{code:json}
{"pio": {"version": { "min": "0.14.0-SNAPSHOT" }}}
{code}
* engine.json
{code:json}
{
"id": "default",
"description": "Default settings",
"engineFactory": "org.apache.predictionio.e2.engine.PythonEngine",
"algorithms": [
{
"name": "default",
"params": {
"name": "BHPApp"
}
}
]
}
{code}
h4. pypio module
Developers can use the pypio module with jupyter notebook and Python code.
First, import the necessary modules.
{code:python}
from pypio import pypio
{code}
Once the module in imported, the first step is to initialize the pypio module.
{code:python}
pypio.init()
{code}
Next, find data from the event store.
{code:python}
event_df = pypio.find('BHPApp')
{code}
And then, save the model.
{code:python}
# model is a PipelineModel, which is produced after a Pipeline’s fit() method
runs
model = ...
pypio.save(model)
{code}
> Enhance PySpark support
> ---
>
> Key: PIO-192
> URL: https://issues.apache.org/jira/browse/PIO-192
> Project: PredictionIO
> Issue Type: Improvement
> Components: Core
>Affects Versions: 0.13.0
>Reporter: Takako Shimamoto
>Assignee: Takako Shimamoto
>Priority: Major
>
> h3. Summary
> Enhance the pypio, which is the Python API for PIO.
> h3. Goals
> The limitations of the current Python support always force developers to have
> access to sbt. This enhancement will get rid of the build phase.
> h3. Description
> A Python engine template requires 3 files:
> * Python code to specify for the --main-py-file option
> * template.json
> {code:json}
> {"pio": {"version": { "min": "0.14.0-SNAPSHOT" }}}
> {code}
> * engine.json
> {code:json}
> {
> "id": "default",
> "description": "Default settings",
> "engineFactory": "org.apache.predictionio.e2.engine.PythonEngine",
> "algorithms": [
> {
> "name": "default",
> "params": {
> "name": "BHPApp"
> }
> }
> ],
> "serving": {
> "params": {
> "columns": ["prediction"]
> }
> }
> }
> {code}
> h4. pypio module
> Developers can use the pypio module with jupyter notebook and Python code.
> First, import the necessary modules.
> {code:python}
> from pypio import pypio
> {code}
> Once the module in imported, the first step is to initialize the pypio module.
> {code:python}
> pypio.init()
> {code}
> Next, find data from the event store.
> {code:python}
> event_df = pypio.find('BHPApp')
> {code}
> And then, save the model.
> {code:python}
> # model is a PipelineModel, which is produced after a Pipeline’s fit() method
> runs
> model = ...
> pypio.save(model)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (PIO-189) ES6 integration test fails
[ https://issues.apache.org/jira/browse/PIO-189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto resolved PIO-189. -- Resolution: Fixed > ES6 integration test fails > -- > > Key: PIO-189 > URL: https://issues.apache.org/jira/browse/PIO-189 > Project: PredictionIO > Issue Type: Sub-task > Components: Core >Reporter: Takako Shimamoto >Assignee: Alexander Merritt >Priority: Major > Fix For: 0.14.0 > > > ES6 integration test fails when > PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=ELASTICSEARCH is specified. > https://travis-ci.org/apache/predictionio/builds/445535291 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (PIO-189) ES6 integration test fails
[ https://issues.apache.org/jira/browse/PIO-189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto reassigned PIO-189: Assignee: Alexander Merritt > ES6 integration test fails > -- > > Key: PIO-189 > URL: https://issues.apache.org/jira/browse/PIO-189 > Project: PredictionIO > Issue Type: Sub-task > Components: Core >Reporter: Takako Shimamoto >Assignee: Alexander Merritt >Priority: Major > Fix For: 0.14.0 > > > ES6 integration test fails when > PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=ELASTICSEARCH is specified. > https://travis-ci.org/apache/predictionio/builds/445535291 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-189) ES6 integration test fails
[ https://issues.apache.org/jira/browse/PIO-189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-189: - Fix Version/s: 0.14.0 > ES6 integration test fails > -- > > Key: PIO-189 > URL: https://issues.apache.org/jira/browse/PIO-189 > Project: PredictionIO > Issue Type: Sub-task > Components: Core >Reporter: Takako Shimamoto >Assignee: Alexander Merritt >Priority: Major > Fix For: 0.14.0 > > > ES6 integration test fails when > PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=ELASTICSEARCH is specified. > https://travis-ci.org/apache/predictionio/builds/445535291 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PIO-189) ES6 integration test fails
Takako Shimamoto created PIO-189: Summary: ES6 integration test fails Key: PIO-189 URL: https://issues.apache.org/jira/browse/PIO-189 Project: PredictionIO Issue Type: Sub-task Reporter: Takako Shimamoto ES6 integration test fails when PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=ELASTICSEARCH is specified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PIO-188) Update the build matrix to the latest supported versions
Takako Shimamoto created PIO-188: Summary: Update the build matrix to the latest supported versions Key: PIO-188 URL: https://issues.apache.org/jira/browse/PIO-188 Project: PredictionIO Issue Type: Improvement Components: Build, Documentation Affects Versions: 0.14.0 Reporter: Takako Shimamoto At first glance, it looks like .travis.yml has covered every version, but there aren't tests for Elasticsearch 6.x or HBase 1.2 currently. Whereas that there are some unnecessary patterns in the build matrix. I put together the latest supported versions. Note that there is no change in the default version. h3. Core versions - Scala -- 2.11.12(default) - Spark -- 2.0.2 -- 2.1.3(default) -- 2.2.2 -- 2.3.2 - Hadoop -- 2.6.5 -- 2.7.3(default) h3. Storage versions - Elasticsearch -- 1.7.3(deprecated) -- 5.6.9(default) -- 6.4.2 - HBase -- 1.2.6(default) - HDFS -- same as above Hadoop I will open a PR. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-185) Non-tracked Link in Apache Project page
[ https://issues.apache.org/jira/browse/PIO-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653293#comment-16653293 ] Takako Shimamoto commented on PIO-185: -- [~dszeto] I got it. thank you. We have to fix build.sbt and contribute-documentation.html.md as well. https://github.com/apache/predictionio/blob/develop/build.sbt#L278-L282 https://github.com/apache/predictionio/blob/develop/docs/manual/source/community/contribute-documentation.html.md > Non-tracked Link in Apache Project page > --- > > Key: PIO-185 > URL: https://issues.apache.org/jira/browse/PIO-185 > Project: PredictionIO > Issue Type: Bug > Components: Documentation >Reporter: Wei Chen >Priority: Minor > > On page: [https://projects.apache.org/project.html?predictionio] > at line 15: > -> *Git repository:* > [https://git-wip-us.apache.org/repos/asf/predictionio.git] > There is no git repo with the link. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-185) Non-tracked Link in Apache Project page
[ https://issues.apache.org/jira/browse/PIO-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652872#comment-16652872 ] Takako Shimamoto commented on PIO-185: -- Oh, nice catch! https://gitbox.apache.org/repos/asf/predictionio.git is correct. [~dszeto] we will ask apache infra to fix it, won't we? > Non-tracked Link in Apache Project page > --- > > Key: PIO-185 > URL: https://issues.apache.org/jira/browse/PIO-185 > Project: PredictionIO > Issue Type: Bug > Components: Documentation >Reporter: Wei Chen >Priority: Minor > > On page: [https://projects.apache.org/project.html?predictionio] > at line 15: > -> *Git repository:* > [https://git-wip-us.apache.org/repos/asf/predictionio.git] > There is no git repo with the link. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-181) Drop Elasticsearch 1.x support
[ https://issues.apache.org/jira/browse/PIO-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-181: - Issue Type: Task (was: Improvement) > Drop Elasticsearch 1.x support > -- > > Key: PIO-181 > URL: https://issues.apache.org/jira/browse/PIO-181 > Project: PredictionIO > Issue Type: Task > Components: Build, Core, Documentation >Affects Versions: 0.14.0 >Reporter: Takako Shimamoto >Priority: Major > > Support for Elasticsearch 1.x will be removed after 0.14.0 release. > See PIO-175 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (PIO-175) Deprecation of Elasticsearch 1.x support
[ https://issues.apache.org/jira/browse/PIO-175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto reassigned PIO-175: Assignee: Takako Shimamoto > Deprecation of Elasticsearch 1.x support > > > Key: PIO-175 > URL: https://issues.apache.org/jira/browse/PIO-175 > Project: PredictionIO > Issue Type: Task > Components: Core >Affects Versions: 0.13.0 >Reporter: Naoki Takezoe >Assignee: Takako Shimamoto >Priority: Major > > Mark Elasticsearch 1.x support as deprecated and announce the future deletion > in 0.14.0. > It will be deleted actually in 0.15.0 from our codebase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-175) Deprecation of Elasticsearch 1.x support
[ https://issues.apache.org/jira/browse/PIO-175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637736#comment-16637736 ] Takako Shimamoto commented on PIO-175: -- I think it's ok to just modify the document. > Deprecation of Elasticsearch 1.x support > > > Key: PIO-175 > URL: https://issues.apache.org/jira/browse/PIO-175 > Project: PredictionIO > Issue Type: Task > Components: Core >Affects Versions: 0.13.0 >Reporter: Naoki Takezoe >Priority: Major > > Mark Elasticsearch 1.x support as deprecated and announce the future deletion > in 0.14.0. > It will be deleted actually in 0.15.0 from our codebase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-175) Deprecation of Elasticsearch 1.x support
[ https://issues.apache.org/jira/browse/PIO-175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637716#comment-16637716 ] Takako Shimamoto commented on PIO-175: -- The task is only to modify the document, right? > Deprecation of Elasticsearch 1.x support > > > Key: PIO-175 > URL: https://issues.apache.org/jira/browse/PIO-175 > Project: PredictionIO > Issue Type: Task > Components: Core >Affects Versions: 0.13.0 >Reporter: Naoki Takezoe >Priority: Major > > Mark Elasticsearch 1.x support as deprecated and announce the future deletion > in 0.14.0. > It will be deleted actually in 0.15.0 from our codebase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-160) Array out of bound exception in JDBCUtils when --env is not supplied to CreateWorkflow
[ https://issues.apache.org/jira/browse/PIO-160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-160: - Fix Version/s: 0.13.0 > Array out of bound exception in JDBCUtils when --env is not supplied to > CreateWorkflow > -- > > Key: PIO-160 > URL: https://issues.apache.org/jira/browse/PIO-160 > Project: PredictionIO > Issue Type: Bug > Components: Core >Affects Versions: 0.12.1 >Reporter: Donald Szeto >Assignee: Donald Szeto >Priority: Major > Fix For: 0.13.0 > > > JDBCUtils.stringToMap() does not check for empty input and will throw an > exception when an engine instance is trained without supplying any --env > argument. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (PIO-98) Issues in building Complementary Purchase Engine Template
[ https://issues.apache.org/jira/browse/PIO-98?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto closed PIO-98. --- Resolution: Won't Do > Issues in building Complementary Purchase Engine Template > - > > Key: PIO-98 > URL: https://issues.apache.org/jira/browse/PIO-98 > Project: PredictionIO > Issue Type: Bug >Reporter: salil chawla >Assignee: Donald Szeto >Priority: Major > > http://predictionio.incubator.apache.org/templates/complementarypurchase/quickstart/ > doesn't get built with 0.11.0-incubating release of predictionIO. > Looks like it is pointing to the packages and classes relevant to the older > version of PredictionIO > for example, > template-scala-parallel-complementarypurchase/src/main/scala/Algorithm.scala > is importing the P2LAlgorithm class with qualified name as > io.prediction.controller.P2LAlgorithm > but ideally it should have been > org.apache.predictionio.controller.P2LAlgorithm -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIO-98) Issues in building Complementary Purchase Engine Template
[ https://issues.apache.org/jira/browse/PIO-98?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610012#comment-16610012 ] Takako Shimamoto commented on PIO-98: - [~dszeto] thanks a lot! > Issues in building Complementary Purchase Engine Template > - > > Key: PIO-98 > URL: https://issues.apache.org/jira/browse/PIO-98 > Project: PredictionIO > Issue Type: Bug >Reporter: salil chawla >Assignee: Donald Szeto >Priority: Major > > http://predictionio.incubator.apache.org/templates/complementarypurchase/quickstart/ > doesn't get built with 0.11.0-incubating release of predictionIO. > Looks like it is pointing to the packages and classes relevant to the older > version of PredictionIO > for example, > template-scala-parallel-complementarypurchase/src/main/scala/Algorithm.scala > is importing the P2LAlgorithm class with qualified name as > io.prediction.controller.P2LAlgorithm > but ideally it should have been > org.apache.predictionio.controller.P2LAlgorithm -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-156) Stale release on download page
[ https://issues.apache.org/jira/browse/PIO-156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-156: - Fix Version/s: 0.13.0 > Stale release on download page > -- > > Key: PIO-156 > URL: https://issues.apache.org/jira/browse/PIO-156 > Project: PredictionIO > Issue Type: Bug > Components: Documentation >Affects Versions: Live Documentation > Environment: > http://predictionio.apache.org/install/install-sourcecode/ >Reporter: Sebb >Assignee: Donald Szeto >Priority: Major > Fix For: 0.13.0, Live Documentation > > > The release 0.12.0-incubating is no longer current; please update the page. > > Also the page is difficult to use, as it does not have individual links for > the sigs and hashes. > I would expect to find such links alongside the link to the source artifact. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-146) Change TM to (R) on text marks
[ https://issues.apache.org/jira/browse/PIO-146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-146: - Target Version/s: 0.12.1 (was: 0.13.0) > Change TM to (R) on text marks > -- > > Key: PIO-146 > URL: https://issues.apache.org/jira/browse/PIO-146 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Affects Versions: 0.12.0-incubating >Reporter: Donald Szeto >Assignee: Donald Szeto >Priority: Major > Fix For: 0.12.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-125) Spark 2.2 support
[ https://issues.apache.org/jira/browse/PIO-125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-125: - Target Version/s: 0.12.1 (was: 0.13.0) > Spark 2.2 support > - > > Key: PIO-125 > URL: https://issues.apache.org/jira/browse/PIO-125 > Project: PredictionIO > Issue Type: Improvement > Components: Core >Reporter: Shinsuke Sugaya >Assignee: Shinsuke Sugaya >Priority: Major > Fix For: 0.12.1 > > > Add Spark 2.2 to scalaSparkDepsVersion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-98) Issues in building Complementary Purchase Engine Template
[ https://issues.apache.org/jira/browse/PIO-98?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-98: Target Version/s: (was: 0.13.0) > Issues in building Complementary Purchase Engine Template > - > > Key: PIO-98 > URL: https://issues.apache.org/jira/browse/PIO-98 > Project: PredictionIO > Issue Type: Bug >Reporter: salil chawla >Assignee: Donald Szeto >Priority: Major > > http://predictionio.incubator.apache.org/templates/complementarypurchase/quickstart/ > doesn't get built with 0.11.0-incubating release of predictionIO. > Looks like it is pointing to the packages and classes relevant to the older > version of PredictionIO > for example, > template-scala-parallel-complementarypurchase/src/main/scala/Algorithm.scala > is importing the P2LAlgorithm class with qualified name as > io.prediction.controller.P2LAlgorithm > but ideally it should have been > org.apache.predictionio.controller.P2LAlgorithm -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-89) Add page to documentation that lists out deprecations
[ https://issues.apache.org/jira/browse/PIO-89?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-89: Target Version/s: (was: 0.13.0) > Add page to documentation that lists out deprecations > - > > Key: PIO-89 > URL: https://issues.apache.org/jira/browse/PIO-89 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Reporter: Sara Asher >Priority: Major > Labels: Documentation > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-75) Improve committer documentation
[ https://issues.apache.org/jira/browse/PIO-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-75: Target Version/s: (was: 0.13.0) > Improve committer documentation > --- > > Key: PIO-75 > URL: https://issues.apache.org/jira/browse/PIO-75 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Reporter: Sara Asher >Assignee: Chan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-79) Create architecture guide for new developers
[ https://issues.apache.org/jira/browse/PIO-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-79: Target Version/s: (was: 0.13.0) > Create architecture guide for new developers > > > Key: PIO-79 > URL: https://issues.apache.org/jira/browse/PIO-79 > Project: PredictionIO > Issue Type: Improvement > Components: Documentation >Reporter: Sara Asher >Assignee: Donald Szeto >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-71) Support Spark ML natively as an alternative to DASE
[ https://issues.apache.org/jira/browse/PIO-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-71: Target Version/s: (was: 0.13.0) > Support Spark ML natively as an alternative to DASE > --- > > Key: PIO-71 > URL: https://issues.apache.org/jira/browse/PIO-71 > Project: PredictionIO > Issue Type: New Feature > Components: Core >Affects Versions: 0.11.0-incubating >Reporter: Donald Szeto >Assignee: Chan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-71) Support Spark ML natively as an alternative to DASE
[ https://issues.apache.org/jira/browse/PIO-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-71: Fix Version/s: (was: 0.13.0) > Support Spark ML natively as an alternative to DASE > --- > > Key: PIO-71 > URL: https://issues.apache.org/jira/browse/PIO-71 > Project: PredictionIO > Issue Type: New Feature > Components: Core >Affects Versions: 0.11.0-incubating >Reporter: Donald Szeto >Assignee: Chan >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PIO-80) Docs on Custom Backend Support are Wrong
[ https://issues.apache.org/jira/browse/PIO-80?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takako Shimamoto updated PIO-80: Target Version/s: (was: 0.13.0) > Docs on Custom Backend Support are Wrong > > > Key: PIO-80 > URL: https://issues.apache.org/jira/browse/PIO-80 > Project: PredictionIO > Issue Type: Bug > Components: Documentation >Reporter: James Ward >Assignee: Sara Asher >Priority: Minor > Labels: newbie > > The docs for Using Another Data Store - > https://predictionio.incubator.apache.org/system/anotherdatastore/ - indicate > that the custom fat jar should go into a "plugins" dir. But the contents of > that dir don't actually go into the CLASSPATH. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PIO-158) More officially deprecate support for Scala 2.10 and Spark 1.x
Takako Shimamoto created PIO-158: Summary: More officially deprecate support for Scala 2.10 and Spark 1.x Key: PIO-158 URL: https://issues.apache.org/jira/browse/PIO-158 Project: PredictionIO Issue Type: Task Reporter: Takako Shimamoto Assignee: Takako Shimamoto https://lists.apache.org/thread.html/a5f1e7d0ae61f69ff9663ea35de42a473aab132636c280fb070e256b@%3Cdev.predictionio.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (PIO-137) Connection pool is not yet initialized.(name:'default)
[
https://issues.apache.org/jira/browse/PIO-137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takako Shimamoto reassigned PIO-137:
Assignee: Takako Shimamoto
> Connection pool is not yet initialized.(name:'default)
> --
>
> Key: PIO-137
> URL: https://issues.apache.org/jira/browse/PIO-137
> Project: PredictionIO
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.11.0-incubating
>Reporter: Mukesh Gupta
>Assignee: Takako Shimamoto
>
> While running http://github.com/actionml/db-cleaner template, I am facing the
> following issue:
> {code:java}
> [WARN] [TaskSetManager] Lost task 0.1 in stage 23.0 (TID 75,
> ip-172-31-105-42.ap-southeast-1.compute.internal):
> java.lang.IllegalStateException: Connection pool is not yet
> initialized.(name:'default)
> at
> scalikejdbc.ConnectionPool$$anonfun$get$1.apply(ConnectionPool.scala:76)
> at
> scalikejdbc.ConnectionPool$$anonfun$get$1.apply(ConnectionPool.scala:74)
> at scala.collection.MapLike$class.getOrElse(MapLike.scala:128)
> at scala.collection.AbstractMap.getOrElse(Map.scala:58)
> at scalikejdbc.ConnectionPool$.get(ConnectionPool.scala:74)
> at scalikejdbc.ConnectionPool$.apply(ConnectionPool.scala:65)
> at scalikejdbc.DB$.connectionPool(DB.scala:152)
> at scalikejdbc.DB$.localTx(DB.scala:262)
> at
> org.apache.predictionio.data.storage.jdbc.JDBCPEvents$$anonfun$delete$1$$anonfun$apply$11.apply(JDBCPEvents.scala:182)
> at
> org.apache.predictionio.data.storage.jdbc.JDBCPEvents$$anonfun$delete$1$$anonfun$apply$11.apply(JDBCPEvents.scala:181)
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> at
> org.apache.predictionio.data.storage.jdbc.JDBCPEvents$$anonfun$delete$1.apply(JDBCPEvents.scala:181)
> at
> org.apache.predictionio.data.storage.jdbc.JDBCPEvents$$anonfun$delete$1.apply(JDBCPEvents.scala:179)
> at
> org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
> at
> org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
> at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
> at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Following are my PIO settings
> {code:java}
> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/elasticsearch
> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
> PIO_STORAGE_SOURCES_PGSQL_INDEX=enabled
> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
> PIO_HOME=/PredictionIO
> PIO_STORAGE_SOURCES_PGSQL_USERNAME=dbuser
> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
> PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://hostname:5432/predictionio
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=es-hostname
> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=es_cluster
> PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
> PIO_STORAGE_SOURCES_PGSQL_PASSWORD=xx
> {code}
> I have tried upgrading the scalike version to 3.1.0 (as hinted by this:
> https://github.com/scalikejdbc/scalikejdbc/commit/b7713a8dbfb72b05f43743a5c281b5d7f7bac824)
> but the issue still remains.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PIO-137) Connection pool is not yet initialized.(name:'default)
[
https://issues.apache.org/jira/browse/PIO-137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249353#comment-16249353
]
Takako Shimamoto commented on PIO-137:
--
The connection pool is initialized at a Spark driver, and then JDBCPEvents
tries to use it in a Spark worker to delete records in the RDD. The correct
solution is to initialize the connection pool (or create a connection object)
at a worker.
I’ll handle this issue.
> Connection pool is not yet initialized.(name:'default)
> --
>
> Key: PIO-137
> URL: https://issues.apache.org/jira/browse/PIO-137
> Project: PredictionIO
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.11.0-incubating
>Reporter: Mukesh Gupta
>
> While running http://github.com/actionml/db-cleaner template, I am facing the
> following issue:
> {code:java}
> [WARN] [TaskSetManager] Lost task 0.1 in stage 23.0 (TID 75,
> ip-172-31-105-42.ap-southeast-1.compute.internal):
> java.lang.IllegalStateException: Connection pool is not yet
> initialized.(name:'default)
> at
> scalikejdbc.ConnectionPool$$anonfun$get$1.apply(ConnectionPool.scala:76)
> at
> scalikejdbc.ConnectionPool$$anonfun$get$1.apply(ConnectionPool.scala:74)
> at scala.collection.MapLike$class.getOrElse(MapLike.scala:128)
> at scala.collection.AbstractMap.getOrElse(Map.scala:58)
> at scalikejdbc.ConnectionPool$.get(ConnectionPool.scala:74)
> at scalikejdbc.ConnectionPool$.apply(ConnectionPool.scala:65)
> at scalikejdbc.DB$.connectionPool(DB.scala:152)
> at scalikejdbc.DB$.localTx(DB.scala:262)
> at
> org.apache.predictionio.data.storage.jdbc.JDBCPEvents$$anonfun$delete$1$$anonfun$apply$11.apply(JDBCPEvents.scala:182)
> at
> org.apache.predictionio.data.storage.jdbc.JDBCPEvents$$anonfun$delete$1$$anonfun$apply$11.apply(JDBCPEvents.scala:181)
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> at
> org.apache.predictionio.data.storage.jdbc.JDBCPEvents$$anonfun$delete$1.apply(JDBCPEvents.scala:181)
> at
> org.apache.predictionio.data.storage.jdbc.JDBCPEvents$$anonfun$delete$1.apply(JDBCPEvents.scala:179)
> at
> org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
> at
> org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
> at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
> at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Following are my PIO settings
> {code:java}
> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/elasticsearch
> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
> PIO_STORAGE_SOURCES_PGSQL_INDEX=enabled
> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
> PIO_HOME=/PredictionIO
> PIO_STORAGE_SOURCES_PGSQL_USERNAME=dbuser
> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
> PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://hostname:5432/predictionio
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=es-hostname
> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=es_cluster
> PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
> PIO_STORAGE_SOURCES_PGSQL_PASSWORD=xx
> {code}
> I have tried upgrading the scalike version to 3.1.0 (as hinted by this:
> https://github.com/scalikejdbc/scalikejdbc/commit/b7713a8dbfb72b05f43743a5c281b5d7f7bac824)
> but the issue still remains.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
