GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio-template-recommender/pull/12
Python 3 support for data/*.py
Adding () to print statement, it works on Python 3.5.2.
You can merge this pull request into a Git repository by running
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/330
I works with @haginot on this issue.
I'm afraid that our elasticsearch support does not work yet...
So, we will check it when deploying it to our environment.
---
If
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
We use the following template.
https://github.com/marevol/incubator-predictionio-template-recommender/tree/0.11.0
(Commented out "data.ratings.take(1)", because i
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
Thank you for your comment.
We will check if it keeps the code for ES 1.x.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
To use the existing code for elasticsearch 1.x, please use
PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH1 and
PIO_STORAGE_SOURCES_ELASTICSEARCH1_* settings.
>
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/336#discussion_r96766370
--- Diff:
data/src/main/scala/org/apache/predictionio/data/storage/elasticsearch/ESAccessKeys.scala
---
@@ -15,44 +15,45
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
@dszeto Did you work on this issue? I think I'm available this week. So,
I'll do that if you do not work yet.
---
If your project is set up for it, you can reply to this
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
@dszeto Could you replace ELASTICSEARCH1 with ELASTICSEARCH5?
https://github.com/apache/incubator-predictionio/blob/feature/es5/bin/pio-start-all#L37,L38
feature/es5
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/349
[PIO-54] Avoid access key started with -
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio retry
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
To solve this issue, how about using plugin directory?
In our branch(https://github.com/jpioug/incubator-predictionio), ES1/ES5
codes in data were extracted to data-elasticsearch
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
Please see our snapshot build:
http://fess.codelibs.org/snapshot/apache-predictionio-0.11.0-v1-SNAPSHOT.zip
Storage implementations, such as hbase, were moved to plugins/data
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
Although I'm not sure about old plugin features, in my fix, I put plugin
jar files with --jars on spark-submit:
https://github.com/jpioug/incubator-predictionio/c
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
compute-classpath.sh deals with plugins directory, but existing code seems
not to put jars in spark-submit.
If plugin feature looks good, I'll create PR for featur
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
+1
As @pferrel 's comment, I think it may be better to use a directory(ex.
lib/spark) other than lib if we want to add arbitrary JARs to spark-submit.
If we put them int
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/352
[PIO-49] Add support for Elasticsearch 5 and move storage implementation
You can merge this pull request into a Git repository by running:
$ git pull https://github.com
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
Please see https://github.com/apache/incubator-predictionio/pull/352
- Move storage implementation to storage directory
- Put storage assembly JARs into lib/spark in
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
Right. To use ES5, please replace pio-data-elasticsearch1-assembly-\*.jar
with pio-data-elasticsearch-assembly-\*.jar in lib/spark directory.
To change JARs location(ex. lib
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
I'll fix it today...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
Fixed!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
Correct.
I'm working on integration tests for ES5. To use ES5 as default, should I
create a new PR?
For UR, @haginot and co-workers work on ES5 support. They
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/355
Please see https://github.com/apache/incubator-predictionio/pull/352
We are working on Elasticsearch 5 support.
This fix removes elasticsearch from core.
---
If your project
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
For integration tests of ES5, 3 tests are failed on
[move-storages-es5]...(https://github.com/marevol/incubator-predictionio/tree/move-storages-es5)
I'll check them
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/355
My concern is core depends on Elasticsearch.
Since Elasticsearch 1.x is EOLed by elastic, it will be removed from PIO in
the future release
and PIO will not contain
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
Almost all failures are fixed.
For remaining problem, integration tests use PDT time, but elasticsearch
uses UTC time.
Therefore, the following assertion is failed in
tests
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
Fixed problems for ES5 support.
Integration tests will be passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/347
@dszeto Could you check and merge this pull request?
He is my co-worker and Scala expert.
---
If your project is set up for it, you can reply to this email and have your
reply
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/358
Update event serialization for ES5
- Modify Event index mapping (change properties field to keyword type)
- Use UTC time format for eventTime/creatationTime
You can merge this
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/359
[PIO-57] Add SBT Native Packager
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio
sbt-native
Github user marevol commented on the pull request:
https://github.com/apache/incubator-predictionio/commit/d78b3cbe912cf57e2f0278e407a0d6432bd12849#commitcomment-21280327
@pferrel I think PIO discards the stacktrace and the root cause is unclear
in the log message.
Could you
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/360
Print stacktrace to pio.log
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio
print_stacktrace
Github user marevol commented on the pull request:
https://github.com/apache/incubator-predictionio/commit/947db5e1b6e1ff629ef5bcb4412c92309e2a1354#commitcomment-21281853
@dszeto To fix travis error, the following change will be needed:
https://github.com/travis-ci/travis-ci
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/195
@jorelcb To support ES 5.x, we are working on PIO-49.
https://github.com/apache/incubator-predictionio/pull/195 will be closed.
For query tuning, I'll check it after
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/359#discussion_r106052297
--- Diff: .travis.yml ---
@@ -47,7 +47,7 @@ env:
- BUILD_TYPE=Unit METADATA_REP=PGSQL EVENTDATA_REP=PGSQL
MODELDATA_REP=PGSQL
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/359
> I am wondering about the reason behind moving bin and conf to assembly
SBT native packager is contained in assembly directory, and putting them to
src/universal
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/359
Using make-distribution.sh, I'll try to modify it to keep the directory
structure.
---
If your project is set up for it, you can reply to this email and have your
reply appe
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/359
bin and conf are back, and I also added --with-rpm/deb option to
make-distribution.sh.
---
If your project is set up for it, you can reply to this email and have your
reply appear
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio-template-attribute-based-classifier/pull/9
Python3 support
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/marevol/incubator
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/359
Synced up with develop branch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/359
My environment is rpmbuild 4.12.0.1 in Ubuntu 16.04.
rpmbuild 5 seems to require rpmGroup.
Adding rpmGroup, I checked it on rpmbuild 4/5.
---
If your project is set up for it
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/369
Add parameters to specify the number of shards/replicas for elasticsearch
To set the number of shards/replicas, add the following values:
- PIO_STORAGE_SOURCES_ELASTICSEARCH_
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/370
Add EVENTDATA_REFRESH for elasticsearch
To improve indexing performance for event data, it's better to be able to
set refresh parameter.
In PIO, refresh is true by de
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/371
[PIO-61] Add S3 Model Data Repository
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio storage_s3
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r112782773
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/373
Improve insert performance for ES storage
- Change properties and n to not indexed field
- Set refresh to false (_version does not need to be refreshed)
and modified pio
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/373
Thank you for merging it.
Our production environment runs on ES 5.1, and my local dev is on ES 5.3.
---
If your project is set up for it, you can reply to this email and have
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r113334667
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r113576948
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/374
ES storage improvement/refactoring
- Use the same ID generation for ES in event data
- Reuse RestClient instance in ESLEvents
- Replace 0 with false in mapping
- Set
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r113695041
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio-template-recommender/pull/16
Add checkpointDir and checkpointInterval
Checkpointing helps with recovery and StackOverflowException caused by long
lineage.
I think it's better t
Github user marevol closed the pull request at:
https://github.com/apache/incubator-predictionio-template-recommender/pull/16
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/371
Added LocalStack for S3 testing.
LocalStack/moto seems to have a problem about ETag handling...
I added DISABLE_CHUNKED_ENCODING option to avoid it in S3 testing.
---
If
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/371
I would like to merge S3 support this week.
For travis testing, it passed in
https://travis-ci.org/jpioug/incubator-predictionio/builds/235569236 though
apache-travis was failed
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/386
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/398
[PIO-83][PIO-84] Move to scala 2.11.8, spark 2.1.1, elasticsearch 5.4.1 as
default env
You can merge this pull request into a Git repository by running:
$ git pull https
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/404
[PIO-100] Remove SLF4J warning message on pio command
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/406
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/427
[PIO-116] PySpark Support
This PR provides PySpark support with minimum PIO changes.
1. Support pyspark on pio-shell
2. Add python files to use pyspark
3. Add --main
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/427
Thank you for checking it. Replaced with if-else.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/428
[PIO-117] Cannot delete event data on ESLEvents
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio-sdk-python/pull/22
[WIP] Add travis test and Refactoring
I'll merge this PR after
[PIO-117](https://issues.apache.org/jira/browse/PIO-117).
You can merge this pull request into
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/430
[PIO-119] Bump up Elasticsearch to 5.5.2
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio es552
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/429
[PIO-118] ClassCastException from NullWritable to Text in ESEventsUtil
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio-sdk-python/pull/22
Travis is not available in this repository...
I'll ask Infra to do that.
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio-sdk-python/pull/22
Run Travis.
---
GitHub user marevol reopened a pull request:
https://github.com/apache/incubator-predictionio-sdk-python/pull/22
Add travis test and Refactoring
I'll merge this PR after
[PIO-117](https://issues.apache.org/jira/browse/PIO-117).
You can merge this pull request into a Git repos
Github user marevol closed the pull request at:
https://github.com/apache/incubator-predictionio-sdk-python/pull/22
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/427
Added __init__.py. It will work on Python 2.7.
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/427
Thanks! I'll merge this PR tomorrow.
---
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio-sdk-python/pull/23#discussion_r138535329
--- Diff: setup.py ---
@@ -3,8 +3,8 @@
except ImportError:
from distutils.core import setup
-__author__
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio-sdk-python/pull/23
Update project info
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio-sdk-python
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/436
[PIO-125] Add Spark 2.2 support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio spark22
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
For test failures on Travis, I think it's Travis problem. [My
Travis](https://travis-ci.org/jpioug/incubator-predictionio/builds/282978334)
result is passed.
Spark 2.2
Github user marevol closed the pull request at:
https://github.com/apache/incubator-predictionio/pull/436
---
GitHub user marevol reopened a pull request:
https://github.com/apache/incubator-predictionio/pull/436
[PIO-125] Add Spark 2.2 support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio spark22
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
Re-run Travis.
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
Oops, my travis result missed some results...
I'll fix this PR. This problem might come from HBase.
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
Re-run Travis.
---
Github user marevol closed the pull request at:
https://github.com/apache/incubator-predictionio/pull/436
---
GitHub user marevol reopened a pull request:
https://github.com/apache/incubator-predictionio/pull/436
[PIO-125] Add Spark 2.2 support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio spark22
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
Spark 2.2 has guava dependency problem and
[SPARK-21549](https://issues.apache.org/jira/browse/SPARK-21549).
I added a workaround for SPARK-21549.
---
Github user marevol closed the pull request at:
https://github.com/apache/incubator-predictionio/pull/436
---
GitHub user marevol reopened a pull request:
https://github.com/apache/incubator-predictionio/pull/436
[PIO-125] Add Spark 2.2 support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio spark22
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
Re-run Travis.
I think it's better to reduce test matrix...
---
84 matches
Mail list logo
