Re: installing environment (stops when compiling "compiler-interface" for Scala)

2017-10-18 Thread Donald Szeto
> 2017-10-18 19:21 GMT+02:00 Donald Szeto : > >> Looks like an out-of-memory issue here. How much memory does the build >> environment has? >> > > A virtual server with 1GB in DigitalOcean . A system with 2GB of RAM would > be enough? > > >

Re: installing environment (stops when compiling "compiler-interface" for Scala)

2017-10-18 Thread Donald Szeto
Scala. > I have KEYS file and source. Java version is "1.8.0_151". > > > luciano@localhost:~/predic$ gpg --import KEYS > gpg: key D3541808: "Suneel Marthi (CODE SIGNING KEY) " > not changed > gpg: key 8BF4ABEB: "Donald Szeto (CODE SIGNING KEY) &quo

Re: [ERROR] [TaskSetManager] Task 2.0 in stage 10.0 had a not serializable result

2017-10-18 Thread Donald Szeto
Chiming in a bit. Looking at the serialization error, it looks like we are just one little step away from getting this to work. Noelia, what does your synthesized data look like? All data that is processed by Spark needs to be serializable. At some point, a non-serializable vector object showing i

Re: Recommendation template

2017-10-18 Thread Donald Szeto
Hey Noelia, Since you are using PIO 0.11 you may want to use the previous template version that was targeted for PIO 0.11. The latest version is targeted to 0.12’s default stack, which is Scala 2.11. The only difference between these two template versions is targeting different PIO versions. Ther

Call for quote for the upcoming PredictionIO graduation announcement

2017-10-17 Thread Donald Szeto
Dear PredictionIO community, With your support and contribution over the past year, the PredictionIO project is now very close to graduating to become a top-level project. The Apache Incubator PMC has voted positively on our proposed graduation, and it will be included in the ASF Board Meeting on

Re: Adding Spark Context (sc) to predict method?

2017-10-10 Thread Donald Szeto
t Ferrel, did your team do something similar, use ids to go get >> aggregate properties in the predict method? I prototyped with the universal >> Recommendation template a couple months back and remember the predict >> endpoint only required ids. >> >> Thanks for the supp

Re: Adding Spark Context (sc) to predict method?

2017-10-05 Thread Donald Szeto
Hi Shane, If you are not looking to do aggregative on Spark when you retrieve additional information from event store, you probably should look at using LEventStore that does not go through Spark. Depending on your use case, the roundtrip time of involving Spark in your predict method might not be

Re: Missing accessKey error, but I am providing the access key

2017-10-03 Thread Donald Szeto
Hi Noelia, The URL should be http://localhost:7070/events.json?accessKey=myaccesskey instead. Regards, Donald On Tue, Oct 3, 2017 at 6:10 AM, Noelia Osés Fernández wrote: > Hi all, > > I'm new with Prediction IO and in order to understand how it works I'm > customizing the Universal Recommende

Re: Extracting Strings from JString with PEventStore.aggregateProperties

2017-10-03 Thread Donald Szeto
Hi Shane, Thanks for your finding! Perhaps we should improve the documentation or even privatize the `fields` member. The use of `get` is noted here: https://github.com/apache/incubator-predictionio/blob/develop/data/src/main/scala/org/apache/predictionio/data/storage/PropertyMap.scala#L29 Regar

Re: PredictionIO NewRelic Monitoring

2017-09-28 Thread Donald Szeto
Hey, I think you are probably looking for a passthrough argument like this. pio deploy ... -- --conf "spark.driver.extraJavaOptions=-javaagent:..." Please take a look at http://spark.apache.org/docs/latest/configuration.html for more details. Regards, Donald On Thu, Sep 28, 2017 at 11:31 AM, R

Re: [ANNOUNCE] Apache PredictionIO 0.12.0-incubating Release

2017-09-27 Thread Donald Szeto
Congrats to everyone who is part of this, and Chan for his first Apache release! On Wed, Sep 27, 2017 at 2:40 PM Chan Lee wrote: > The Apache PredictionIO team would like to announce the release of Apache > PredictionIO 0.12.0-incubating. > > Release notes are here: > > https://github.com/apache

Re: Text Classification Engine Tutorial

2017-09-27 Thread Donald Szeto
kawakibi wrote: > no.. i didnt make any changes ... > > On Thu, Sep 14, 2017 at 10:49 PM, Donald Szeto wrote: > >> Did you make any change to the template or using a stock one? Looks like >> a Lucene incompatibility. >> >> On Thu, Sep 14, 2017 at 1:40

Re: Unable to connect to all storage backends successfully

2017-09-27 Thread Donald Szeto
9200 > > # > > # For more information, consult the network module documentation. > > # > > # ----- Discovery > > -- > > # > > # Pass an initial list of hosts to perform discovery when new node is

Re: Vagrant installation of PredictionIO not working

2017-09-21 Thread Donald Szeto
Hi Noelia, Sorry that you ran into the orphaned Vagrant page. We haven’t had a chance to update Vagrant yet, so we removed all links going to that page. If you recall how you arrive at that page, please let us know. Meanwhile, please install from source code, or try one of the community-built Doc

Re: Unable to connect to all storage backends successfully

2017-09-20 Thread Donald Szeto
Hey Jim, Did you build PIO 0.12 with ES 1.4 support? ES 1.x is being deprecated in 0.12 so the default build will use ES 5.x support. See the upcoming release notes: https://github.com/apache/incubator-predictionio/blob/release/0.12.0/RELEASE.md#behavior-changes Regards, Donald On Wed, Sep 20,

Re: Text Classification Engine Tutorial

2017-09-14 Thread Donald Szeto
kSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.dep

Re: Text Classification Engine Tutorial

2017-09-12 Thread Donald Szeto
Hi, What version of Elasticsearch are you running? Do you see more information from pio.log at where you ran "pio train"? Regards, Donald On Thu, Sep 7, 2017 at 1:54 AM tajrij kawakibi wrote: > Hello, > > i just finish this 2 tutorial successfully, > > > https://predictionio.incubator.apache.o

Re: [ERROR] [LocalFSModels] Requested array size exceeds VM limit

2017-09-05 Thread Donald Szeto
Hi Abhimanyu, Have you try giving the JVM a larger heap memory, e.g. pio deploy -- --driver-memory 4g Regards, Donald On Tue, Aug 29, 2017 at 9:06 AM, Abhimanyu Nagrath < abhimanyunagr...@gmail.com> wrote: > Hi , > I am new to predictionIo I am using template https://github.com/ > alexice/templ

New PMC member and committer: Mars Hall

2017-07-28 Thread Donald Szeto
Hi all, The Project Management Committee (PMC) for Apache PredictionIO (incubating) has asked Mars Hall to become a PMC member and committer, and we are pleased to announce that he has accepted. Mars has been working on PredictionIO since 0.10 and has suggested and made changes to the core so tha

Re: Regarding Examples under examples/experimental

2017-07-14 Thread Donald Szeto
+1. If some of them are found useful, they should be moved to examples/ and have documentation about it. On Thu, Jul 13, 2017 at 6:55 PM takako shimamoto wrote: > Hi, > > Examples under examples/experimental have been neglected for a long time. > Probably no one won't make efforts for maintenanc

Re: Eventserver API in an Engine?

2017-07-12 Thread Donald Szeto
Many good discussions. Let me provide my input on these issues. Multiple installations of PredictionIO should use different database names. An analogy would be Wordpress installations that expect its own metadata database. I understand the downside to this is that some users only have access to on

Re: Kafka support as Event Store

2017-07-05 Thread Donald Szeto
Hi Thomas, Supporting Kafka is definitely interesting and desirable. Are you looking to sinking your Kafka messages to event store for batch processing, or stream processing directly from Kafka? The latter would require more work because Apache PIO does not yet support streaming properly. Folks f

Re: Eventserver API in an Engine?

2017-06-28 Thread Donald Szeto
Hey Mars, Thanks for the suggestion and I agree with your point on the metadata part. Essentially I think the app and channel concept should be instead logically grouped together with event, not metadata. I think in some advanced use cases, event storage should not even be a hard requirement as e

Re: PredictionIO return array of objects

2017-06-17 Thread Donald Szeto
am assuming > this is done in Serving.scala? > > > On Fri, Jun 16, 2017 at 2:18 PM Donald Szeto wrote: > >> It also depends on how much you are willing to denormalize. If your >> items' metadata don't change, it would be fine to send items' metadata in >

Re: Passing an array of objects to get predictions?

2017-06-16 Thread Donald Szeto
access to JIRA? > > Best, > > Shane > > *Shane Johnson | 801.360.3350* > LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook > <https://www.facebook.com/shane.johnson.71653> > > On Fri, Jun 16, 2017 at 10:10 AM, Donald Szeto wrote: > >> Hi Shane

Re: PredictionIO return array of objects

2017-06-16 Thread Donald Szeto
It also depends on how much you are willing to denormalize. If your items' metadata don't change, it would be fine to send items' metadata in as part of the event. If you cannot denormalize and your scale is huge, you might want to consider putting metadata in a KV store. On Fri, Jun 16, 2017 at

Re: Passing an array of objects to get predictions?

2017-06-16 Thread Donald Szeto
Hi Shane, Look for the query class that is part of the template (it is a type parameter for the algorithm class). There you can customize your input query to accept an array of objects, then modify the predict method in the algorithm class to handle them. I believe this is getting asked more that

Re: App creation never exits

2017-06-10 Thread Donald Szeto
Hi Dan, Access key generation involves an entropy source which usually is pretty small on VMs or containers. When entropy runs out, the random number generator will block wait until there is entropy. Try using http://www.issihosts.com/haveged/ to seed your entropy. Regards, Donald On Sat, Jun 1

Re: Update default build targets

2017-06-08 Thread Donald Szeto
Unfortunately with OSS it is not easy to survey accurately the user base unless we provide some sort of opt-in feedback (automatically or by putting up a poll on the web site). We have many users who don't subscribe to mailing lists. I personally prefer updating source build defaults to a reasonab

Re: Sparkling Water-Deep Learning Energy Forecasting template

2017-06-08 Thread Donald Szeto
Hi Sudhanshu, What versions of Spark, Sparkling Water, PredictionIO are you running on? The community might not have tested the template on the latest versions of these technologies. Regards, Donald On Tue, Jun 6, 2017 at 9:43 PM SUDHANSHU KAUSHIK < sudhanshukaushik1.61...@gmail.com> wrote: > H

Re: Error during deploy

2017-06-08 Thread Donald Szeto
Hey Emilio, How are you storing your model? Failing to invert means it cannot deserialize your model from disk. It's definitely an error message that we could improve. Regards, Donald On Tue, Jun 6, 2017 at 5:48 AM Emilio Dettori wrote: > I'm working on a modified version of Text Classificatio

Re: Update default build targets

2017-06-05 Thread Donald Szeto
Hey all, this has a huge impact to the default build, so If you see any issue with this, please let us know as soon as possible. On Sun, Jun 4, 2017 at 10:25 PM, Shinsuke Sugaya wrote: > Hi all, > > We have a plan to change default build targets in PIO-83 and PIO-84. > Current versions look too

Re: PIO 0.12.0 and Graduation Time?

2017-06-05 Thread Donald Szeto
Hey Andrew, is it possible for us to create non-release binary distributions and ask Incubator PMC to help take a look without starting a voting thread? This would allow us iron out issues in parallel to working on 0.12. On Wed, May 31, 2017 at 9:08 AM, Andrew Purtell wrote: > Really glad to see

Re: What do "testing data points" in docs/manual/source/evaluation/index.html.md?

2017-06-05 Thread Donald Szeto
Hi Marius, Test data points come from the `readEval` method. References: https://predictionio.incubator.apache.org/api/current/#org.apache.predictionio.controller.PDataSource https://github.com/apache/incubator-predictionio-template-recommender/blob/develop/src/main/scala/DataSource.scala#L66 Re

Re: Perplexed by "No suitable driver"

2017-05-26 Thread Donald Szeto
Was this line correctly set? https://github.com/apache/incubator-predictionio/blob/develop/conf/pio-env.sh.template#L30 On Sat, May 27, 2017 at 12:00 AM Mars Hall wrote: > Also, re: the Postgres connection string, it is correct for my setup. My > docker-compose creates a Postgres host named "po

Re: 32 ERROR Storage$: Error initializing storage client for source PGSQL

2017-05-20 Thread Donald Szeto
Hey Shane, One quick workaround is to manually edit this line for now: https://github.com/apache/incubator-predictionio/blob/develop/bin/pio-shell#L62 and adding the JDBC assembly JAR after the main assembly JAR. Sorry for the brief reply as I'm traveling. I will follow up with more details whe

Re: PIO 0.12.0 and Graduation Time?

2017-05-20 Thread Donald Szeto
Thanks Pat for starting this. Added some edits to it. Everyone please feel free to comment. PMCs please feel free to request edit access. Very happy that we started the discussion! On Thu, May 18, 2017 at 1:50 PM Pat Ferrel wrote: > Should we apply for graduation to Apache TLP soon, like maybe

Re: Host PIO on Github (not ASF Git)

2017-05-18 Thread Donald Szeto
It is not clear to me from the gitbox page that we can move our source out of ASF controlled servers and still be covered by ASF Legal. It's a pretty big change if true. Did I miss an announcement from ASF? Personally am happy to have things on GitHub because the flow would be a lot simpler. On T

Re: New PMC member and committer: Takahiro Hagino

2017-05-17 Thread Donald Szeto
up. > > > > He will talk about PredictionIO at JJUG Cross Community Conference > > that is a largest Java technical conference in Japan this Saturday. > > I'm looking forward to his talk! > > > > 2017-05-18 0:53 GMT+09:00 Donald Szeto : > > > Correction

New product manager: Sara Asher

2017-05-17 Thread Donald Szeto
Hi all, The Project Management Committee (PMC) for Apache PredictionIO (incubating) has asked Sara Asher to become a product manager, and we are pleased to announce that she has accepted. Sara is a Director of Product Management for Salesforce Einstein, where she creates products that let people

Re: New PMC member and committer: Takahiro Hagino

2017-05-17 Thread Donald Szeto
Correction: Takahiro is not a committer of Apache Portals. Sorry for the misinformation. On Wed, May 17, 2017 at 8:50 AM, Donald Szeto wrote: > Hi all, > > The Project Management Committee (PMC) for Apache PredictionIO > (incubating) has asked Takahiro Hagino to become a PM

New PMC member and committer: Takahiro Hagino

2017-05-17 Thread Donald Szeto
Hi all, The Project Management Committee (PMC) for Apache PredictionIO (incubating) has asked Takahiro Hagino to become a PMC member and committer, and we are pleased to announce that he has accepted. He is also a committer of the Apache Portals project. He has made major contributions to the Pre

New PMC member and committer: Takako Shimamoto

2017-05-17 Thread Donald Szeto
Hi all, The Project Management Committee (PMC) for Apache PredictionIO (incubating) has asked Takako Shimamoto to become a PMC member and committer, and we are pleased to announce that she has accepted. She has made major contributions to the PredictionIO 0.11.0 release by adding Elasticsearch 5

Re: Can i install Apache prediction IO on Windows8 machine

2017-05-16 Thread Donald Szeto
You might want to try Docker on Windows with one of these community contributed Docker container images: http://predictionio.incubator.apache.org/community/projects/#docker-installation-for-predictionio On Tue, May 16, 2017 at 12:07 PM Naresh Peddy wrote: > Hi Team, > > Currently am using window

New PMC member and committer: Shinsuke Sugaya

2017-05-10 Thread Donald Szeto
Hi all, The Project Management Committee (PMC) for Apache PredictionIO (incubating) has asked Shinsuke Sugaya to become a PMC member and committer, and we are pleased to announce that he has accepted. He is also a committer of the Apache Portals project. He has made major contributions to the Pre

Re: Spark pipeline in predictionIO

2017-05-10 Thread Donald Szeto
We are going to publish a roadmap soon and one of roadmap items is to support Spark ML pipelines natively in parallel of DASE. If you are interested in helping developing the core code to support it, please reach out in the dev mailing list. On Wed, May 10, 2017 at 1:02 PM Ravi Kiran wrote: > C

Re: I get "Error: Could not find or load main class org.apache.predictionio.tools.console.Console"

2017-05-09 Thread Donald Szeto
Did you build your source according to http://predictionio.incubator.apache.org/install/install-sourcecode/ ? On Mon, May 8, 2017 at 7:51 PM Some idoit on the internet < branda...@gmail.com> wrote: > I got it when I did ./bin/pio-start-all > how do I fix it? >

New PMC member and committer: Naoki Takezoe

2017-05-08 Thread Donald Szeto
Hi all, The Project Management Committee (PMC) for Apache PredictionIO (incubating) has asked Naoki Takezoe to become a PMC member and committer, and we are pleased to announce that he has accepted. He has made major contributions to the PredictionIO 0.11.0 release by adding Elasticsearch 5 suppo

Re: Missing lib folder and contents

2017-05-04 Thread Donald Szeto
Once you've done make-distribution.sh there should be a new tarball named PredictionIO-0.11.0-incubating.tar.gz at the root level of your codetree. We recommend extracting and using that distribution instead of directly from the codetree. The behavior you are seeing is a bug anyway, and is already

Re: [ERROR] [Engine$] No engine found. Your build might have failed. Aborting. - using PredictionIO-0.11.0-incubating scala.version=2.11.6 spark.version=2.1.0 elasticsearch.version=2.3.3

2017-05-01 Thread Donald Szeto
Hi, Can you try changing scalaVersion in the template's build.sbt? Regards, Donald On Mon, May 1, 2017 at 5:17 PM Barthe van Doorn wrote: > PredictionIO-0.11.0-incubating > scala.version=2.11.6 > spark.version=2.1.0 > elasticsearch.version=2.3.3 > > When I try to build an engine, I get error [

Re: Ruby Api Docs

2017-04-28 Thread Donald Szeto
Hi, Sorry for the broken link. Please help file a JIRA issue if possible. Meanwhile you can generate RDoc directly from Ruby SDK source code. Regards, Donald On Fri, Apr 28, 2017 at 2:29 PM André Hatlo wrote: > Hello > I have a bachelor theisis with PIO, Spark and ROR. > I'm trying to figure

Re: Predition.io setup with Azure

2017-04-28 Thread Donald Szeto
Hi Pankil, Looks like HDInsight runs Spark on YARN. You would need to point PIO to the right Hadoop config files, and specify the correct master URL (yarn) when you run "pio train". Regards, Donald On Fri, Apr 28, 2017 at 12:12 PM Pankil Doshi wrote: > Hi, > > Has anyone tried running predicti

[ANNOUNCE] Apache PredictionIO 0.11.0-incubating Release

2017-04-25 Thread Donald Szeto
The Apache PredictionIO team would like to announce the release of Apache PredictionIO 0.11.0-incubating. Release notes are here: https://github.com/apache/incubator-predictionio/blob/v0.11.0-incubating/RELEASE.md Apache PredictionIO (incubating) is an open source Machine Learning Server built on

Re: pio train "wiki"

2017-04-11 Thread Donald Szeto
I haven't looked at the source code of the template but a quick look at the output of the sanity check it said "total observation, 0". Do you have any training data? On Tue, Apr 11, 2017 at 4:14 AM YOUSSEF ASSATA wrote: > please why i'm getting this error >

Re: Prediction.IO error while creating new app

2017-04-11 Thread Donald Szeto
Is the java command on your search path pointing to your Java 8 installation? In any case, you can set your JAVA_HOME environment variable to make sure pio is using the right Java installation. On Tue, Apr 11, 2017 at 12:21 AM Ravi Kiran wrote: > I am starting new app using query > > pio ap

Re: how to unsubscribe ?

2017-04-10 Thread Donald Szeto
Send an email to user-unsubscr...@predictionio.incubator.apache.org. On Mon, Apr 10, 2017 at 8:13 PM Pengcheng Fang wrote: > how to unsubscribe ? > > > >

Re: Existing javascript SDK for predicitonIO (0.10) ?

2017-04-10 Thread Donald Szeto
You can also create access keys for existing apps that have write permissions to certain event names only. It is useful for client side event collection, and is how some major analytics vendor JS SDKs limit client side keys from polluting your event log. Please take a look at `pio help accesskey`

Re: Empty result even though there are events

2017-04-04 Thread Donald Szeto
Are you using the appName in engine.json that corresponds to the appId you used when you did `pio import`? Setting a new appName in engine.json will not automatically create a new app. Apps are created when you do `pio app new`. On Tue, Apr 4, 2017 at 8:13 PM Vaghawan Ojha wrote: Hi, Yes, I cou

Re: PIO train error on Spark/Hbase remote cluster

2017-03-31 Thread Donald Szeto
setup through Ambari. > > > On Mar 31, 2017, at 6:29 AM, Donald Szeto wrote: > > Hi Malay, > > Is your Spark cluster a standalone deployment or based on YARN? > > Regards, > Donald > > On Thu, Mar 30, 2017 at 11:48 PM Malay Tripathi > wrote: > >> Hello,

Re: The recommendation engine shows only one result for all users

2017-03-31 Thread Donald Szeto
Hi, A statistics of your event data's distribution would be helpful. Regards, Donald On Tue, Mar 28, 2017 at 11:27 AM Vaghawan Ojha wrote: > Hi, > > I've added more than 30 events in the engine, and I'm using this > template ( > https://github.com/apache/incubator-predictionio-template-rec

Re: PIO train error on Spark/Hbase remote cluster

2017-03-31 Thread Donald Szeto
Hi Malay, Is your Spark cluster a standalone deployment or based on YARN? Regards, Donald On Thu, Mar 30, 2017 at 11:48 PM Malay Tripathi wrote: > Hello, > > I am running pio train *on an edge node* of distributed 8 node spark > cluster & 3 node Hbase. > When I run "pio train" the job runs but

Re: [PredictionIO Error] Running Hbase

2017-03-12 Thread Donald Szeto
Hi Amy, Since event server keeps adding events to the backend, the storage will grow indefinitely unless you implement some sort of data retention policy that periodically. In 0.11, there are two options for this situation: - You may use SelfCleaningDataSource. Backing up your existing data is hi

Re: PredictionIO with remote Spark and Elasticsearch

2017-03-01 Thread Donald Szeto
Hi Clifford, To use a remote Spark cluster, use passthrough command line arguments on the CLI, e.g. pio train -- --master spark://your_master_url Anything after a lone -- will be passed to spark-submit verbatim. For more information try "pio help". To use a remote Elasticsearch cluster, please

Re: Modifying the eventserver

2017-01-25 Thread Donald Szeto
at gives out a classnotfound > error for slf4jlogger, > > Best, > -ZZ > On Tue, Jan 24, 2017 at 2:11 PM Donald Szeto wrote: > > Hi, > > This question is more suitable for the dev list, so I am moving the thread > there. > > You will need to use make-distribution.sh f

Re: Modifying the eventserver

2017-01-24 Thread Donald Szeto
Hi, This question is more suitable for the dev list, so I am moving the thread there. You will need to use make-distribution.sh for the command line to take effect. The command line uses the assembly JAR produced by the make-distribution.sh script. It would be great to add some kind of support i

Re: Spark Executor lost error

2017-01-19 Thread Donald Szeto
Do you have more detail logs from Spark executors? Regards, Donald On Wed, Dec 14, 2016 at 3:26 AM Bansari Shah wrote: > Hi all, > > I am trying to use spark environment in predict function of ML engine for > text analysis. It extends P2LAlgorithm algorithm. System works on > standalone cluster

Re: server.conf in SSLConfiguration is null

2017-01-19 Thread Donald Szeto
, Jan 19, 2017 at 9:16 AM Venkatesh Kumar Srinivasan < venkat...@studiodiseno.com> wrote: > Yes. I am calling from different script. > > > > Regards > > Venki > > > > > > *From: *Donald Szeto > *Reply-To: * > *Date: *Thursday, 19 January 2017 a

Re: server.conf in SSLConfiguration is null

2017-01-19 Thread Donald Szeto
How are you calling CreateServer? Are you bypassing the command line interface? Regards, Donald On Thu, Jan 19, 2017 at 5:52 AM Venkatesh Kumar Srinivasan < venkat...@studiodiseno.com> wrote: > Hello > > > > I am trying to deploy a model by calling CreateServer class, but when > class loader ini

Re: PredictionIO :: [error] (*:update) sbt.ResolveException: unresolved dependency: io.prediction#pio-build;0.9.0: not found

2017-01-11 Thread Donald Szeto
& Regards, > Yasothai Rasappan > SSE *|* *yasoth...@vishwak.com* > *Vishwak Solutions Pvt Ltd* > ( *044 - **4222 2300* *| Extn - 439* > *From:* Pat Ferrel [mailto:p...@occamsmachete.com ] > *Sent:* 11 January 2017 00:24 > *To:* user@predictionio.incubator.apache.org

Re: Issue while training Recommendation Engine

2016-11-01 Thread Donald Szeto
this implementation?? I really appreciate any > comments about that. > > Do you have any reference or documentation to try Increase Spark driver > memory locally?? > > Best Regards, > > On Mon, Oct 31, 2016 at 12:51 PM, Donald Szeto wrote: > >> Hi Amado, >> >

Re: Best way to implement description of expected events and queries

2016-11-01 Thread Donald Szeto
31, 2016 at 6:41 PM, Donald Szeto wrote: > >> Hi Sebastiaan, >> >> Would this be sufficient for your use case: http://stackoverflow.com >> /questions/26500539/scala-getting-field-and-type-of-field-of-a-case-class >> >> This can be put in a trait as a method and

Re: Best way to implement description of expected events and queries

2016-10-31 Thread Donald Szeto
Hi Sebastiaan, Would this be sufficient for your use case: http://stackoverflow.com/questions/26500539/scala-getting-field-and-type-of-field-of-a-case-class This can be put in a trait as a method and mix in to different case classes. If the name of the type signature is enough for your GUI that w

Re: Issue while training Recommendation Engine

2016-10-31 Thread Donald Szeto
Hi Amado, You have several options: 1. Train on a real Spark cluster with more executor memory. 2. Increase Spark driver memory if you want to test locally. 3. Reduce number of iterations in engine.json. Regards, Donald On Sun, Oct 30, 2016 at 7:42 AM, Amado Espinosa wrote: > > Hi, I stuck tr

Re: PredictionIO | Similar Product Template Evaluation

2016-10-31 Thread Donald Szeto
Hi Amal, Unfortunately not all templates come baked in with evaluation. Try start building your own by following http://predictionio.incubator.apache.org/evaluation/. You can also reference other template's evaluation, such as https://github.com/apache/incubator-predictionio-template-recommender/b

Re: Hbase as Event Store | Data Import Code for Reference

2016-10-31 Thread Donald Szeto
Hi Amal, If you have a lot of data to import in batch, you might want to check this out: http://predictionio.incubator.apache.org/datacollection/batchimport/ The Python script goes through the Event Server, which uses the Storage API. The HBase driver code is here: https://github.com/apache/incub

Re: sbt-plugin

2016-10-23 Thread Donald Szeto
Hi Georg, This plugin will probably be deprecated for 0.10.0 and onward so that the engine build can be easily built be all other IDEs. The old version plugin will still work for now though. Regards, Donald On Thursday, October 20, 2016, Georg Heiler wrote: > Hi, > > will the sbt plugin be upd

Re: Installing on Linux/Mac through tar

2016-10-23 Thread Donald Szeto
A binary release can be released once all transitive dependencies have gone through license compatibility check and appropriate license files and notices included. This is a lengthy and manual process so would highly appreciate any help. Regards, Donald On Wednesday, October 19, 2016, Pat Ferrel

Re: PredictionIO with Remote Spark | pio train error

2016-10-23 Thread Donald Szeto
Hi Amal, For yarn-cluster mode, please use the --scratch-ufo argument and point it to a location on HDFS where you have write access to. PIO will copy the necessary files to the HDFS location for your yarn-cluster Spark driver to read. Regards, Donald On Tuesday, October 18, 2016, amal kumar wr

Re: Dependency issue in pio build.

2016-10-23 Thread Donald Szeto
Hi, This might be related to the misnamed parent artifact. Have you try again and see if this has been resolved now? Regards, Donald On Monday, October 17, 2016, infoquest india wrote: > I am also facing same issue. Please help. > > Thanks > Gaurav > http://www.infoquestsolutions.com > Turning

Re: Install v10.0-incubating

2016-10-23 Thread Donald Szeto
Hi Georg, Has this been resolved? It looks like a network glitch where SBT itself cannot be downloaded. Regarding installation, you will need to follow the official documentation on building PredictionIO from source. A binary can be released when we have all transitive dependencies go through lic

Re: PredictionIO Error | HBase as Event Server

2016-10-23 Thread Donald Szeto
Hi Amal, It looks like your PIO host cannot connect to your HBase host ( *ip-xxx-xx-xx-xx.us-west-2.compute.internal/xxx.xx.xx.xx:16020*). Could you verify that the destination port is open and both hosts can communicate? Regards, Donald On Thursday, October 13, 2016, amal kumar wrote: > Hi, >

Re: Unresolved dependency

2016-10-11 Thread Donald Szeto
Hi Bansari, For production template, please used release artifacts on the central Maven repository: http://search.maven.org/#artifactdetails%7Corg.apache.predictionio%7Capache-predictionio-core_2.10%7C0.10.0-incubating%7Cjar In your case, please use "org.apache.predictionio"%% "apache-predic

Re: Error in pio status

2016-10-10 Thread Donald Szeto
Hi Bansari, Looks like the underlying problem is ZooKeeper not being able to start. Are you using the embedded ZooKeeper or a standalone ZooKeeper cluster? If you could afford it, it might help to clear the ZooKeeper data and try again. Regards, Donald On Mon, Oct 10, 2016 at 1:24 AM, Bansari Sh

[ANNOUNCE] Apache PredictionIO 0.10.0-incubating Release

2016-10-08 Thread Donald Szeto
The Apache PredictionIO team would like to announce the release of Apache PredictionIO 0.10.0-incubating. Apache PredictionIO (incubating) is an open source Machine Learning Server built on top of state-of-the-art open source stack, that enables developers to manage and deploy production-ready pre

Re: PredictionIO with redshift event source | Unable to configure

2016-10-06 Thread Donald Szeto
Hi Amal, Since you are using Redshift and their JDBC driver probably collides with those provided from Postgres, you might have to use a different data storage for your meta data and model data. Regards, Donald On Wednesday, October 5, 2016, amal kumar wrote: > Hello, > > I am trying to config

Re: Unable to Access Event Server or Engine from Web Browser (Outside SSH)

2016-10-06 Thread Donald Szeto
Hi Amal, Could you try disabling IPv6 of your Java processes and see if it helps? export JAVA_OPTS=-Djava.net.preferIPv4Stack=true pio eventserver --ip 0.0.0.0 Regards, Donald On Thursday, October 6, 2016, amal kumar wrote: > Hi Gustavo, > > I have tried both curl and web browser from outside

Re: predictionIO classpath

2016-10-05 Thread Donald Szeto
Hi Cristina, If you are using 0.9.x, you can keep using the old "io.prediction" package name for your engine source code. When you follow the tutorial online, just use "io.prediction" in place of "org.apache.predictionio" wherever you come across it. If you are trying 0.10.0 though, make sure you

Re: Use of common argument scratch-uri and remote deployment

2016-10-05 Thread Donald Szeto
cluster. In this case does scratch-uri will work or we have to > follow another process. > > Please suggest me. > > Thank you > Regards, > Bansari > > On Wed, Oct 5, 2016 at 1:30 AM, Donald Szeto wrote: > >> Hi Bansari, >> >> The --scratch-uri swit

Re: Use of common argument scratch-uri and remote deployment

2016-10-04 Thread Donald Szeto
AM, Bansari Shah wrote: > > Hi Donald, > > I am running my spark cluster of 3 node with YARN and spark driver is > within cluster. > > Thanks > Regards, > Bansari > > On Tue, Oct 4, 2016 at 9:59 PM, Donald Szeto wrote: > >> Hi Bansari, >> >> How

Re: Use of common argument scratch-uri and remote deployment

2016-10-04 Thread Donald Szeto
Hi Bansari, How are you running your Spark cluster? Standalone, YARN, or Mesos? Are you running the Spark driver on the client or within the cluster? Regards, Donald On Tue, Oct 4, 2016 at 5:55 AM, Bansari Shah wrote: > Hi, > Can you please guide me how to use 'scratch-uri' argument in case of

Re: Launching PredictionIO on aws

2016-10-03 Thread Donald Szeto
Hi Jenna, The AWS image is no longer supported as of now. You may set up a raw AWS EC2 instance and install PredictionIO there. Regards, Donald On Mon, Oct 3, 2016 at 2:36 PM, Jenna Kwon wrote: > Hello everyone, > > I would love to deploy my engine and was wondering what the current state > fo

Re: Running predictionIO on a web server other than localhost

2016-10-03 Thread Donald Szeto
Hi Jenna, You probably want to have a reverse proxy in front of your app (localhost:3000) so that it is exposed to the Internet via standard ports (80 for HTTP or 443 for HTTPS). Your app can communicate to the event server and engine over localhost. They don't need to be exposed to the Internet.

Re: predictionIO classpath

2016-10-03 Thread Donald Szeto
Hi Cristina, Sorry for the confusion. We are pretty close to our first official release. We will send out a note and update all docs once that happens. Regards, Donald On Mon, Oct 3, 2016 at 9:05 AM, Chan Lee wrote: > Hi Cristina, > > You're right. The classpath change is part of v0.10 which i

Re: delay of engines

2016-09-26 Thread Donald Szeto
Sorry for side-tracking. I think Kappa architecture is a promising paradigm, but including batch processing from the canonical store to the serving layer store should still be necessary. I believe this somewhat hybrid Kappa-Lambda architecture would be generic enough to handle many use cases. If th

Re: How to access Spark Context in predict?

2016-09-26 Thread Donald Szeto
Hi Hasan, Does your randomForestModel contain any RDD? If so, implement your algorithm by extending PAlgorithm, have your model extend PersistentModel, and implement PersistentModelLoader to save and load your model. You will be able to perform RDD operations within predict() by using the model's

Re: Remove engine registration

2016-09-21 Thread Donald Szeto
ing "No automatic version matching of PIO binary distribution and >> artifacts version used in the engine template": >> >> The Heroku slug contains the PredictionIO binary distribution used to >> build the engine, so there's never a version matching issue. I guess

Remove engine registration

2016-09-16 Thread Donald Szeto
Hi all, I want to start the discussion of removing engine registration. How many people actually take advantage of being able to run pio commands everywhere outside of an engine template directory? This will be a nontrivial change on the operational side so I want to gauge the potential impact to

New PMC member and committer: Paul Li

2016-09-15 Thread Donald Szeto
Hi all, The Project Management Committee (PMC) for Apache PredictionIO (incubating) has asked Paul Li to become a PMC member and committer, and we are pleased to announce that he has accepted. Paul Li was a regular committer to the PredictionIO codebase before its donation to the Apache Software

Re: pio build & train - server available?

2016-09-08 Thread Donald Szeto
pio build and pio train are independent processes from pio eventserver and pio deploy, so you can still take events and serve queries. On Thu, Sep 8, 2016 at 11:53 AM, Gustavo Frederico < gustavo.freder...@thinkwrap.com> wrote: > Is the server available for events and queries while executing pio

Re: Can The Classification Engine Handle Multiple Sets of Independent Data

2016-09-07 Thread Donald Szeto
Hi, If you are using the stock classification engine template, your easiest way out would be to use multiple engine servers. If that is not desired, you could modify the engine template to handle different data sets independently. Regards, Donald On Tue, Sep 6, 2016 at 5:49 AM, Dipen Patel wrot

  1   2   >