Re: Spark 2.x/scala 2.11.x release

big +1 If you are planning to branch off the 0.13.0 tag let me know, I have a speedup that is in my scala 2.11 fork of 0.13.0 that needs to be released From: Andrew Palumbo Reply: dev@mahout.apache.org

Re: Dynamically change parameter list

meters in this step. I'll let you know if it works. Thanks, Tihomie On Feb 12, 2018 23:34, "Pat Ferrel" <p...@occamsmachete.com <mailto:p...@occamsmachete.com>> wrote: This is an interesting question. As we make more mature full featured engines they will begin to employ hyper

Re: Dynamically change parameter list

This is an interesting question. As we make more mature full featured engines they will begin to employ hyper parameter search techniques or reinforcement params. This means that there is a new stage in the workflow or a feedback loop not already accounted for. Short answer is no, unless you want

Re: pio train on Amazon EMR

I agree, we looked at using EMR and found that we liked some custom Terraform + Docker much better. The existing EMR defined by AWS requires refactoring PIO or using it in yarn’s cluster mode. EMR is not meant to host any application code except what is sent into Spark in serialized form.

Re: Frequent Pattern Mining - No engine found. Your build might have failed. Aborting.

This list is for support of ActionML products, not general PIO support. You can get that on the Apache PIO user mailing list, where I have forwarded this question. Several uses of FPM are supported by the Universal Recommender, such as Shopping cart recommendations. That is a template we

Re: PIO error

Unfortunately I can’t possibly guess without more information. What do the logs say when pio cannot be started? Are all these pio instances separate, not in a cluster? In other words does each pio server have all necessary services running on them? I assume none is sleeping like a laptop does? I

Re: Prediction IO install failed in Linux

This would be very difficult to do. Even if you used a machine connected to the internet to download things like pio, spark, etc. the very build tools used (sbt) expect to be able to get code from various repositories on the internet. To build templates would further complicate this since each

Re: Need Help Setting up prediction IO

ng Pio for now, and everything is working great. Thanks for the support though. Few question- 1.Does Pio latest support Mongodb or NoSQL? 2.Memory uses by Pio, Is there any max memory limit set, If need be can it be set? Thanks Rajesh On Jan 11, 2018 10:25 PM, "Pat

The Universal Recommender v0.7.0

We have been waiting to release the UR v0.7.0 for testing (done) and the release of Mahout v0.13.1 (not done) Today we have released the UR v0.7.0 anyway. This comes with: Support for PIO v0.12.0 Requires Scala 2.11 (can be converted to use Scala 2.10 but it’s a manual process) Requires

Re: Need Help Setting up prediction IO

The version in the artifact built by Scala should only have the major version number so 2.10 or 2.11. PIO 0.10.0 needs 2.10. Where, and what variable did you set to 2.10.4? That is the problem. There will never be a lib built for 2.10.4, it will always be 2.10. On Jan 11, 2018, at 5:15 AM,

Re: Using Dataframe API vs. RDD API?

ike: val RDD_2_DF = sqlContext.createDataFrame(yourRDD).toDF("col1", "col2") On Thu, 4 Jan 2018 at 23:10 Pat Ferrel <p...@occamsmachete.com <mailto:p...@occamsmachete.com>> wrote: Actually there are libs that will read DFs from HBase https://svn.apache.org/repos/asf

Re: Using Dataframe API vs. RDD API?

ecause the schema of the EventStore is not guaranteed to remain as-is. The safest way is to translate or get DFs integrated to PIO. I think there is an existing Jira that request Spark ML support, which assumes DFs. On Jan 4, 2018, at 12:25 PM, Pat Ferrel <p...@occamsmachete.com> wrote: Funny yo

Re: Using Dataframe API vs. RDD API?

Funny you should ask this. Yes, we are working on a DF based Universal Recommender but you have to convert the RDD into a DF since PIO does not read out data in the form of a DF (yet). This is a fairly simple step of maybe one line of code but would be better supported in PIO itself. The issue

Re: Error: "unable to undeploy"

The UR does not require more than one deploy (assuming the server runs forever). Retraining the UR automatically re-deploys the new model. All other Engines afaik do require retrain-redeploy. Users should be aware that PIO is a framework that provides no ML function whatsoever. It supports a

Re: App still returns results after pio app data-delete

BTW there is a new Chrome extension that lets you browse ES and create any JSON query. Just found it myself after Sense stopped working in Chrome. Try ElasticSearch Head, found in the Chrome store. On Jan 2, 2018, at 9:53 AM, Pat Ferrel <p...@occamsmachete.com> wrote: Have a look at

Re: App still returns results after pio app data-delete

u want to delete. On Jan 2, 2018, at 12:22 AM, Noelia Osés Fernández <no...@vicomtech.org> wrote: Thanks for the explanation! How do I delete the ES index? is it just DELETE /my_index_name? Happy New Year! On 22 December 2017 at 19:42, Pat Ferrel <p...@occamsmachete.com <mailto:p...

Re: Recommendation return score more than 5

templates/recommendation/quickstart/> 2017-12-22 22:12 GMT+03:00 Pat Ferrel <p...@occamsmachete.com <mailto:p...@occamsmachete.com>>: Implicit means you assign a score to the event based on your own guess. Explicit uses ratings the user makes. One score is a guess by you (like a 4 for buy) a

Re: How to import item properties dynamically?

The properties go into the Event Store immediately but you have to train to get them into the model, this assuming your template support item properties. If yo uare using the UR, the properties will not get into the model until the next `pio train…` On Dec 22, 2017, at 3:37 AM, Noelia Osés

Re: Recommendation return score more than 5

kes an score, and the buy implicitly puts the rating at 4 (out of 5, as I think). And I still did not understand exactly where to look for me and what to correct, so that incoming and predicted estimates were on the same scale. 2017-12-19 4:10 GMT+03:00 Pat Ferrel <p...@occamsmachete

Re: Recommended Configuration

That is enough for a development machine and may work if you data is relatively small but for big data clusters of CPU with a fair amount of RAM and Storage are required. The telling factor is partly how big your data is but also how is combines to form models, which will depend on which

Re: New Website

ate the vote on the logo and the site. Sent from my Verizon Wireless 4G LTE smartphone Original message -------- From: Pat Ferrel <p...@occamsmachete.com> Date: 12/13/2017 09:47 (GMT-08:00) To: dev@mahout.apache.org Subject: Re: New Website Due to 8 years of Ruby cruft I can’t get the Je

Re: New Website

<https://sarcasticresonance.files.wordpress.com/2017/01/cubes1.png?w=721=2> On Dec 6, 2017, at 11:27 AM, Pat Ferrel <p...@occamsmachete.com> wrote: Since you’ve already built it can you share a screen shot? The mockup I saw on Slack looked awesome. Also a logo change is a lot more far reaching so can we have

Re: New Website

Since you’ve already built it can you share a screen shot? The mockup I saw on Slack looked awesome. Also a logo change is a lot more far reaching so can we have at least a little discussion? On Dec 6, 2017, at 10:18 AM, Andrew Musselman wrote: +1, looks great

Re: User features to tailor recs in UR queries?

The User’s possible indicators of taste are encoded in the usage data. Gender and other “profile" type data can be encoded a (user-id, gender, gender-id) but this is used and a secondary indicator, not as a filter. Only item properties are used a filters for some very practical reasons. For one

[jira] [Assigned] (MAHOUT-2023) Drivers broken, scopt classes not found

[ https://issues.apache.org/jira/browse/MAHOUT-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pat Ferrel reassigned MAHOUT-2023: -- Assignee: Trevor Grant (was: Pat Ferrel) > Drivers broken, scopt classes not fo

[jira] [Commented] (MAHOUT-2023) Drivers broken, scopt classes not found

[ https://issues.apache.org/jira/browse/MAHOUT-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267877#comment-16267877 ] Pat Ferrel commented on MAHOUT-2023: This is a big issue. It shows up when you run a Spark CLI

Re: Prepping Release

https://issues.apache.org/jira/browse/MAHOUT-2023 is the only blocker I see. It’s a big one since it make drivers and GPU bindings not work in clusters (I think). But the fix is probably easy. On Nov 27, 2017, at 8:06 AM, Jim Jagielski

[jira] [Resolved] (MAHOUT-2020) Maven repo structure malformed

[ https://issues.apache.org/jira/browse/MAHOUT-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pat Ferrel resolved MAHOUT-2020. Resolution: Fixed Trevor found a script in Spark that seems to fix this when used during a build

Re: Log-likelihood based correlation test?

ou say "compare to each item in the model", do you mean each row in PtP? On 21 November 2017 at 19:56, Pat Ferrel <p...@occamsmachete.com <mailto:p...@occamsmachete.com>> wrote: No PtP non-zero elements have LLR calculated. The highest scores in the row are kept, or ones ab

[jira] [Commented] (MAHOUT-2023) Drivers broken, scopt classes not found

[ https://issues.apache.org/jira/browse/MAHOUT-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259971#comment-16259971 ] Pat Ferrel commented on MAHOUT-2023: Yep, the mahout...dependency-reduced.jar excludes anything

[jira] [Comment Edited] (MAHOUT-2023) Drivers broken, scopt classes not found

[ https://issues.apache.org/jira/browse/MAHOUT-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258255#comment-16258255 ] Pat Ferrel edited comment on MAHOUT-2023 at 11/20/17 10:40 PM: --- ok

Re: Log-likelihood based correlation test?

That would be a nice feature if it doesn't exist. The way I've gotten some insight into what the model is doing is by when using PIO / UR is by inspecting the the ElasticSearch index that gets created because it has the "significant" values populated in the documents (though not the actual

[jira] [Commented] (MAHOUT-2023) Drivers broken, scopt classes not found

[ https://issues.apache.org/jira/browse/MAHOUT-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258258#comment-16258258 ] Pat Ferrel commented on MAHOUT-2023: Whoa, that is a big clue I think. Everything without

[jira] [Comment Edited] (MAHOUT-2023) Drivers broken, scopt classes not found

[ https://issues.apache.org/jira/browse/MAHOUT-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258255#comment-16258255 ] Pat Ferrel edited comment on MAHOUT-2023 at 11/18/17 11:32 PM: --- ok

Re: Error in getting Total Events in a predictionIo App

You should use pio 0.12.0 if you need Elasticsearch 5.x On Nov 14, 2017, at 6:39 AM, Abhimanyu Nagrath wrote: Hi , I am new to predictionIo using version V0.11-incubating (spark - 2.6.1 , hbase - 1.2.6 , elasticsearch - 5.2.1) . Started the prediction server with

Re: Which template for predicting ratings?

What I was saying is the UR can use ratings, but not predict them. Use MLlib ALS recommenders if you want to predict them for all items. On Nov 13, 2017, at 9:32 AM, Pat Ferrel <p...@occamsmachete.com> wrote: What we did in the article I attached is assume 1-2 is dislike, and 4-5 i

Re: Does PIO support [ --master yarn --deploy-mode cluster ]?

yarn-cluster mode is supported but extra config needs to be set so the driver can be run on a remote machine. I have seen instructions for this on the PIO mailing list. On Nov 12, 2017, at 7:30 PM, wei li wrote: Hi Pat Thanks a lot for your advice. We are using

Re: "LLR with time"

gt; Thanks for your thoughts, I am happy I can rule something out given the >> domain (poisson llr). Luckily the domain I'm working on is event >> recommendations, so there is a natural deterministic item expiry (as >> compared to christmas like stuff). >> >> Again

Re: "LLR with time"

t) vs itemA(t-1) > .. > > and derive multiple indicators per item to be indexed. > > But this all relies on discretizing time into buckets and not looking at > the distribution of time between events like in presentation above - maybe > there is something way smarter > > Johann

Re: "LLR with time"

. On Nov 10, 2017, at 4:12 PM, Pat Ferrel <p...@occamsmachete.com> wrote: So your idea is to find anomalies in event frequencies to detect “hot” items? Interesting, maybe Ted will chime in. What I do is take the frequency, first, and second, derivatives as measures of popularity, incr

Re: "LLR with time"

So your idea is to find anomalies in event frequencies to detect “hot” items? Interesting, maybe Ted will chime in. What I do is take the frequency, first, and second, derivatives as measures of popularity, increasing popularity, and increasingly increasing popularity. Put another way popular,

Re: PIO + ES5 + Universal Recommender

If so, it is not said in the PredictionIO installation instructions nor on the Mahout instructions. I apologise if this is an obvious question for those familiar with the Apache projects, but for an outsider like me it helps when everything (even the most silly details) is spelled out. Thanks

Re: PIO + ES5 + Universal Recommender

Very sorry, it was incorrectly set to private. Try it again. On Nov 7, 2017, at 7:26 AM, Pat Ferrel <p...@occamsmachete.com <mailto:p...@occamsmachete.com>> wrote: https://github.com/actionml/mahout <https://github.com/actionml/mahout>

Re: PIO + ES5 + Universal Recommender

sitory 'https://github.com/actionml/mahout.git/ <https://github.com/actionml/mahout.git/>' not found On 3 November 2017 at 22:27, Pat Ferrel <p...@occamsmachete.com <mailto:p...@occamsmachete.com>> wrote: The exclusion rules are working now along with the integration-test. We have

Re: Implementing cart and wishlist item events into Ecommerce recommendation template

library and expect to modify the template code. There is also the Complimentary Purchase template, which does shopping carts but, from my rather prejudiced viewpoint, if you need to switch templates use one that supports every use-case you are likely to need. On Nov 4, 2017, at 9:34 AM, Pat

Re: Implementing cart and wishlist item events into Ecommerce recommendation template

The Universal Recommender supports several types of “item-set” recommendations: 1) Complimentary Purchases. which are things bought with what you have in the shopping cart. This is done by training on (cart-id, “add-to-cart”, item-id) and querying with the current items in the user’s cart. 2)

Re: PIO + ES5 + Universal Recommender

Ack, I hate this &^%&%^& touchbar! What I meant to say was: We have a version of the universal recommender working with PIO-0.12.0 that is ready for brave souls to test. This includes some speedups and quality of recommendation improvements, not yet documented. Known bugs: exclusion rules

PIO + ES5 + Universal Recommender

We have a version working here: https://github.com/actionml/universal-recommender.git checkout 0.7.0-SNAPSHOT once you pull the repo. Known bug: exclusion rules not working. This will be fixed before release in the next few days Issues:

[jira] [Comment Edited] (MAHOUT-2020) Maven repo structure malformed

[ https://issues.apache.org/jira/browse/MAHOUT-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234411#comment-16234411 ] Pat Ferrel edited comment on MAHOUT-2020 at 11/1/17 5:22 PM: - Nothing to do

[jira] [Issue Comment Deleted] (MAHOUT-1951) Drivers don't run with remote Spark

[ https://issues.apache.org/jira/browse/MAHOUT-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pat Ferrel updated MAHOUT-1951: --- Comment: was deleted (was: odd, I did not resolve this as history says, at least did not mean

[jira] [Comment Edited] (MAHOUT-2020) Maven repo structure malformed

[ https://issues.apache.org/jira/browse/MAHOUT-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190340#comment-16190340 ] Pat Ferrel edited comment on MAHOUT-2020 at 11/1/17 1:23 AM: - This may

[jira] [Updated] (MAHOUT-2020) Maven repo structure malformed

[ https://issues.apache.org/jira/browse/MAHOUT-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pat Ferrel updated MAHOUT-2020: --- Description: The maven repo is built with scala 2.10 always in the parent pom's

[jira] [Comment Edited] (MAHOUT-2020) Maven repo structure compatibility with SBT

[ https://issues.apache.org/jira/browse/MAHOUT-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221263#comment-16221263 ] Pat Ferrel edited comment on MAHOUT-2020 at 10/26/17 9:28 PM: -- my script

[jira] [Commented] (MAHOUT-2020) Maven repo structure compatibility with SBT

[ https://issues.apache.org/jira/browse/MAHOUT-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221258#comment-16221258 ] Pat Ferrel commented on MAHOUT-2020: The problem here is with the multi-artifact build. The last

Templates First

PredictionIO is completely useless without a Template yet we seem as a group too focused on releasing PIO without regard for Templates. This IMO must change. 90% of users will never touch the code of a template and only 1% will actually create a template. These guesses come from list questions.

Templates First

PredictionIO is completely useless without a Template yet we seem as a group too focused on releasing PIO without regard for Templates. This IMO must change. 90% of users will never touch the code of a template and only 1% will actually create a template. These guesses come from list questions.

Re: installing environment (stops when compiling "compiler-interface" for Scala)

Memory depends on your data and the engine you are using. Spark puts all data into memory across the Spark cluster so if that is one machine, 4g will not allow more than toy or example data. Remember that PIO and Machine Learning in general works best with big data. BTW my laptop has 16g and

Upgrading to PredictionIO 0.12.0

PIO-0.12.0 by default, compiles and runs expecting ES5. If you are upgrading (not installing from clean) you will have an issue because ES1 indexes are not upgradable in any simple way. The simplest way to upgrade to pio-0.12.0 and ES5 is to do `pio export` to backup BEFORE upgrading—so export

Upgrading to PredictionIO 0.12.0

PIO-0.12.0 by default, compiles and runs expecting ES5. If you are upgrading (not installing from clean) you will have an issue because ES1 indexes are not upgradable in any simple way. The simplest way to upgrade to pio-0.12.0 and ES5 is to do `pio export` to backup BEFORE upgrading—so export

Re: PredictionIO Universal Recommender user rating

Yes, this is a very important point. We have found that the % of video viewed is indeed a very important factor but rather than sending some fraction to indicate the length viewed we have taken the approach before to determine the % that indicates the user liked the video. This we do by

Re: Universal Recommender and PredictionIO 0.12.0 incompatibility

s.h...@salesforce.com> wrote: Hi Pat, On 4 October 2017 at 22:04, Pat Ferrel <p...@actionml.com <mailto:p...@actionml.com>> wrote: It looks like PIO 0.12.0 will require a code change in the UR. PIO changed ES1 support drastically when ES5 support was added and broke the UR code.

Re: [ERROR] Timeout when read recent events

When you query for all users in batch, the system is easily overloaded. This is the worst case query situation where no caching applies (for instance). 1) run batch queries at low input load time, because you are competing with input for access to HBase 2) throttle your query speed and/or

[jira] [Created] (MAHOUT-2023) Drivers broken, scopt classes not found

Pat Ferrel created MAHOUT-2023: -- Summary: Drivers broken, scopt classes not found Key: MAHOUT-2023 URL: https://issues.apache.org/jira/browse/MAHOUT-2023 Project: Mahout Issue Type: Bug

Re: [ERROR] [TaskSetManager] Task 2.0 in stage 10.0 had a not serializable result

What version of Scala. Spark, PIO, and UR are you using? On Oct 4, 2017, at 6:10 AM, Noelia Osés Fernández wrote: Hi all, I'm still trying to create a very simple app to learn to use PredictionIO and still having trouble. I have done pio build no problem. But when I do

Re: Running Mahout on a Spark cluster

<trevor.d.gr...@gmail.com> wrote: The spark is included via maven classifier- the sbt line should be libraryDependencies += "org.apache.mahout" % "mahout-spark_2.11" % "0.13.1-SNAPSHOT" classifier "spark_2.1" On Tue, Oct 3, 2017 at 2:55 PM, Pat

[jira] [Commented] (MAHOUT-2020) Maven repo structure compatibility with SBT

[ https://issues.apache.org/jira/browse/MAHOUT-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190340#comment-16190340 ] Pat Ferrel commented on MAHOUT-2020: This may be a non-issue. Trevor said in email: {quote}The spark

[jira] [Commented] (MAHOUT-2019) SparseRowMatrix assign ops user for loops instead of iterateNonZero and so can be optimized

[ https://issues.apache.org/jira/browse/MAHOUT-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190338#comment-16190338 ] Pat Ferrel commented on MAHOUT-2019: This may be a non-issue: Trevor said in email: {quote

[jira] [Updated] (MAHOUT-2019) SparseRowMatrix assign ops user for loops instead of iterateNonZero and so can be optimized

[ https://issues.apache.org/jira/browse/MAHOUT-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pat Ferrel updated MAHOUT-2019: --- Priority: Major (was: Minor) > SparseRowMatrix assign ops user for loops instead of iterateNonZ

[jira] [Updated] (MAHOUT-2019) SparseRowMatrix assign ops user for loops instead of iterateNonZero and so can be optimized

[ https://issues.apache.org/jira/browse/MAHOUT-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pat Ferrel updated MAHOUT-2019: --- Priority: Minor (was: Major) > SparseRowMatrix assign ops user for loops instead of iterateNonZ

[jira] [Created] (MAHOUT-2020) Maven repo structure compatibility with SBT

Pat Ferrel created MAHOUT-2020: -- Summary: Maven repo structure compatibility with SBT Key: MAHOUT-2020 URL: https://issues.apache.org/jira/browse/MAHOUT-2020 Project: Mahout Issue Type: Bug

Re: Running Mahout on a Spark cluster

Actually if you require scala 2.11 and spark 2.1 you have to use the current master (o.13.0 does not support these) and also can’t use sbt, unless you have some trick I haven’t discovered. On Oct 3, 2017, at 12:55 PM, Pat Ferrel <p...@occamsmachete.com> wrote: I’m the aforementioned p

Re: Running Mahout on a Spark cluster

I’m the aforementioned pferrel @Hoa, thanks for that reference, I forgot I had that example. First don’t use the Hadoop part of Mahout, it is not supported and will be deprecated. The Spark version of cooccurrence will be supported. You find it in the SimilarityAnalysis object. If you go back

[jira] [Created] (MAHOUT-2019) SparseRowMatrix assign ops user for loops instead of iterateNonZero and so can be optimized

Pat Ferrel created MAHOUT-2019: -- Summary: SparseRowMatrix assign ops user for loops instead of iterateNonZero and so can be optimized Key: MAHOUT-2019 URL: https://issues.apache.org/jira/browse/MAHOUT-2019

Re: [DISCUSS] Proposed resolution to graduate the PredictionIO podling

oh, nm I found it. Pasted below, there were no dissenters to Donald’s detailed assessment. On Sep 29, 2017, at 3:27 PM, Pat Ferrel <p...@occamsmachete.com> wrote: Actually we did go over the maturity checklist ourselves. Donald, maybe you can forwards the thread here.

Re: [DISCUSS] Proposed resolution to graduate the PredictionIO podling

oh, nm I found it. Pasted below, there were no dissenters to Donald’s detailed assessment. On Sep 29, 2017, at 3:27 PM, Pat Ferrel <p...@occamsmachete.com> wrote: Actually we did go over the maturity checklist ourselves. Donald, maybe you can forwards the thread here.

Re: [DISCUSS] Proposed resolution to graduate the PredictionIO podling

Actually we did go over the maturity checklist ourselves. Donald, maybe you can forwards the thread here. On Sep 29, 2017, at 2:04 PM, Bertrand Delacretaz wrote: Hi John, On Fri, Sep 29, 2017 at 2:59 PM, John D. Ament wrote: > ...I

Re: [RESULT][VOTE] Resolution to create a TLP from graduating Incubator podling

gt;> * Donald Szeto <don...@apache.org> >> * Felipe Oliveira <fel...@apache.org> >> * James Taylor <jtay...@apache.org> >> * Justin Yip <yipjus...@apache.org> >> * Kenneth Chan <kenn...@apache.org> >>

Re: Eventserver API in an Engine?

fy baseline PredictionIO deployment, both conceptually & technically. My vision with this thread is to: Enable single-process, single network-listener PredictionIO app deployment (i.e. Queries & Events APIs in the same process.) Attempting to address some previous questions & sta

Re: How to training and deploy on different machine?

erver] gets a copy of the template from machine [TrainingServer] (only need to do this once) Then run `pio deploy` It is not a Spark driver or executor for training Write a cron job of `pio deploy` It is permanent. Thanks Brian On Wed, Sep 20, 2017 at 11:16 PM, Pat Ferrel <p...@occamsmach

Re: How to training and deploy on different machine?

Yes, this is the recommended config (Postgres is not, but later). Spark is only needed during training but the `pio train` process creates drives and executors in Spark. The driver will be the `pio train` machine so you must install pio on it. You should have 2 Spark machines at least because

Re: Unable to connect to all storage backends successfully

meaning is “firstcluster” the cluster name in your Elasticsearch configuration? On Sep 19, 2017, at 8:54 PM, Vaghawan Ojha wrote: I think the problem is with Elasticsearch, are you sure the cluster exists in elasticsearch configuration? On Wed, Sep 20, 2017 at 8:17

[jira] [Closed] (PIO-32) create component upgrade releases

[ https://issues.apache.org/jira/browse/PIO-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pat Ferrel closed PIO-32. - Resolution: Fixed > create component upgrade releases > - > >

Re: [VOTE] Apache PredictionIO (incubating) 0.12.0 Release (RC2)

The last release was hung up by the IPMC regarding content licensing issues and libraries used by the doc site, which we promised to address in this release. Have these been resolved, don’t recall the specifics? It would be great to fly through the IPMC vote without issue. On Sep 14, 2017, at

Re: Universal Recommender - search by subtext/Unicode

dation that contains some text in the “subtext”? Not sure what you mean by “subtext” - I mean if the I look for an item like "ifone" instead of "iphone", ie. make an error in the spelling, would it still work ?? On Wed, Sep 13, 2017 at 1:37 PM, Pat Ferrel <p...

Re: Universal Recommender : seasonality of product

This is done with blacklisting. The default config blacklists all items in the training data that the users has taken the primary event on. So if your primary event is “buy” then once a user has bought a particular table they will not be recommended that table again until the “buy” event ages

Re: Universal Recommender - search by subtext/Unicode

1) Yes, of course. Use UTF-8 encoding. 2) I don’t understand this question. The UR is not a search engine, what kind of recommendation are looking for? The best recommendation from a list of items? The best recommendation that contains some text in the “subtext”? Not sure what you mean by

Re: Graduation to TLP

> wants to remove themselves. It may be good. In fact, most of committers carry out a task with limited time. Anyway I hope the project will progress in a good direction. 2017-09-06 3:43 GMT+09:00 Pat Ferrel <p...@occamsmachete.com>: > I personally don’t see much benefit in r

Re: Validate the built model

We do cross-validation tests to see how well the model predicts actual behavior. As to the best data mix, cross-validation works with any engine tuning or data input. Typically this requires re-traiing between test runs so make sure you use exatly the same training/test split. If you want to

Re: Train a model without stopping

The UR does this automatically. Once deployed you never have to deploy a second time. When a new `pio train` happens the new model is hot-swapped to replace the old, which is then erased, so there is no re-deploy and no downtime. Yes, it uses Elasticsearch aliases but most other Templates do

Re: Graduation to TLP

implicitly CS20, CS30, and CS40 from > main ASF doc) > - IN10, IN20 > > Let me know what you think. > > On Fri, Sep 1, 2017 at 10:32 AM, Pat Ferrel <p...@occamsmachete.com> wrote: > >> The Chair, PMC, and Committers may be different after graduation. >> PM

Re: Recommender for social media

Actually IMO it is not more complex, it is just far better documented and more flexible. If you don’t need the features it is just as simple as the Apache PIO Templates. I could argue the UR is simpler since you don’t need to $set every item and user, they are determined automatically from the

Re: Securing Event Server on Heroku?

TLS/SSL is required along with authentication of the HTTPS requests. I’m not familiar with Heroku but the Proxy must authenticate the incoming connections. Nginx has basic auth and is a fast proxy, for instance. A cheap, dirty, and not recommended unless it is your only option, is to set your

Re: Error: Could not find or load main class org.apache.predictionio.tools.console.Console

/passion8/2769147c5352df4dad610100226f3b66> system : Ubuntu 16.04.3 x64 -- Paritosh Piplewar Sent with Airmail On 30 August 2017 at 5:22:43 PM, Pat Ferrel (p...@occamsmachete.com <mailto:p...@occamsmachete.com>) wrote: > Can you explain how you installed and what your problem is? The link below &

Re: Graduation to TLP

se. +1 for graduation. I was away for a while due to family issues, but I'd be happy to volunteer for release management. On Wed, Aug 30, 2017 at 4:30 PM, Pat Ferrel <p...@occamsmachete.com> wrote: > Along with the link Donald gave, check this out: > http://community.apache.org/apache-way/apach

Re: Graduation to TLP

. Would be great to understand what the VP vs Member responsibilities look like. Let's graduate. +1 *Mars On Wed, Aug 30, 2017 at 15:21 Pat Ferrel <p...@occamsmachete.com> wrote: > I have had several people tell me they want to wait until PIO is not > incubating before using it

Re: Graduation to TLP

I have had several people tell me they want to wait until PIO is not incubating before using it. This even after explaining that “incubating” has more to do with getting into the Apache Way of doing things and has no direct link to quality or community. I can only conclude from this that

Re: spark-itemsimilarity scalability / Spark parallelism issues (SimilarityAnalysis.cooccurrencesIDSs)

Matt, I’m interested in following up on this. If you can’t do a PR, can you describe what you did a bit more? On Aug 21, 2017, at 12:05 PM, Pat Ferrel <p...@occamsmachete.com> wrote: Matt I’ll create a feature branch of Mahout in my git repo for simplicity (we are in code freeze for

Re: Error: Could not find or load main class org.apache.predictionio.tools.console.Console

Can you explain how you installed and what your problem is? The link below doesn’t contain much information. On Aug 29, 2017, at 9:02 PM, Paritosh Piplewar wrote: Yes I'm running pio from inside bin directory. Sent from my iPhone On 30-Aug-2017, at 3:41 AM, Mars

Re: sbt.ResolveException: unresolved dependency: org.apache.predictionio#pio-build;0.11.0-incubrating

You template is linking to org.apache.predictionio#pio-build;0.10.0-incubrating, what do you have installed? org.apache.predictionio#pio-build;0.11.0-incubrating? Looks like you have to change your templates build.sbt to link to the artifact you have built. On Aug 22, 2017, at 3:52 AM,

Re: spark-itemsimilarity scalability / Spark parallelism issues (SimilarityAnalysis.cooccurrencesIDSs)

_________ From: Pat Ferrel <p...@occamsmachete.com> Sent: Monday, August 21, 2017 2:26:58 PM To: user@mahout.apache.org Subject: Re: spark-itemsimilarity scalability / Spark parallelism issues (SimilarityAnalysis.cooccurrencesIDSs) That looks like ancient code from the old mapreduce days.

<    1   2   3   4   5   6   7   8   9   10   >