[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-01 Thread chanlee514
Github user chanlee514 commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 Right now, the commands throw java.io.FileNotFoundException regarding engine.json. May be better if we throw an error message saying the commands should be executed inside engine d

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-01 Thread pferrel
Github user pferrel commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 Ideally `pio build` and `pio train` should run anywhere by supplying any params needed. @chanlee514 metadata still is needed in a service like Elasticsearch so every place th

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-01 Thread chanlee514
Github user chanlee514 commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 @pferrel: I agree, and that would be possible with a general metadata registry server which would be the next step. The reason I started by removing manifest.json is because the pr

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-01 Thread dszeto
Github user dszeto commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 @pferrel For `pio build` we probably can enhance it such that it can take a working directory for compilation to run. If we look at some other build tools, they also rely on having cer

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-02 Thread pferrel
Github user pferrel commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 ok, this looks like a good step @dszeto we would not do `pio build` we would do `sbt build` in the directory that has the template and this is fine IMO. The confusion users ha

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-02 Thread pferrel
Github user pferrel commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 One more question. After push how do we run the PredictionServer or train on multiple machines? In the past this required copying the manifest.json and making sure binaries are in the

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-02 Thread pferrel
Github user pferrel commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 @dszeto asked: "Do you currently have any production deployments that rely heavily on engine IDs and versions? That would be a bigger immediate concern regarding this change."

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-05 Thread dszeto
Github user dszeto commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 @chanlee514 @pferrel I have moved the discussion to dev@. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-12 Thread pferrel
Github user pferrel commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 too much noise in @dev, I have a simple question. If I train on one machine, how do I deploy on another. Currently I copy the manifest--well actually the entire directory--fr

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-13 Thread pferrel
Github user pferrel commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 @chanlee514 said: If the absolute filepath is the same in the machines (since SHA-1 hash of filepath is used as engineVersion), there would be no changes in the above

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-13 Thread pferrel
Github user pferrel commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 The file path is required to be the same on any 2 machines and is explicitly in the metadata. If you have the deploy machine dir in a different path than the train machine dir (actual

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2017-01-12 Thread jimlyndon
Github user jimlyndon commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 +1 Actually ran into this issue a couple of months ago. Luckily myself and our dev ops guy immediately realized the issue (abs paths required) and fixed our process: We bu

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2017-01-15 Thread pferrel
Github user pferrel commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 Actually the manifest may be gone but the requirement of the same absolute path is maintained, I believe. A hash of the path to the template code is used to create the id and is part

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2017-01-17 Thread chanlee514
Github user chanlee514 commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 @pferrel: Yes the hash of engine directory path is used as engineVersion (which kind of acts as the id). However, I'm working on making updates to this PR so that users ca

[GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2017-01-17 Thread pferrel
Github user pferrel commented on the issue: https://github.com/apache/incubator-predictionio/pull/328 Cool, that's awesome. This news means the scope has changed, which is great but brings up the visibility of the id. Could I suggest that the user-visible engine-instance-id b

Re: [GitHub] incubator-predictionio issue #328: [PIO-47] Eliminate enginemanifest for sta...

2016-12-13 Thread Chan Lee
If the absolute filepath is the same in the machines (since SHA-1 hash of filepath is used as engineVersion), there would be no changes in the above workflow. You can just copy the engine directory and deploy. If not, you can also provide engine-id, engine-version as CLI arguments. There's some ad