At this point, I'm just searching the internet for "Not a valid command: assemblyPackageDependency" errors, which I image you are too.
Does this error occur locally when you `pio build` this engine? Are there any diffs in your local code? How are you running locally? Are you using the buildpack's "local dev" setup? On Fri, Mar 9, 2018 at 6:48 PM, Shane Johnson <sh...@liftiq.com> wrote: > Mars, to test what may be happening I just reverted to the original > buildpack (https://github.com/heroku/predictionio-buildpack.git) and > removed the variable PREDICTIONIO_DIST_URL and I am still getting the > same error. I don't know where I would have introduced this. Have you seen > it before, perhaps this is what is causing the custom dist to not be used? > I'm guessing the build failed before and I was overlooking that. Now I need > to find out why the build is failing:) > > Quietly logging. (Set `PIO_VERBOSE=true` for detailed build log.) > > [INFO] [Engine$] Using command > '/tmp/build_28c7923a4f807cd0eaa37b606989fe11/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt' > at > /tmp/build_28c7923a4f807cd0eaa37b606989fe11/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e > to build. > > [INFO] [Engine$] If the path above is incorrect, this process will > fail. > > [INFO] [Engine$] Uber JAR disabled. Making sure > lib/pio-assembly-0.12.0-incubating.jar is absent. > > [INFO] [Engine$] Going to run: > /tmp/build_28c7923a4f807cd0eaa37b606989fe11/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt > package assemblyPackageDependency in > /tmp/build_28c7923a4f807cd0eaa37b606989fe11/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e > > [ERROR] [Engine$] [error] Not a valid command: > assemblyPackageDependency > > [ERROR] [Engine$] [error] Not a valid key: assemblyPackageDependency > (similar: sbtDependency) > > [ERROR] [Engine$] [error] assemblyPackageDependency > > [ERROR] [Engine$] [error] ^ > > [ERROR] [Engine$] Return code of build command: > /tmp/build_28c7923a4f807cd0eaa37b606989fe11/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt > package assemblyPackageDependency is 1. Aborting. > > ! Push rejected, failed to compile PredictionIO app. > > ! Push failed > > > > > *Shane Johnson | LIFT IQ* > *Founder | CEO* > > *www.liftiq.com <http://www.liftiq.com/>* or *sh...@liftiq.com > <sh...@liftiq.com>* > mobile: (801) 360-3350 > LinkedIn <https://www.linkedin.com/in/shanewjohnson/> | Twitter > <https://twitter.com/SWaldenJ> | Facebook > <https://www.facebook.com/shane.johnson.71653> > > > > On Fri, Mar 9, 2018 at 11:35 AM, Shane Johnson <sh...@liftiq.com> wrote: > >> It looks like the URL is the correct URL of the custom PredictionIO dist. >> It looks like there is another error that might be occurring here when I >> went to deploy. >> >> -----> JVM Common app detected >> >> -----> Installing JDK 1.8... done >> >> -----> PredictionIO app detected >> >> -----> Install core components >> >> *+ PredictionIO >> (https://s3-us-west-1.amazonaws.com/predictionio/0.12.0-incubating/apache-predictionio-0.12.0-incubating-bin.tar.gz >> >> <https://s3-us-west-1.amazonaws.com/predictionio/0.12.0-incubating/apache-predictionio-0.12.0-incubating-bin.tar.gz>)* >> >> + Spark (spark-2.1.1-bin-hadoop2.7) >> >> -----> Install supplemental components >> >> + PostgreSQL (JDBC) >> >> + S3 HDFS (AWS SDK) >> >> + S3 HDFS (Hadoop-AWS) >> >> Writing default 'core-site.xml.erb' >> >> + local Maven repo from buildpack (contents) >> >> -----> Configure PredictionIO >> >> Writing default 'pio-env.sh' >> >> Writing default 'spark-defaults.conf.erb' >> >> + Maven repo from buildpack (build.sbt entry) >> >> Set-up environment via '.profile.d/' scripts >> >> -----> Install JVM (heroku/jvm-common) >> >> -----> PredictionIO engine >> >> Quietly logging. (Set `PIO_VERBOSE=true` for detailed build log.) >> >> [INFO] [Engine$] Using command >> '/tmp/build_a27b71b6803e5ee2db3367fa69981626/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt' >> at >> /tmp/build_a27b71b6803e5ee2db3367fa69981626/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e >> to build. >> >> [INFO] [Engine$] If the path above is incorrect, this process will >> fail. >> >> [INFO] [Engine$] Uber JAR disabled. Making sure >> lib/pio-assembly-0.12.0-incubating.jar is absent. >> >> [INFO] [Engine$] Going to run: >> /tmp/build_a27b71b6803e5ee2db3367fa69981626/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt >> package assemblyPackageDependency in >> /tmp/build_a27b71b6803e5ee2db3367fa69981626/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e >> >> [ERROR] [Engine$] [error] Not a valid command: >> assemblyPackageDependency >> >> [ERROR] [Engine$] [error] Not a valid key: assemblyPackageDependency >> (similar: sbtDependency) >> >> [ERROR] [Engine$] [error] assemblyPackageDependency >> >> [ERROR] [Engine$] [error] ^ >> >> [ERROR] [Engine$] Return code of build command: >> /tmp/build_a27b71b6803e5ee2db3367fa69981626/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt >> package assemblyPackageDependency is 1. Aborting. >> >> ! Push rejected, failed to compile PredictionIO app. >> >> ! Push failed >> >> >> >> >> *Shane Johnson | LIFT IQ* >> *Founder | CEO* >> >> *www.liftiq.com <http://www.liftiq.com/>* or *sh...@liftiq.com >> <sh...@liftiq.com>* >> mobile: (801) 360-3350 >> LinkedIn <https://www.linkedin.com/in/shanewjohnson/> | Twitter >> <https://twitter.com/SWaldenJ> | Facebook >> <https://www.facebook.com/shane.johnson.71653> >> >> >> >> On Fri, Mar 9, 2018 at 11:17 AM, Mars Hall <mars.h...@salesforce.com> >> wrote: >> >>> I'm lost as to how such direct manipulation of CLASSPATH is not >>> appearing in the logged spark-submit command. >>> >>> What could cause this!? >>> >>> I just pushed a version of the buildpack which should help debug. >>> Assuming only a single buildpack is assigned to the app, here's how to set >>> it: >>> >>> heroku buildpacks:set https://github.com/heroku/pred >>> ictionio-buildpack#debug-custom-dist >>> >>> Then redeploy the engine an check the build log for the line: >>> >>> + PredictionIO ($URL) >>> >>> Please confirm that it is the URL of your custom PredictionIO dist. >>> >>> On Fri, Mar 9, 2018 at 2:47 PM, Shane Johnson <sh...@liftiq.com> wrote: >>> >>>> Thanks Donald and Mars, >>>> >>>> I created a new distribution ( >>>> <https://s3-us-west-1.amazonaws.com/predictionio/0.12.0-incubating/apache-predictionio-0.12.0-incubating-bin.tar.gz> >>>> https://s3-us-west-1.amazonaws.com/predictionio/0.12.0-incu >>>> bating/apache-predictionio-0.12.0-incubating-bin.tar.gz) with the >>>> added CLASSPATH code and pointed to the distribution with >>>> the PREDICTIONIO_DIST_URL variable within the engine app in Heroku. >>>> >>>> CLASSPATH="/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar >>>> :$CLASSPATH" >>>> echo "$CLASSPATH" >>>> >>>> It didn't seem to force the aws-java-sdk to load first as I reviewed >>>> the release logs. Should the aws-java-sdk.jar show up as the first file >>>> within the --jars section when this is added CLASSPATH="/app/Predicti >>>> onIO-dist/lib/spark/aws-java-sdk.jar:$CLASSPATH". >>>> >>>> I'm still getting the NoSuchMethodError when the *aws-java-sdk.jar* loads >>>> after the *pio-data-s3-assembly-0.12.0-incubating.jar**. *Do you have >>>> other suggestions to try? I was also testing locally to change the order of >>>> the --jars but changes to the compute-classpath.sh didn't seem to change >>>> the order of the jars in the logs. >>>> >>>> Running train on release… >>>> >>>> Picked up JAVA_TOOL_OPTIONS: -Xmx12g -Dfile.encoding=UTF-8 >>>> >>>> [INFO] [Runner$] Submission command: >>>> /app/PredictionIO-dist/vendors/spark-hadoop/bin/spark-submit >>>> --driver-memory 13g --class >>>> org.apache.predictionio.workflow.CreateWorkflow --jars >>>> file:/app/PredictionIO-dist/lib/postgresql_jdbc.jar,file:/app/target/scala-2.11/template-scala-parallel-liftscoring-assembly-0.1-SNAPSHOT-deps.jar,file:/app/target/scala-2.11/template-scala-parallel-liftscoring_2.11-0.1-SNAPSHOT.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-hdfs-assembly-0.12.0-incubating.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-localfs-assembly-0.12.0-incubating.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-elasticsearch-assembly-0.12.0-incubating.jar,file:/app/PredictionIO-dist/lib/spark/hadoop-aws.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-hbase-assembly-0.12.0-incubating.jar,*file:/app/PredictionIO-dist/lib/spark/pio-data-s3-assembly-0.12.0-incubating.jar*,file:/app/PredictionIO-dist/lib/spark/pio-data-jdbc-assembly-0.12.0-incubating.jar,*file:/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar* >>>> --files >>>> file:/app/PredictionIO-dist/conf/log4j.properties,file:/app/PredictionIO-dist/conf/core-site.xml >>>> --driver-class-path >>>> /app/PredictionIO-dist/conf:/app/PredictionIO-dist/conf:/app/PredictionIO-dist/lib/postgresql_jdbc.jar:/app/PredictionIO-dist/conf >>>> --driver-java-options -Dpio.log.dir=/app >>>> file:/app/PredictionIO-dist/lib/pio-assembly-0.12.0-incubating.jar >>>> --engine-id org.template.liftscoring.LiftScoringEngine --engine-version >>>> 0c35eebf403cf91fe77a64921d76aa1ca6411d20 --engine-variant >>>> file:/app/engine.json --verbosity 0 --json-extractor Both --env >>>> >>>> >>>> Error: >>>> >>>> Exception in thread "main" java.lang.NoSuchMethodError: >>>> com.amazonaws.services.s3.transfer.TransferManager.<init>(Lcom/amazonaws/services/s3/AmazonS3;Ljava/util/concurrent/ThreadPoolExecutor;)V >>>> >>>> at >>>> org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:287) >>>> >>>> at >>>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669) >>>> >>>> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94) >>>> >>>> >>>> >>>> >>>> >>>> *Shane Johnson | LIFT IQ* >>>> *Founder | CEO* >>>> >>>> *www.liftiq.com <http://www.liftiq.com/>* or *sh...@liftiq.com >>>> <sh...@liftiq.com>* >>>> mobile: (801) 360-3350 >>>> LinkedIn <https://www.linkedin.com/in/shanewjohnson/> | Twitter >>>> <https://twitter.com/SWaldenJ> | Facebook >>>> <https://www.facebook.com/shane.johnson.71653> >>>> >>>> >>>> >>>> On Wed, Mar 7, 2018 at 1:01 PM, Mars Hall <mars.h...@salesforce.com> >>>> wrote: >>>> >>>>> Shane, >>>>> >>>>> On Wed, Mar 7, 2018 at 4:49 AM, Shane Johnson <sh...@liftiq.com> >>>>> wrote: >>>>> >>>>>> >>>>>> Re: adding a line to ensure a jar is loaded first. Is this what you >>>>>> are referring to...(line at the bottom in red)? >>>>>> >>>>> >>>>> >>>>> I believe the code would need to look like this to effect the output >>>>> classpath as intended: >>>>> >>>>> >>>>>> CLASSPATH="/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar >>>>>> :$CLASSPATH" >>>>>> echo "$CLASSPATH" >>>>>> >>>>> >>>>> >>>>> aws-java-sdk.jar is already in the CLASSPATH though, So, the script >>>>> will need to be skip or remove it first. >>>>> >>>>> -- >>>>> *Mars Hall >>>>> 415-818-7039 <(415)%20818-7039> >>>>> Customer Facing Architect >>>>> Salesforce Platform / Heroku >>>>> San Francisco, California >>>>> >>>> >>>> >>> >>> >>> -- >>> *Mars Hall >>> 415-818-7039 <(415)%20818-7039> >>> Customer Facing Architect >>> Salesforce Platform / Heroku >>> San Francisco, California >>> >> >> > -- *Mars Hall 415-818-7039 Customer Facing Architect Salesforce Platform / Heroku San Francisco, California
