VO, i was just replying to ur response, here u are... with quick response on my post. thanks to that. as far as error, i know that where did i commit mistake, but i did changed the appName as it mentioned in official article, so could u please check DataSource.scala file of mine which i hv mentioned in my first mail.
Thanks, -DAN On Thu, Jul 27, 2017 at 2:31 PM, Vaghawan Ojha <[email protected]> wrote: > As per the error "[ERROR] [Common$] Invalid app name null" you haven't > configured the engine.jso properly it seems. > > On Thu, Jul 27, 2017 at 2:44 PM, Darshan A.N. <[email protected]> > wrote: > >> hi Tom Chan, >> i did tried with directory u told me to do. >> >> but got an error, i know where did that error emerging from, but as for >> the document i myself edited the appName to appId, >> >> now the error is like this, >> >> >> >> >> darshan@darshu:~/PredictionIO/tapster-episode-similar$ pio train >> SLF4J: Class path contains multiple SLF4J bindings. >> SLF4J: Found binding in [jar:file:/home/darshan/Predic >> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubatin >> g.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: Found binding in [jar:file:/home/darshan/Predic >> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/ >> impl/StaticLoggerBinder.class] >> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >> explanation. >> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >> [INFO] [Runner$] Submission command: /home/darshan/PredictionIO/ven >> dors//spark-1.6.3/bin/spark-submit --class >> org.apache.predictionio.workflow.CreateWorkflow >> --jars file:/home/darshan/PredictionIO/lib/mysql-connector-java-5. >> 1.37.jar,file:/home/darshan/PredictionIO/tapster-episode-sim >> ilar/target/scala-2.10/template-scala-parallel-similarproduc >> t_2.10-0.1-SNAPSHOT.jar,file:/home/darshan/PredictionIO/ >> tapster-episode-similar/target/scala-2.10/template- >> scala-parallel-similarproduct-assembly-0.1-SNAPSHOT-deps. >> jar,file:/home/darshan/PredictionIO/lib/spark/pio- >> data-hbase-assembly-0.11.0-incubating.jar,file:/home/ >> darshan/PredictionIO/lib/spark/pio-data-hdfs-assembly-0.11. >> 0-incubating.jar,file:/home/darshan/PredictionIO/lib/spark >> /pio-data-jdbc-assembly-0.11.0-incubating.jar,file:/home/ >> darshan/PredictionIO/lib/spark/pio-data-elasticsearch1-assem >> bly-0.11.0-incubating.jar,file:/home/darshan/PredictionI >> O/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar --files >> file:/home/darshan/PredictionIO/conf/log4j.properties >> --driver-class-path /home/darshan/PredictionIO/con >> f:/home/darshan/PredictionIO/lib/mysql-connector-java-5.1.37.jar >> --driver-java-options -Dpio.log.dir=/home/darshan >> file:/home/darshan/PredictionIO/lib/pio-assembly-0.11.0-incubating.jar >> --engine-id org.example.similarproduct.SimilarProductEngine >> --engine-version 7ff66e8607f7d36b79cb9e9f3b97b53287e553f7 >> --engine-variant >> file:/home/darshan/PredictionIO/tapster-episode-similar/engine.json >> --verbosity 0 --json-extractor Both --env PIO_ENV_LOADED=1,PIO_STORAGE_S >> OURCES_MYSQL_PASSWORD=pio,PIO_STORAGE_REPOSITORIES_METADATA_ >> NAME=pio_meta,PIO_FS_BASEDIR=/home/darshan/.pio_store,PIO_ >> STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost:3306/ >> pio,PIO_HOME=/home/darshan/PredictionIO,PIO_FS_ >> ENGINESDIR=/home/darshan/.pio_store/engines,PIO_STORAGE_SOUR >> CES_MYSQL_TYPE=jdbc,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE >> =MYSQL,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=MYSQL,PIO_ >> STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_MYSQL_ >> USERNAME=pio,PIO_FS_TMPDIR=/home/darshan/.pio_store/tmp, >> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_ >> STORAGE_REPOSITORIES_EVENTDATA_SOURCE=MYSQL,PIO_CONF_DIR=/home/darshan/ >> PredictionIO/conf >> [INFO] [Engine] Extracting datasource params... >> [INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be >> used. >> [INFO] [Engine] Datasource params: (,DataSourceParams(null)) >> [INFO] [Engine] Extracting preparator params... >> [INFO] [Engine] Preparator params: (,Empty) >> [INFO] [Engine] Extracting serving params... >> [INFO] [Engine] Serving params: (,Empty) >> [WARN] [Utils] Your hostname, darshu resolves to a loopback address: >> 127.0.0.1; using 192.168.2.103 instead (on interface wlx001ea6631cc7) >> [WARN] [Utils] Set SPARK_LOCAL_IP if you need to bind to another address >> [INFO] [Remoting] Starting remoting >> [INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp:// >> [email protected]:45309] >> [INFO] [Engine$] EngineWorkflow.train >> [INFO] [Engine$] DataSource: org.example.similarproduct.Dat >> aSource@168b4cb0 >> [INFO] [Engine$] Preparator: org.example.similarproduct.Pre >> parator@45545e7a >> [INFO] [Engine$] AlgorithmList: List(org.example.similarproduc >> t.ALSAlgorithm@23cbbd07) >> [INFO] [Engine$] Data sanity check is on. >> [ERROR] [Common$] Invalid app name null >> Exception in thread "main" java.lang.IllegalArgumentException: Invalid >> app name null >> at org.apache.predictionio.data.store.Common$$anonfun$appNameTo >> Id$2.apply(Common.scala:50) >> at org.apache.predictionio.data.store.Common$$anonfun$appNameTo >> Id$2.apply(Common.scala:48) >> at scala.Option.getOrElse(Option.scala:120) >> at org.apache.predictionio.data.store.Common$.appNameToId(Commo >> n.scala:48) >> at org.apache.predictionio.data.store.PEventStore$.aggregatePro >> perties(PEventStore.scala:108) >> at org.example.similarproduct.DataSource.readTraining(DataSourc >> e.scala:31) >> at org.example.similarproduct.DataSource.readTraining(DataSourc >> e.scala:18) >> at org.apache.predictionio.controller.PDataSource.readTrainingB >> ase(PDataSource.scala:40) >> at org.apache.predictionio.controller.Engine$.train(Engine.scala:644) >> at org.apache.predictionio.controller.Engine.train(Engine.scala:177) >> at org.apache.predictionio.workflow.CoreWorkflow$.runTrain( >> CoreWorkflow.scala:67) >> at org.apache.predictionio.workflow.CreateWorkflow$.main(Create >> Workflow.scala:250) >> at org.apache.predictionio.workflow.CreateWorkflow.main(CreateW >> orkflow.scala) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >> ssorImpl.java:62) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >> thodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy >> $SparkSubmit$$runMain(SparkSubmit.scala:731) >> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit >> .scala:181) >> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) >> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) >> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) >> >> i m using MySQL as my repository. i followed the article from official >> website only, still i m not able to execute the pio build or pio train. My >> src/main/scala DataSource.scala program is still as it is in previously >> mentioned. >> >> Thanks, >> -DAN >> >> On Wed, Jul 26, 2017 at 3:22 PM, Tom Chan <[email protected]> wrote: >> >>> darshan@darshu:~/PredictionIO/tapster-episode-similar/src/main/scala$ >>> pio train >>> >>> Can you try the command from the tapster-episode-similar directory? >>> >>> Tom >>> >>> On Jul 26, 2017 2:45 AM, "Darshan A.N." <[email protected]> wrote: >>> >>>> hi team, >>>> i am trying to install demo tapster. i followed >>>> http://predictionio.incubator.apache.org/demo/tapster/. It took >>>> more than a week to install predictionio. >>>> now that i m installed the PIO, its throwing me error while running the >>>> $pio build command. the error goes like this: >>>> darshan@darshu:~/PredictionIO/tapster-episode-similar$ pio build >>>> SLF4J: Class path contains multiple SLF4J bindings. >>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic >>>> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.ja >>>> r!/org/slf4j/impl/StaticLoggerBinder.class] >>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic >>>> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/imp >>>> l/StaticLoggerBinder.class] >>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >>>> explanation. >>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >>>> [INFO] [Engine$] Using command '/home/darshan/PredictionIO/sbt/sbt' at >>>> /home/darshan/PredictionIO/tapster-episode-similar to build. >>>> [INFO] [Engine$] If the path above is incorrect, this process will fail. >>>> [INFO] [Engine$] Uber JAR disabled. Making sure >>>> lib/pio-assembly-0.11.0-incubating.jar is absent. >>>> [INFO] [Engine$] Going to run: /home/darshan/PredictionIO/sbt/sbt >>>> package assemblyPackageDependency in /home/darshan/PredictionIO/tap >>>> ster-episode-similar >>>> [ERROR] [Engine$] [error] /home/darshan/PredictionIO/tap >>>> ster-episode-similar/src/main/scala/DataSource.scala:63: not found: >>>> value eventsDb >>>> [ERROR] [Engine$] [error] val viewEventsRDD: RDD[ViewEvent] = >>>> eventsDb.find( >>>> [ERROR] [Engine$] [error] ^ >>>> [ERROR] [Engine$] [error] one error found >>>> [ERROR] [Engine$] [error] (compile:compileIncremental) Compilation >>>> failed >>>> [ERROR] [Engine$] [error] Total time: 5 s, completed 26 Jul, 2017 >>>> 1:56:50 AM >>>> [ERROR] [Engine$] Return code of build command: >>>> /home/darshan/PredictionIO/sbt/sbt package assemblyPackageDependency >>>> is 1. Aborting. >>>> [INFO] [Engine$] Looking for an engine... >>>> [INFO] [Engine$] Found template-scala-parallel-simila >>>> rproduct_2.10-0.1-SNAPSHOT.jar >>>> [INFO] [Engine$] Found template-scala-parallel-simila >>>> rproduct-assembly-0.1-SNAPSHOT-deps.jar >>>> [INFO] [Engine$] Build finished successfully. >>>> [INFO] [Pio$] Your engine is ready for training. >>>> >>>> >>>> and the file DataSource.scala, >>>> >>>> >>>> package org.example.similarproduct >>>> >>>> import org.apache.predictionio.controller.PDataSource >>>> import org.apache.predictionio.controller.EmptyEvaluationInfo >>>> import org.apache.predictionio.controller.EmptyActualResult >>>> import org.apache.predictionio.controller.Params >>>> import org.apache.predictionio.data.storage.Event >>>> import org.apache.predictionio.data.store.PEventStore >>>> >>>> import org.apache.spark.SparkContext >>>> import org.apache.spark.SparkContext._ >>>> import org.apache.spark.rdd.RDD >>>> >>>> import grizzled.slf4j.Logger >>>> >>>> case class DataSourceParams(appName: String) extends Params >>>> >>>> class DataSource(val dsp: DataSourceParams) >>>> extends PDataSource[TrainingData, >>>> EmptyEvaluationInfo, Query, EmptyActualResult] { >>>> >>>> @transient lazy val logger = Logger[this.type] >>>> >>>> override >>>> def readTraining(sc: SparkContext): TrainingData = { >>>> >>>> // create a RDD of (entityID, User) >>>> val usersRDD: RDD[(String, User)] = PEventStore.aggregatePropertie >>>> s( >>>> appName = dsp.appName, >>>> entityType = "user" >>>> )(sc).map { case (entityId, properties) => >>>> val user = try { >>>> User() >>>> } catch { >>>> case e: Exception => { >>>> logger.error(s"Failed to get properties ${properties} of" + >>>> s" user ${entityId}. Exception: ${e}.") >>>> throw e >>>> } >>>> } >>>> (entityId, user) >>>> }.cache() >>>> >>>> // create a RDD of (entityID, Item) >>>> val itemsRDD: RDD[(String, Item)] = PEventStore.aggregatePropertie >>>> s( >>>> appName = dsp.appName, >>>> entityType = "item" >>>> )(sc).map { case (entityId, properties) => >>>> val item = try { >>>> // Assume categories is optional property of item. >>>> Item(categories = properties.getOpt[List[String] >>>> ]("categories")) >>>> } catch { >>>> case e: Exception => { >>>> logger.error(s"Failed to get properties ${properties} of" + >>>> s" item ${entityId}. Exception: ${e}.") >>>> throw e >>>> } >>>> } >>>> (entityId, item) >>>> }.cache() >>>> >>>> // get all "user" "view" "item" events >>>> val viewEventsRDD: RDD[ViewEvent] = eventsDb.find( >>>> appId = dsp.appId, >>>> entityType = Some("user"), >>>> eventNames = Some(List("like")), >>>> // targetEntityType is optional field of an event. >>>> targetEntityType = Some(Some("item")))(sc) >>>> // eventsDb.find() returns RDD[Event] >>>> .map { event => >>>> val viewEvent = try { >>>> event.event match { >>>> case "like" => ViewEvent( >>>> user = event.entityId, >>>> item = event.targetEntityId.get, >>>> t = event.eventTime.getMillis) >>>> case _ => throw new Exception(s"Unexpected event ${event} >>>> is read.") >>>> } >>>> } catch { >>>> case e: Exception => { >>>> logger.error(s"Cannot convert ${event} to ViewEvent." + >>>> s" Exception: ${e}.") >>>> throw e >>>> } >>>> } >>>> viewEvent >>>> }.cache() >>>> >>>> new TrainingData( >>>> users = usersRDD, >>>> items = itemsRDD, >>>> viewEvents = viewEventsRDD >>>> ) >>>> } >>>> } >>>> >>>> case class User() >>>> >>>> case class Item(categories: Option[List[String]]) >>>> >>>> case class ViewEvent(user: String, item: String, t: Long) >>>> >>>> class TrainingData( >>>> val users: RDD[(String, User)], >>>> val items: RDD[(String, Item)], >>>> val viewEvents: RDD[ViewEvent] >>>> ) extends Serializable { >>>> override def toString = { >>>> s"users: [${users.count()} (${users.take(2).toList}...)]" + >>>> s"items: [${items.count()} (${items.take(2).toList}...)]" + >>>> s"viewEvents: [${viewEvents.count()}] (${viewEvents.take(2).toList}. >>>> ..)" >>>> } >>>> } >>>> >>>> >>>> >>>> >>>> while training, it gives me following error, >>>> >>>> >>>> >>>> >>>> darshan@darshu:~/PredictionIO/tapster-episode-similar/src/main/scala$ >>>> pio train >>>> SLF4J: Class path contains multiple SLF4J bindings. >>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic >>>> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.ja >>>> r!/org/slf4j/impl/StaticLoggerBinder.class] >>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic >>>> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/imp >>>> l/StaticLoggerBinder.class] >>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >>>> explanation. >>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >>>> [WARN] [Template$] /home/darshan/PredictionIO/tap >>>> ster-episode-similar/src/main/scala/template.json does not exist. >>>> Template metadata will not be available. (This is safe to ignore if you are >>>> not working on a template.) >>>> Exception in thread "main" java.io.FileNotFoundException: >>>> /home/darshan/PredictionIO/tapster-episode-similar/src/main/scala/engine.json >>>> (No such file or directory) >>>> at java.io.FileInputStream.open0(Native Method) >>>> at java.io.FileInputStream.open(FileInputStream.java:195) >>>> at java.io.FileInputStream.<init>(FileInputStream.java:138) >>>> at scala.io.Source$.fromFile(Source.scala:90) >>>> at scala.io.Source$.fromFile(Source.scala:75) >>>> at org.apache.predictionio.tools.console.Console$.getEngineInfo >>>> (Console.scala:724) >>>> at org.apache.predictionio.tools.RunWorkflow$.runWorkflow(RunWo >>>> rkflow.scala:54) >>>> at org.apache.predictionio.tools.commands.Engine$.train(Engine. >>>> scala:186) >>>> at org.apache.predictionio.tools.console.Pio$.train(Pio.scala:85) >>>> at org.apache.predictionio.tools.console.Console$$anonfun$main$ >>>> 1.apply(Console.scala:626) >>>> at org.apache.predictionio.tools.console.Console$$anonfun$main$ >>>> 1.apply(Console.scala:611) >>>> at scala.Option.map(Option.scala:145) >>>> at org.apache.predictionio.tools.console.Console$.main(Console. >>>> scala:611) >>>> at org.apache.predictionio.tools.console.Console.main(Console.s >>>> cala) >>>> >>>> >>>> >>>> i tried almost all the things, but could not find the proper solution. >>>> please help me... >>>> i know you may feel bit odd about this mail, but in need of your help. >>>> >>>> thanks, >>>> >>>> >>>> -DAN >>>> >>> >> >
