Hi donald, ES version is 1.7.6
this is log when i ran "pio train " 2017-09-14 15:38:06,133 INFO org.apache.predictionio.tools.Runner$ [main] - Submission command: /Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6/bin/spark-submit --class org.apache.predictionio.workflow.CreateWorkflow --jars file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/tex-classifier/target/scala-2.10/org-example-textclassification_2.10-0.1-SNAPSHOT.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/tex-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hbase-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar --files file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/conf/log4j.properties --driver-class-path /Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/conf --driver-java-options -Dpio.log.dir=/Users/tajrijkawakibi file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar --engine-id org.example.textclassification.TextClassificationEngine --engine-version 440c18b2a720f53b28d2ec839cf1f5d7660485d9 --engine-variant file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/tex-classifier/engine.json --verbosity 0 --json-extractor Both --env PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/Users/tajrijkawakibi/.pio_store,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost,PIO_STORAGE_SOURCES_HBASE_HOME=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/vendors/hbase-1.2.6,PIO_HOME=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating,PIO_FS_ENGINESDIR=/Users/tajrijkawakibi/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/Users/tajrijkawakibi/.pio_store/models,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/vendors/elasticsearch-1.7.6,PIO_FS_TMPDIR=/Users/tajrijkawakibi/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/conf,PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs 2017-09-14 15:38:08,668 INFO org.apache.predictionio.controller.Engine [main] - Extracting datasource params... 2017-09-14 15:38:08,754 INFO org.apache.predictionio.workflow.WorkflowUtils$ [main] - No 'name' is found. Default empty String will be used. 2017-09-14 15:38:08,993 INFO org.apache.predictionio.controller.Engine [main] - Datasource params: (,DataSourceParams(Text-1,None)) 2017-09-14 15:38:08,993 INFO org.apache.predictionio.controller.Engine [main] - Extracting preparator params... 2017-09-14 15:38:08,995 INFO org.apache.predictionio.workflow.WorkflowUtils$ [main] - No 'name' is found. Default empty String will be used. 2017-09-14 15:38:09,003 INFO org.apache.predictionio.controller.Engine [main] - Preparator params: (,PreparatorParams(1,500)) 2017-09-14 15:38:09,020 INFO org.apache.predictionio.controller.Engine [main] - Extracting serving params... 2017-09-14 15:38:09,023 INFO org.apache.predictionio.controller.Engine [main] - Serving params: (,Empty) 2017-09-14 15:38:09,427 ERROR org.apache.predictionio.data.storage.Storage$ [main] - Error initializing storage client for source ELASTICSEARCH java.lang.NoSuchFieldError: LUCENE_3_6 at org.elasticsearch.Version.<clinit>(Version.java:43) at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:172) at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:133) at org.apache.predictionio.data.storage.elasticsearch.StorageClient.liftedTree1$1(StorageClient.scala:40) at org.apache.predictionio.data.storage.elasticsearch.StorageClient.<init>(StorageClient.scala:33) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.predictionio.data.storage.Storage$.getClient(Storage.scala:223) at org.apache.predictionio.data.storage.Storage$.org$apache$predictionio$data$storage$Storage$$updateS2CM(Storage.scala:254) at org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$1.apply(Storage.scala:215) at org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$1.apply(Storage.scala:215) at scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:189) at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:91) at org.apache.predictionio.data.storage.Storage$.sourcesToClientMeta(Storage.scala:215) at org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:284) at org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:269) at org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:371) at org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:247) at org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) On Tue, Sep 12, 2017 at 10:52 PM, Donald Szeto <[email protected]> wrote: > Hi, > > What version of Elasticsearch are you running? Do you see more information > from pio.log at where you ran "pio train"? > > Regards, > Donald > > On Thu, Sep 7, 2017 at 1:54 AM tajrij kawakibi <[email protected]> > wrote: > >> Hello, >> >> i just finish this 2 tutorial successfully, >> >> https://predictionio.incubator.apache.org/templates/recommendation/ >> quickstart/ >> >> https://predictionio.incubator.apache.org/templates/classification/ >> quickstart/ >> >> then i try >> >> https://predictionio.incubator.apache.org/demo/textclassification/ ( the >> first sample , which is Sample email data for spam classification ) >> >> i follow step by step successfully ( create new app, import data, pio >> build) but When i try to train i encounter error >> >> *pio train* >> *SLF4J: Class path contains multiple SLF4J bindings.* >> *SLF4J: Found binding in >> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >> *SLF4J: Found binding in >> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >> *SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings >> <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.* >> *SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]* >> *[WARN] [WorkflowUtils$] Environment variable POSTGRES_JDBC_DRIVER is >> pointing to a nonexistent file >> /home/admin/PredictionIO-0.11.0-incubating/lib/postgresql-42.0.0.jar. >> Ignoring.* >> *[WARN] [WorkflowUtils$] Environment variable MYSQL_JDBC_DRIVER is >> pointing to a nonexistent file >> /home/admin/PredictionIO-0.11.0-incubating/lib/mysql-connector-java-5.1.41.jar. >> Ignoring.* >> *[INFO] [Runner$] Submission command: >> /home/admin/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6/bin/spark-submit >> --class org.apache.predictionio.workflow.CreateWorkflow --jars >> file:/home/admin/text-classifier/target/scala-2.10/org-example-textclassification_2.10-0.1-SNAPSHOT.jar,file:/home/admin/text-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hbase-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar >> --files >> file:/home/admin/PredictionIO-0.11.0-incubating/conf/log4j.properties >> --driver-class-path >> /home/admin/PredictionIO-0.11.0-incubating/conf:/home/admin/PredictionIO-0.11.0-incubating/lib/postgresql-42.0.0.jar:/home/admin/PredictionIO-0.11.0-incubating/lib/mysql-connector-java-5.1.41.jar >> --driver-java-options -Dpio.log.dir=/root >> file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar >> --engine-id org.example.textclassification.TextClassificationEngine >> --engine-version 3aeef180d388b6b932edf67acade44b6904d9135 --engine-variant >> file:/home/admin/text-classifier/engine.json --verbosity 0 --json-extractor >> Both --env >> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/root/.pio_store,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost,PIO_STORAGE_SOURCES_HBASE_HOME=/home/admin/PredictionIO-0.11.0-incubating/vendors/hbase-1.2.6,PIO_HOME=/home/admin/PredictionIO-0.11.0-incubating,PIO_FS_ENGINESDIR=/root/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/root/.pio_store/models,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/home/admin/PredictionIO-0.11.0-incubating/vendors/elasticsearch-1.7.6,PIO_FS_TMPDIR=/root/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/home/admin/PredictionIO-0.11.0-incubating/conf,PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs* >> *[INFO] [Engine] Extracting datasource params...* >> *[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be >> used.* >> *[INFO] [Engine] Datasource params: (,DataSourceParams(MyTextApp,None))* >> *[INFO] [Engine] Extracting preparator params...* >> *[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be >> used.* >> *[INFO] [Engine] Preparator params: (,PreparatorParams(1,500))* >> *[INFO] [Engine] Extracting serving params...* >> *[INFO] [Engine] Serving params: (,Empty)* >> *[ERROR] [Storage$] Error initializing storage client for source >> ELASTICSEARCH* >> *Exception in thread "main" >> org.apache.predictionio.data.storage.StorageClientException: Data source >> ELASTICSEARCH was not properly initialized.* >> * at >> org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)* >> * at >> org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)* >> * at scala.Option.getOrElse(Option.scala:120)* >> * at >> org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:284)* >> * at >> org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:269)* >> * at >> org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:371)* >> * at >> org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:247)* >> * at >> org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala)* >> * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* >> * at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)* >> * at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)* >> * at java.lang.reflect.Method.invoke(Method.java:498)* >> * at >> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)* >> * at >> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)* >> * at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)* >> * at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)* >> * at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)* >> >> My conf at pio-env.sh is >> >> - Elastic search as Metadata repo >> - Hbase as eventdata repo >> - localfs as model repo >> >> result from pio status >> >> pio status >> *SLF4J: Class path contains multiple SLF4J bindings.* >> *SLF4J: Found binding in >> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >> *SLF4J: Found binding in >> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >> *SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings >> <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.* >> *SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]* >> *[INFO] [Management$] Inspecting PredictionIO...* >> *[INFO] [Management$] PredictionIO 0.11.0-incubating is installed at >> /home/admin/PredictionIO-0.11.0-incubating* >> *[INFO] [Management$] Inspecting Apache Spark...* >> *[INFO] [Management$] Apache Spark is installed at >> /home/admin/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6* >> *[INFO] [Management$] Apache Spark 1.6.3 detected (meets minimum >> requirement of 1.3.0)* >> *[INFO] [Management$] Inspecting storage backend connections...* >> *[INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...* >> *[INFO] [Storage$] Verifying Model Data Backend (Source: LOCALFS)...* >> *[INFO] [Storage$] Verifying Event Data Backend (Source: HBASE)...* >> *[INFO] [Storage$] Test writing to Event Store (App Id 0)...* >> *[INFO] [HBLEvents] The table pio_event:events_0 doesn't exist yet. >> Creating now...* >> *[INFO] [HBLEvents] Removing table pio_event:events_0...* >> *[INFO] [Management$] Your system is all ready to go.* >> >> >> result from pio build >> >> pio build --verbose >> *SLF4J: Class path contains multiple SLF4J bindings.* >> *SLF4J: Found binding in >> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >> *SLF4J: Found binding in >> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >> *SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings >> <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.* >> *SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]* >> *[INFO] [Engine$] Using command >> '/home/admin/PredictionIO-0.11.0-incubating/sbt/sbt' at >> /home/admin/text-classifier to build.* >> *[INFO] [Engine$] If the path above is incorrect, this process will fail.* >> *[INFO] [Engine$] Uber JAR disabled. Making sure >> lib/pio-assembly-0.11.0-incubating.jar is absent.* >> *[INFO] [Engine$] Going to run: >> /home/admin/PredictionIO-0.11.0-incubating/sbt/sbt package >> assemblyPackageDependency in /home/admin/text-classifier* >> *[INFO] [Engine$] [info] Loading project definition from >> /home/admin/text-classifier/project* >> *[INFO] [Engine$] [info] Set current project to >> org.example.textclassification (in build file:/home/admin/text-classifier/)* >> *[INFO] [Engine$] [success] Total time: 2 s, completed Sep 7, 2017 >> 8:43:48 AM* >> *[INFO] [Engine$] [info] Including from cache: scala-library.jar* >> *[INFO] [Engine$] [info] Including from cache: lucene-core-6.5.1.jar* >> *[INFO] [Engine$] [info] Checking every *.class/*.jar file's SHA-1.* >> *[INFO] [Engine$] [info] Merging files...* >> *[INFO] [Engine$] [warn] Merging 'META-INF/MANIFEST.MF' with strategy >> 'discard'* >> *[INFO] [Engine$] [warn] Strategy 'discard' was applied to a file* >> *[INFO] [Engine$] [info] Assembly up to date: >> /home/admin/text-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar* >> *[INFO] [Engine$] [success] Total time: 4 s, completed Sep 7, 2017 >> 8:43:53 AM* >> *[INFO] [Engine$] Compilation finished successfully.* >> *[INFO] [Engine$] Looking for an engine...* >> *[INFO] [Engine$] Found >> org-example-textclassification_2.10-0.1-SNAPSHOT.jar* >> *[INFO] [Engine$] Found >> org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar* >> *[INFO] [Engine$] Build finished successfully.* >> *[INFO] [Pio$] Your engine is ready for training.* >> >> *is there something wrong with my conf ? or am i missing something ?* >> >>
