Did you make any change to the template or using a stock one? Looks like a Lucene incompatibility.
On Thu, Sep 14, 2017 at 1:40 AM tajrij kawakibi <[email protected]> wrote: > Hi donald, > > ES version is 1.7.6 > > this is log when i ran "pio train " > > 2017-09-14 15:38:06,133 INFO org.apache.predictionio.tools.Runner$ [main] > - Submission command: > /Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6/bin/spark-submit > --class org.apache.predictionio.workflow.CreateWorkflow --jars > file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/tex-classifier/target/scala-2.10/org-example-textclassification_2.10-0.1-SNAPSHOT.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/tex-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hbase-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar > --files > file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/conf/log4j.properties > --driver-class-path > /Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/conf > --driver-java-options -Dpio.log.dir=/Users/tajrijkawakibi > file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar > --engine-id org.example.textclassification.TextClassificationEngine > --engine-version 440c18b2a720f53b28d2ec839cf1f5d7660485d9 --engine-variant > file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/tex-classifier/engine.json > --verbosity 0 --json-extractor Both --env > PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/Users/tajrijkawakibi/.pio_store,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost,PIO_STORAGE_SOURCES_HBASE_HOME=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/vendors/hbase-1.2.6,PIO_HOME=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating,PIO_FS_ENGINESDIR=/Users/tajrijkawakibi/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/Users/tajrijkawakibi/.pio_store/models,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/vendors/elasticsearch-1.7.6,PIO_FS_TMPDIR=/Users/tajrijkawakibi/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/conf,PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs > 2017-09-14 15:38:08,668 INFO org.apache.predictionio.controller.Engine > [main] - Extracting datasource params... > 2017-09-14 15:38:08,754 INFO > org.apache.predictionio.workflow.WorkflowUtils$ [main] - No 'name' is > found. Default empty String will be used. > 2017-09-14 15:38:08,993 INFO org.apache.predictionio.controller.Engine > [main] - Datasource params: (,DataSourceParams(Text-1,None)) > 2017-09-14 15:38:08,993 INFO org.apache.predictionio.controller.Engine > [main] - Extracting preparator params... > 2017-09-14 15:38:08,995 INFO > org.apache.predictionio.workflow.WorkflowUtils$ [main] - No 'name' is > found. Default empty String will be used. > 2017-09-14 15:38:09,003 INFO org.apache.predictionio.controller.Engine > [main] - Preparator params: (,PreparatorParams(1,500)) > 2017-09-14 15:38:09,020 INFO org.apache.predictionio.controller.Engine > [main] - Extracting serving params... > 2017-09-14 15:38:09,023 INFO org.apache.predictionio.controller.Engine > [main] - Serving params: (,Empty) > 2017-09-14 15:38:09,427 ERROR > org.apache.predictionio.data.storage.Storage$ [main] - Error initializing > storage client for source ELASTICSEARCH > java.lang.NoSuchFieldError: LUCENE_3_6 > at org.elasticsearch.Version.<clinit>(Version.java:43) > at > org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:172) > at > org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:133) > at > org.apache.predictionio.data.storage.elasticsearch.StorageClient.liftedTree1$1(StorageClient.scala:40) > at > org.apache.predictionio.data.storage.elasticsearch.StorageClient.<init>(StorageClient.scala:33) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.predictionio.data.storage.Storage$.getClient(Storage.scala:223) > at > org.apache.predictionio.data.storage.Storage$.org$apache$predictionio$data$storage$Storage$$updateS2CM(Storage.scala:254) > at > org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$1.apply(Storage.scala:215) > at > org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$1.apply(Storage.scala:215) > at > scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:189) > at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:91) > at > org.apache.predictionio.data.storage.Storage$.sourcesToClientMeta(Storage.scala:215) > at > org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:284) > at > org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:269) > at > org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:371) > at > org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:247) > at > org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > > On Tue, Sep 12, 2017 at 10:52 PM, Donald Szeto <[email protected]> wrote: > >> Hi, >> >> What version of Elasticsearch are you running? Do you see more >> information from pio.log at where you ran "pio train"? >> >> Regards, >> Donald >> >> On Thu, Sep 7, 2017 at 1:54 AM tajrij kawakibi <[email protected]> >> wrote: >> >>> Hello, >>> >>> i just finish this 2 tutorial successfully, >>> >>> >>> https://predictionio.incubator.apache.org/templates/recommendation/quickstart/ >>> >>> >>> https://predictionio.incubator.apache.org/templates/classification/quickstart/ >>> >>> then i try >>> >>> https://predictionio.incubator.apache.org/demo/textclassification/ ( >>> the first sample , which is Sample email data for spam classification ) >>> >>> i follow step by step successfully ( create new app, import data, pio >>> build) but When i try to train i encounter error >>> >>> *pio train* >>> *SLF4J: Class path contains multiple SLF4J bindings.* >>> *SLF4J: Found binding in >>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >>> *SLF4J: Found binding in >>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >>> *SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings >>> <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.* >>> *SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]* >>> *[WARN] [WorkflowUtils$] Environment variable POSTGRES_JDBC_DRIVER is >>> pointing to a nonexistent file >>> /home/admin/PredictionIO-0.11.0-incubating/lib/postgresql-42.0.0.jar. >>> Ignoring.* >>> *[WARN] [WorkflowUtils$] Environment variable MYSQL_JDBC_DRIVER is >>> pointing to a nonexistent file >>> /home/admin/PredictionIO-0.11.0-incubating/lib/mysql-connector-java-5.1.41.jar. >>> Ignoring.* >>> *[INFO] [Runner$] Submission command: >>> /home/admin/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6/bin/spark-submit >>> --class org.apache.predictionio.workflow.CreateWorkflow --jars >>> file:/home/admin/text-classifier/target/scala-2.10/org-example-textclassification_2.10-0.1-SNAPSHOT.jar,file:/home/admin/text-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hbase-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar >>> --files >>> file:/home/admin/PredictionIO-0.11.0-incubating/conf/log4j.properties >>> --driver-class-path >>> /home/admin/PredictionIO-0.11.0-incubating/conf:/home/admin/PredictionIO-0.11.0-incubating/lib/postgresql-42.0.0.jar:/home/admin/PredictionIO-0.11.0-incubating/lib/mysql-connector-java-5.1.41.jar >>> --driver-java-options -Dpio.log.dir=/root >>> file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar >>> --engine-id org.example.textclassification.TextClassificationEngine >>> --engine-version 3aeef180d388b6b932edf67acade44b6904d9135 --engine-variant >>> file:/home/admin/text-classifier/engine.json --verbosity 0 --json-extractor >>> Both --env >>> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/root/.pio_store,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost,PIO_STORAGE_SOURCES_HBASE_HOME=/home/admin/PredictionIO-0.11.0-incubating/vendors/hbase-1.2.6,PIO_HOME=/home/admin/PredictionIO-0.11.0-incubating,PIO_FS_ENGINESDIR=/root/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/root/.pio_store/models,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/home/admin/PredictionIO-0.11.0-incubating/vendors/elasticsearch-1.7.6,PIO_FS_TMPDIR=/root/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/home/admin/PredictionIO-0.11.0-incubating/conf,PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs* >>> *[INFO] [Engine] Extracting datasource params...* >>> *[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will >>> be used.* >>> *[INFO] [Engine] Datasource params: (,DataSourceParams(MyTextApp,None))* >>> *[INFO] [Engine] Extracting preparator params...* >>> *[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will >>> be used.* >>> *[INFO] [Engine] Preparator params: (,PreparatorParams(1,500))* >>> *[INFO] [Engine] Extracting serving params...* >>> *[INFO] [Engine] Serving params: (,Empty)* >>> *[ERROR] [Storage$] Error initializing storage client for source >>> ELASTICSEARCH* >>> *Exception in thread "main" >>> org.apache.predictionio.data.storage.StorageClientException: Data source >>> ELASTICSEARCH was not properly initialized.* >>> * at >>> org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)* >>> * at >>> org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)* >>> * at scala.Option.getOrElse(Option.scala:120)* >>> * at >>> org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:284)* >>> * at >>> org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:269)* >>> * at >>> org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:371)* >>> * at >>> org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:247)* >>> * at >>> org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala)* >>> * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* >>> * at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)* >>> * at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)* >>> * at java.lang.reflect.Method.invoke(Method.java:498)* >>> * at >>> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)* >>> * at >>> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)* >>> * at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)* >>> * at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)* >>> * at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)* >>> >>> My conf at pio-env.sh is >>> >>> - Elastic search as Metadata repo >>> - Hbase as eventdata repo >>> - localfs as model repo >>> >>> result from pio status >>> >>> pio status >>> *SLF4J: Class path contains multiple SLF4J bindings.* >>> *SLF4J: Found binding in >>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >>> *SLF4J: Found binding in >>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >>> *SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings >>> <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.* >>> *SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]* >>> *[INFO] [Management$] Inspecting PredictionIO...* >>> *[INFO] [Management$] PredictionIO 0.11.0-incubating is installed at >>> /home/admin/PredictionIO-0.11.0-incubating* >>> *[INFO] [Management$] Inspecting Apache Spark...* >>> *[INFO] [Management$] Apache Spark is installed at >>> /home/admin/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6* >>> *[INFO] [Management$] Apache Spark 1.6.3 detected (meets minimum >>> requirement of 1.3.0)* >>> *[INFO] [Management$] Inspecting storage backend connections...* >>> *[INFO] [Storage$] Verifying Meta Data Backend (Source: >>> ELASTICSEARCH)...* >>> *[INFO] [Storage$] Verifying Model Data Backend (Source: LOCALFS)...* >>> *[INFO] [Storage$] Verifying Event Data Backend (Source: HBASE)...* >>> *[INFO] [Storage$] Test writing to Event Store (App Id 0)...* >>> *[INFO] [HBLEvents] The table pio_event:events_0 doesn't exist yet. >>> Creating now...* >>> *[INFO] [HBLEvents] Removing table pio_event:events_0...* >>> *[INFO] [Management$] Your system is all ready to go.* >>> >>> >>> result from pio build >>> >>> pio build --verbose >>> *SLF4J: Class path contains multiple SLF4J bindings.* >>> *SLF4J: Found binding in >>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >>> *SLF4J: Found binding in >>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]* >>> *SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings >>> <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.* >>> *SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]* >>> *[INFO] [Engine$] Using command >>> '/home/admin/PredictionIO-0.11.0-incubating/sbt/sbt' at >>> /home/admin/text-classifier to build.* >>> *[INFO] [Engine$] If the path above is incorrect, this process will >>> fail.* >>> *[INFO] [Engine$] Uber JAR disabled. Making sure >>> lib/pio-assembly-0.11.0-incubating.jar is absent.* >>> *[INFO] [Engine$] Going to run: >>> /home/admin/PredictionIO-0.11.0-incubating/sbt/sbt package >>> assemblyPackageDependency in /home/admin/text-classifier* >>> *[INFO] [Engine$] [info] Loading project definition from >>> /home/admin/text-classifier/project* >>> *[INFO] [Engine$] [info] Set current project to >>> org.example.textclassification (in build file:/home/admin/text-classifier/)* >>> *[INFO] [Engine$] [success] Total time: 2 s, completed Sep 7, 2017 >>> 8:43:48 AM* >>> *[INFO] [Engine$] [info] Including from cache: scala-library.jar* >>> *[INFO] [Engine$] [info] Including from cache: lucene-core-6.5.1.jar* >>> *[INFO] [Engine$] [info] Checking every *.class/*.jar file's SHA-1.* >>> *[INFO] [Engine$] [info] Merging files...* >>> *[INFO] [Engine$] [warn] Merging 'META-INF/MANIFEST.MF' with strategy >>> 'discard'* >>> *[INFO] [Engine$] [warn] Strategy 'discard' was applied to a file* >>> *[INFO] [Engine$] [info] Assembly up to date: >>> /home/admin/text-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar* >>> *[INFO] [Engine$] [success] Total time: 4 s, completed Sep 7, 2017 >>> 8:43:53 AM* >>> *[INFO] [Engine$] Compilation finished successfully.* >>> *[INFO] [Engine$] Looking for an engine...* >>> *[INFO] [Engine$] Found >>> org-example-textclassification_2.10-0.1-SNAPSHOT.jar* >>> *[INFO] [Engine$] Found >>> org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar* >>> *[INFO] [Engine$] Build finished successfully.* >>> *[INFO] [Pio$] Your engine is ready for training.* >>> >>> *is there something wrong with my conf ? or am i missing something ?* >>> >>> >
