What template and version are you using? On Tue, Sep 26, 2017 at 9:28 PM Jim Miller <jemiller1...@gmail.com> wrote:
> Hi guys, > > I have another error that I cannot get past: > > *vagrant:MyRecommendation/ (develop*) $ pio train > > [21:47:39]* > *[ERROR] [Console$] unable to read engine factory from > /home/vagrant/MyRecommendation/engine.json. Aborting.* > > > Does anyone have any advice? > > Thanks, > > Jim > > -- > Jim Miller > > On September 26, 2017 at 7:42:02 PM, Jim Miller (jemiller1...@gmail.com) > wrote: > > Shinsuke, you rock! That was it. > > Thank you so much, > > Jim > > -- > Jim Miller > > On September 24, 2017 at 7:29:28 PM, Shinsuke Sugaya ( > shinsuke.sug...@gmail.com) wrote: > > Hi, > > > PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300 > > Elasticsearch 5 uses 9200 port. > Could you try 9200 port? > > Thanks, > shinsuke > > > 2017-09-24 3:02 GMT+09:00 Jim Miller <jemiller1...@gmail.com>: > > Hi Donald, > > > > Tried just now and received the following error: > > > > vagrant:~/ $ pio status > > [13:34:52] > > [INFO] [Management$] Inspecting PredictionIO... > > [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at > > /home/vagrant/pio/PredictionIO-0.12.0-incubating > > [INFO] [Management$] Inspecting Apache Spark... > > [INFO] [Management$] Apache Spark is installed at > > > /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-2.1.1-bin-hadoop2.7 > > [INFO] [Management$] Apache Spark 2.1.1 detected (meets minimum > requirement > > of 1.3.0) > > [INFO] [Management$] Inspecting storage backend connections... > > [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)... > > [ERROR] [Management$] Unable to connect to all storage backends > > successfully. > > The following shows the error message from the storage backend. > > > > error while performing request (java.lang.RuntimeException) > > > > Dumping configuration of initialized storage backend sources. > > Please make sure they are correct. > > > > Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> > > > /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-5.5.2, > > HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstCluster, TYPE -> > > elastic search > > > > > > HERE IS MY PIO-ENV.SH > > > > # PredictionIO Main Configuration > > # > > # This section controls core behavior of PredictionIO. It is very likely > > that > > # you need to change these to fit your site. > > > > # SPARK_HOME: Apache Spark is a hard dependency and must be configured. > > # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7 > > SPARK_HOME=$PIO_HOME/vendors/spark-2.1.1-bin-hadoop2.7 > > > > POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar > > MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar > > > > # ES_CONF_DIR: You must configure this if you have advanced configuration > > for > > # your Elasticsearch setup. > > ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-5.5.2 > > # HADOOP_CONF_DIR=/opt/hadoop > > > > # HBASE_CONF_DIR: You must configure this if you intend to run > PredictionIO > > # with HBase on a remote cluster. > > HBASE_CONF_DIR=$PIO_HOME/hbase-1.3.1/conf > > > > # Filesystem paths where PredictionIO uses as block storage. > > PIO_FS_BASEDIR=$HOME/.pio_store > > PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines > > PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp > > > > # PredictionIO Storage Configuration > > # > > # This section controls programs that make use of PredictionIO's built-in > > # storage facilities. Default values are shown below. > > # > > # For more information on storage configuration please refer to > > # http://predictionio.incubator.apache.org/system/anotherdatastore/ > > > > # Storage Repositories > > > > # Default is to use PostgreSQL > > PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta > > PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH > > > > PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event > > PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE > > > > PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model > > PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS > > > > # Storage Data Sources > > > > # PostgreSQL Default Settings > > # Please change "pio" to your database name in > PIO_STORAGE_SOURCES_PGSQL_URL > > # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and > > # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly > > # PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc > > # PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio > > # PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio > > # PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio > > > > # MySQL Example > > # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc > > # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio > > # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio > > # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio > > > > # Elasticsearch Example > > # PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch > > # PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost > > # PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200 > > # PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http > > # > > > PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2 > > # Optional basic HTTP auth > > # PIO_STORAGE_SOURCES_ELASTICSEARCH_USERNAME=my-name > > # PIO_STORAGE_SOURCES_ELASTICSEARCH_PASSWORD=my-secret > > # Elasticsearch 1.x Example > > PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch > > PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=firstCluster > > PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost > > PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300 > > > PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2 > > > > # Local File System Example > > PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs > > PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models > > > > # HBase Example > > PIO_STORAGE_SOURCES_HBASE_TYPE=hbase > > PIO_STORAGE_SOURCES_HBASE_HOME=$PIO_HOME/vendors/hbase-1.3.1 > > > > # AWS S3 Example > > # PIO_STORAGE_SOURCES_S3_TYPE=s3 > > # PIO_STORAGE_SOURCES_S3_BUCKET_NAME=pio_bucket > > # PIO_STORAGE_SOURCES_S3_BASE_PATH=pio_model > > > > > > ELASTICSEARCH.YML > > > > #======================== Elasticsearch Configuration > > ========================= > > # > > # NOTE: Elasticsearch comes with reasonable defaults for most settings. > > # Before you set out to tweak and tune the configuration, make sure > > you > > # understand what are you trying to accomplish and the consequences. > > # > > # The primary way of configuring a node is via this file. This template > > lists > > # the most important settings you may want to configure for a production > > cluster. > > # > > # Please consult the documentation for further information on > configuration > > options: > > # https://www.elastic.co/guide/en/elasticsearch/reference/index.html > > # > > # ---------------------------------- Cluster > > ----------------------------------- > > # > > # Use a descriptive name for your cluster: > > # > > cluster.name: firstCluster > > # > > # ------------------------------------ Node > > ------------------------------------ > > # > > # Use a descriptive name for the node: > > # > > #node.name: node-1 > > # > > # Add custom attributes to the node: > > # > > #node.attr.rack: r1 > > # > > # ----------------------------------- Paths > > ------------------------------------ > > # > > # Path to directory where to store the data (separate multiple locations > by > > comma): > > # > > #path.data: /path/to/data > > # > > # Path to log files: > > # > > #path.logs: /path/to/logs > > # > > # ----------------------------------- Memory > > ----------------------------------- > > # > > # Lock the memory on startup: > > # > > #bootstrap.memory_lock: true > > # > > # Make sure that the heap size is set to about half the memory available > > # on the system and that the owner of the process is allowed to use this > > # limit. > > # > > # Elasticsearch performs poorly when the system is swapping the memory. > > # > > # ---------------------------------- Network > > ----------------------------------- > > # > > # Set the bind address to a specific IP (IPv4 or IPv6): > > # > > #network.host: 192.168.0.1 > > # > > # Set a custom port for HTTP: > > # > > #http.port: 9200 > > # > > # For more information, consult the network module documentation. > > # > > # --------------------------------- Discovery > > ---------------------------------- > > # > > # Pass an initial list of hosts to perform discovery when new node is > > started: > > # The default list of hosts is ["127.0.0.1", "[::1]"] > > # > > #discovery.zen.ping.unicast.hosts: ["host1", "host2"] > > # > > # Prevent the "split brain" by configuring the majority of nodes (total > > number of master-eligible nodes / 2 + 1): > > # > > #discovery.zen.minimum_master_nodes: 3 > > # > > # For more information, consult the zen discovery module documentation. > > # > > # ---------------------------------- Gateway > > ----------------------------------- > > # > > # Block initial recovery after a full cluster restart until N nodes are > > started: > > # > > #gateway.recover_after_nodes: 3 > > # > > # For more information, consult the gateway module documentation. > > # > > # ---------------------------------- Various > > ----------------------------------- > > # > > # Require explicit names when deleting indices: > > # > > #action.destructive_requires_name: true > > > > > > Thanks for all of your help! > > > > Jim > > -- > > Jim Miller > > > > On September 20, 2017 at 9:14:37 PM, Jim Miller (jemiller1...@gmail.com) > > wrote: > > > > Hi Donald, > > > > I did not. I will read the release notes and update accordingly. > > > > Thanks! > > > > Jim > > > > -- > > Jim Miller > > > > On September 20, 2017 at 1:01:53 PM, Donald Szeto (don...@apache.org) > wrote: > > > > Hey Jim, > > > > Did you build PIO 0.12 with ES 1.4 support? ES 1.x is being deprecated in > > 0.12 so the default build will use ES 5.x support. > > > > See the upcoming release notes: > > > https://github.com/apache/incubator-predictionio/blob/release/0.12.0/RELEASE.md#behavior-changes > > > > Regards, > > Donald > > > > On Wed, Sep 20, 2017 at 8:14 AM, Jim Miller <jemiller1...@gmail.com> > wrote: > >> > >> Yes. I’m following this tutorial but using 0.12.0 instead of 0.10.0: > >> > >> > https://medium.freecodecamp.org/building-an-recommendation-engine-with-apache-prediction-io-ml-server-aed0319e0d8 > >> > >> -- > >> Jim Miller > >> > >> On September 20, 2017 at 10:51:39 AM, Pat Ferrel (p...@occamsmachete.com > ) > >> wrote: > >> > >> meaning is “firstcluster” the cluster name in your Elasticsearch > >> configuration? > >> > >> > >> On Sep 19, 2017, at 8:54 PM, Vaghawan Ojha <vaghawan...@gmail.com> > wrote: > >> > >> I think the problem is with Elasticsearch, are you sure the cluster > exists > >> in elasticsearch configuration? > >> > >> On Wed, Sep 20, 2017 at 8:17 AM, Jim Miller <jemiller1...@gmail.com> > >> wrote: > >>> > >>> Hi, > >>> > >>> I’m using PredictionIO 0.12.0-incubating with ElasticSearch and Hbase: > >>> > >>> PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4 > >>> PredictionIO-0.12.0-incubating/vendors/hbase-1.0.0 > >>> PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6 > >>> > >>> > >>> All starts with no errors but with pio status I get: > >>> > >>> [INFO] [Management$] Inspecting PredictionIO... > >>> [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at > >>> /home/vagrant/pio/PredictionIO-0.12.0-incubating > >>> [INFO] [Management$] Inspecting Apache Spark... > >>> [INFO] [Management$] Apache Spark is installed at > >>> > /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6 > >>> [INFO] [Management$] Apache Spark 1.5.1 detected (meets minimum > >>> requirement of 1.3.0) > >>> [INFO] [Management$] Inspecting storage backend connections... > >>> [INFO] [Storage$] Verifying Meta Data Backend (Source: > ELASTICSEARCH)... > >>> [ERROR] [Management$] Unable to connect to all storage backends > >>> successfully. > >>> The following shows the error message from the storage backend. > >>> > >>> Connection closed > >>> > (org.apache.predictionio.shaded.org.apache.http.ConnectionClosedException) > >>> > >>> Dumping configuration of initialized storage backend sources. > >>> Please make sure they are correct. > >>> > >>> Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> > >>> > /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4, > >>> HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstcluster, TYPE -> > >>> elasticsearch > >>> > >>> > >>> Can anyone give me an idea of what I need to fix this issue? Here is > >>> > >>> > >>> # PredictionIO Main Configuration > >>> # > >>> # This section controls core behavior of PredictionIO. It is very > likely > >>> that > >>> # you need to change these to fit your site. > >>> > >>> # SPARK_HOME: Apache Spark is a hard dependency and must be configured. > >>> # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7 > >>> SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6 > >>> > >>> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar > >>> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar > >>> > >>> # ES_CONF_DIR: You must configure this if you have advanced > configuration > >>> for > >>> # your Elasticsearch setup. > >>> ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-1.4.4/conf > >>> > >>> # HADOOP_CONF_DIR: You must configure this if you intend to run > >>> PredictionIO > >>> # with Hadoop 2. > >>> HADOOP_CONF_DIR=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6/conf > >>> > >>> # HBASE_CONF_DIR: You must configure this if you intend to run > >>> PredictionIO > >>> # with HBase on a remote cluster. > >>> HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf > >>> > >>> # Filesystem paths where PredictionIO uses as block storage. > >>> PIO_FS_BASEDIR=$HOME/.pio_store > >>> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines > >>> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp > >>> > >>> # PredictionIO Storage Configuration > >>> # > >>> # This section controls programs that make use of PredictionIO's > built-in > >>> # storage facilities. Default values are shown below. > >>> # > >>> # For more information on storage configuration please refer to > >>> # http://predictionio.incubator.apache.org/system/anotherdatastore/ > >>> > >>> # Storage Repositories > >>> > >>> # Default is to use PostgreSQL > >>> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta > >>> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH > >>> > >>> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event > >>> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE > >>> > >>> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model > >>> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS > >>> > >>> # Storage Data Sources > >>> > >>> # PostgreSQL Default Settings > >>> # Please change "pio" to your database name in > >>> PIO_STORAGE_SOURCES_PGSQL_URL > >>> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and > >>> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly > >>> # PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc > >>> # PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio > >>> # PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio > >>> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio > >>> > >>> # MySQL Example > >>> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc > >>> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio > >>> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio > >>> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio > >>> > >>> # Elasticsearch Example > >>> # PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch > >>> # PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost > >>> # PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200 > >>> # PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http > >>> # > >>> > PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2 > >>> # Optional basic HTTP auth > >>> # PIO_STORAGE_SOURCES_ELASTICSEARCH_USERNAME=my-name > >>> # PIO_STORAGE_SOURCES_ELASTICSEARCH_PASSWORD=my-secret > >>> # Elasticsearch 1.x Example > >>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch > >>> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=firstcluster > >>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost > >>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300 > >>> > >>> > PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-1.4.4 > >>> > >>> # Local File System Example > >>> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs > >>> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models > >>> > >>> # HBase Example > >>> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase > >>> PIO_STORAGE_SOURCES_HBASE_HOME=$PIO_HOME/vendors/hbase-1.0.0 > >>> > >>> # AWS S3 Example > >>> # PIO_STORAGE_SOURCES_S3_TYPE=s3 > >>> # PIO_STORAGE_SOURCES_S3_BUCKET_NAME=pio_bucket > >>> # PIO_STORAGE_SOURCES_S3_BASE_PATH=pio_model > >>> > >>> > >>> Thanks, > >>> > >>> Jim > >>> > >>> > >>> -- > >>> Jim Miller > >> > >> > >> > > > >
