Hi Donald, Tried just now and received the following error:
vagrant:~/ $ pio status [13:34:52] [INFO] [Management$] Inspecting PredictionIO... [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at /home/vagrant/pio/PredictionIO-0.12.0-incubating [INFO] [Management$] Inspecting Apache Spark... [INFO] [Management$] Apache Spark is installed at /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-2.1.1-bin-hadoop2.7 [INFO] [Management$] Apache Spark 2.1.1 detected (meets minimum requirement of 1.3.0) [INFO] [Management$] Inspecting storage backend connections... [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)... [ERROR] [Management$] Unable to connect to all storage backends successfully. The following shows the error message from the storage backend. error while performing request (java.lang.RuntimeException) Dumping configuration of initialized storage backend sources. Please make sure they are correct. Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-5.5.2, HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstCluster, TYPE -> elastic search HERE IS MY PIO-ENV.SH # PredictionIO Main Configuration # # This section controls core behavior of PredictionIO. It is very likely that # you need to change these to fit your site. # SPARK_HOME: Apache Spark is a hard dependency and must be configured. # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7 SPARK_HOME=$PIO_HOME/vendors/spark-2.1.1-bin-hadoop2.7 POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar # ES_CONF_DIR: You must configure this if you have advanced configuration for # your Elasticsearch setup. ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-5.5.2 # HADOOP_CONF_DIR=/opt/hadoop # HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO # with HBase on a remote cluster. HBASE_CONF_DIR=$PIO_HOME/hbase-1.3.1/conf # Filesystem paths where PredictionIO uses as block storage. PIO_FS_BASEDIR=$HOME/.pio_store PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp # PredictionIO Storage Configuration # # This section controls programs that make use of PredictionIO's built-in # storage facilities. Default values are shown below. # # For more information on storage configuration please refer to # http://predictionio.incubator.apache.org/system/anotherdatastore/ # Storage Repositories # Default is to use PostgreSQL PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS # Storage Data Sources # PostgreSQL Default Settings # Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly # PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc # PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio # PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio # PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio # MySQL Example # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio # Elasticsearch Example # PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch # PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost # PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200 # PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http # PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2 # Optional basic HTTP auth # PIO_STORAGE_SOURCES_ELASTICSEARCH_USERNAME=my-name # PIO_STORAGE_SOURCES_ELASTICSEARCH_PASSWORD=my-secret # Elasticsearch 1.x Example PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=firstCluster PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300 PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2 # Local File System Example PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models # HBase Example PIO_STORAGE_SOURCES_HBASE_TYPE=hbase PIO_STORAGE_SOURCES_HBASE_HOME=$PIO_HOME/vendors/hbase-1.3.1 # AWS S3 Example # PIO_STORAGE_SOURCES_S3_TYPE=s3 # PIO_STORAGE_SOURCES_S3_BUCKET_NAME=pio_bucket # PIO_STORAGE_SOURCES_S3_BASE_PATH=pio_model ELASTICSEARCH.YML #======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: firstCluster # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # #node.name: node-1 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # #network.host: 192.168.0.1 # # Set a custom port for HTTP: # #http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # #discovery.zen.ping.unicast.hosts: ["host1", "host2"] # # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1): # #discovery.zen.minimum_master_nodes: 3 # # For more information, consult the zen discovery module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true Thanks for all of your help! Jim -- Jim Miller On September 20, 2017 at 9:14:37 PM, Jim Miller (jemiller1...@gmail.com) wrote: Hi Donald, I did not. I will read the release notes and update accordingly. Thanks! Jim -- Jim Miller On September 20, 2017 at 1:01:53 PM, Donald Szeto (don...@apache.org) wrote: Hey Jim, Did you build PIO 0.12 with ES 1.4 support? ES 1.x is being deprecated in 0.12 so the default build will use ES 5.x support. See the upcoming release notes: https://github.com/apache/incubator-predictionio/blob/release/0.12.0/RELEASE.md#behavior-changes Regards, Donald On Wed, Sep 20, 2017 at 8:14 AM, Jim Miller <jemiller1...@gmail.com> wrote: Yes. I’m following this tutorial but using 0.12.0 instead of 0.10.0: https://medium.freecodecamp.org/building-an-recommendation-engine-with-apache-prediction-io-ml-server-aed0319e0d8 -- Jim Miller On September 20, 2017 at 10:51:39 AM, Pat Ferrel (p...@occamsmachete.com) wrote: meaning is “firstcluster” the cluster name in your Elasticsearch configuration? On Sep 19, 2017, at 8:54 PM, Vaghawan Ojha <vaghawan...@gmail.com> wrote: I think the problem is with Elasticsearch, are you sure the cluster exists in elasticsearch configuration? On Wed, Sep 20, 2017 at 8:17 AM, Jim Miller <jemiller1...@gmail.com> wrote: Hi, I’m using PredictionIO 0.12.0-incubating with ElasticSearch and Hbase: PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4 PredictionIO-0.12.0-incubating/vendors/hbase-1.0.0 PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6 All starts with no errors but with pio status I get: [INFO] [Management$] Inspecting PredictionIO... [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at /home/vagrant/pio/PredictionIO-0.12.0-incubating [INFO] [Management$] Inspecting Apache Spark... [INFO] [Management$] Apache Spark is installed at /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6 [INFO] [Management$] Apache Spark 1.5.1 detected (meets minimum requirement of 1.3.0) [INFO] [Management$] Inspecting storage backend connections... [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)... [ERROR] [Management$] Unable to connect to all storage backends successfully. The following shows the error message from the storage backend. Connection closed (org.apache.predictionio.shaded.org.apache.http.ConnectionClosedException) Dumping configuration of initialized storage backend sources. Please make sure they are correct. Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4, HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstcluster, TYPE -> elasticsearch Can anyone give me an idea of what I need to fix this issue? Here is # PredictionIO Main Configuration # # This section controls core behavior of PredictionIO. It is very likely that # you need to change these to fit your site. # SPARK_HOME: Apache Spark is a hard dependency and must be configured. # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7 SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6 POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar # ES_CONF_DIR: You must configure this if you have advanced configuration for # your Elasticsearch setup. ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-1.4.4/conf # HADOOP_CONF_DIR: You must configure this if you intend to run PredictionIO # with Hadoop 2. HADOOP_CONF_DIR=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6/conf # HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO # with HBase on a remote cluster. HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf # Filesystem paths where PredictionIO uses as block storage. PIO_FS_BASEDIR=$HOME/.pio_store PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp # PredictionIO Storage Configuration # # This section controls programs that make use of PredictionIO's built-in # storage facilities. Default values are shown below. # # For more information on storage configuration please refer to # http://predictionio.incubator.apache.org/system/anotherdatastore/ # Storage Repositories # Default is to use PostgreSQL PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS # Storage Data Sources # PostgreSQL Default Settings # Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly # PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc # PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio # PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio # PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio # MySQL Example # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio # Elasticsearch Example # PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch # PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost # PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200 # PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http # PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2 # Optional basic HTTP auth # PIO_STORAGE_SOURCES_ELASTICSEARCH_USERNAME=my-name # PIO_STORAGE_SOURCES_ELASTICSEARCH_PASSWORD=my-secret # Elasticsearch 1.x Example PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=firstcluster PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300 PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-1.4.4 # Local File System Example PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models # HBase Example PIO_STORAGE_SOURCES_HBASE_TYPE=hbase PIO_STORAGE_SOURCES_HBASE_HOME=$PIO_HOME/vendors/hbase-1.0.0 # AWS S3 Example # PIO_STORAGE_SOURCES_S3_TYPE=s3 # PIO_STORAGE_SOURCES_S3_BUCKET_NAME=pio_bucket # PIO_STORAGE_SOURCES_S3_BASE_PATH=pio_model Thanks, Jim -- Jim Miller
