I think the index is created using the transport client so requires no auth and happens before the writing starts, which uses the REST API. So the REST API seems to not be working at all.
Check that the Spark Executor can connect to REST with your config. Also include "es.index.auto.create": "true" On Nov 23, 2016, at 6:56 AM, Harsh Mathur <[email protected]> wrote: Hi, The problem is in training phase of the UR template, the model (correlators), is built, but unable to persist in elastic search. The connectivity is there because indices are getting created, though no data is being pushed. Help appreciated. My SparkConf is as follows: "es.nodes": "hostname", "es.port": port, "es.net.http.auth.user": "username", "es.net.http.auth.pass": "password", "es.net.ssl": true However, I can see a new index getting created but no data being pushed (timeout exception) https://basic_auth@host:port/_cat/indices?v green open urindex_1479911005241 4 0 0 0 460b 460b green open pio_meta 4 0 19 1 210.8kb 210.8kb green open urindex_1479909625467 4 0 0 0 460b 460b green open urindex_1479908293213 4 0 0 0 460b 460b green open urindex_1479911811014 4 0 0 0 460b 460b green open urindex_1479910315092 4 0 0 0 460b 460b As you can see after every retry, I can see a new index. The Exception I get is: [INFO] [URAlgorithm] Correlators created now putting into URModel [INFO] [URModel] Converting cooccurrence matrices into correlators [INFO] [URModel] Group all properties RDD [INFO] [URModel] ES fields[10]: List(available, id, defaultRank, categories, expires, popRank, purchase, countries, view, date) [Stage 103:> (0 + 2) / 5][INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection timed out (Connection timed out) [INFO] [HttpMethodDirector] Retrying request [INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection timed out (Connection timed out) [INFO] [HttpMethodDirector] Retrying request Regards Harsh Mathur [email protected] <mailto:[email protected]> “Perseverance is the hard work you do after you get tired of doing the hard work you already did."
