Hi Xiaoxiao,
Have you tried including hbase-site.xml in your conf on classpath?
Will
On Wed, Feb 20, 2019 at 2:50 PM Xiaoxiao Wang <xxw...@23andme.com.invalid>
wrote:
> Hi, who may help
>
> We are running a Hadoop application that needs to use phoenix JDBC
> connection from the workers.
> The connection works, but when too many connection established at the same
> time, it throws RPC timeouts
>
> Error: java.io.IOException:
> org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36,
> exceptions: Wed Feb 20 20:02:43 UTC 2019, null,
> java.net.SocketTimeoutException:
> callTimeout=60000, callDuration=60506. ...
>
> So we have figured we should probably set a higher hbase.rpc.timeout
> value, but then it comes to the issue:
>
> A little bit background on how we run the application
>
> Here is how we get PhoenixConnection from java program
> DriverManager.getConnection("jdbc:phoenix:host", props)
> And we trigger the program by using
> hadoop jar $test_jar
>
>
> We have tried multiple approaches to load hbase/phoenix configuration, but
> none of them get respected by PhoenixConnection, here are the methods we
> tried
> * Pass hbase_conf_dir through HADOOP_CLASSPATH, so run the hadoop
> application like HADOOP_CLASSPATH=/etc/hbase/conf/ hadoop jar $test_jar .
> However, PhoenixConnection doesn’t respect the parameters
> * Tried passing -Dhbase.rpc.timeout=1800, which is picked up by hbase conf
> object, but not PhoniexConnection
> * Explicitly set those parameters and pass them to the PhoenixConnection
> props.setProperty("hbase.rpc.timeout", "1800");
> props.setProperty(“phoenix.query.timeoutMs", "1800");
> Also didn’t get respected by PhoenixConnection
> * also tried what is suggested by phoenix here
> https://phoenix.apache.org/#connStr , use :longRunning together with
> those properties, still didn’t seem to work
>
>
> Besides all those approaches we tried, I have explicitly output those
> parameters we care from the connection,
> connection.getQueryServices().getProps()
> The default values I got are 60000 for hbase.rpc.timeout, and 600k for
> phoenix.query.timeoutMs , so I have tried to run a query lthat would run
> longer than 10 mins, Ideally it should timeout, however, it runs over 20
> mins and didn’t timeout. So I’m wondering how PhoenixConnection respect
> those properties?
>
>
> So with some of your help, we’d like to know if there’s any thing wrong
> with our approaches. And we’d like to get rid of those SocketTimeExceptions.
> We are using phoenix-core version is 4.7.0-clabs-phoenix1.3.0 , and our
> phoenix-client version is phoenix-4.7.0-clabs-phoenix1.3.0.23 (we have
> tried phoenix-4.14.0-HBase-1.3 as well, which didn’t work either).
>
>
> Thanks for your time
>
>
>
>
>
