Hi Shewtha,
Thanks for the link. I followed the link, but am still getting the following
error in hiveserver2.log. It seems to me that atlasclient is refused when
trying to connect to atlas server via http. I already have
atlas.http.authentication.enabled=false set to false in both application and
client properties files.
any other places I need to check?
thanks
Herman.
2016-01-20 13:55:22,568 INFO [Atlas Logger 3]: hook.HiveHook
(HiveHook.java:fireAndForget(192)) - Entered Atlas hook for hook type
POST_EXEC_HOOK operation CREATETABLE
2016-01-20 13:55:22,572 INFO [HiveServer2-Background-Pool: Thread-191]:
log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG
method=releaseLocks start=1453316122568 end=1453316122572 duration=4
from=org.apache.hadoop.hive.ql.Driver>
2016-01-20 13:55:22,573 INFO [HiveServer2-Background-Pool: Thread-191]:
log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=Driver.run
start=1453316122342 end=1453316122573 duration=231
from=org.apache.hadoop.hive.ql.Driver>
2016-01-20 13:55:22,634 INFO [Atlas Logger 3]: security.SecureClientUtils
(SecureClientUtils.java:getClientConnectionHandler(90)) - Real User: hive
(auth:SIMPLE), is from ticket cache? false
2016-01-20 13:55:22,635 INFO [Atlas Logger 3]: security.SecureClientUtils
(SecureClientUtils.java:getClientConnectionHandler(93)) - doAsUser: hue
2016-01-20 13:55:22,635 INFO [Atlas Logger 2]: hook.HiveHook
(HiveHook.java:run(182)) - Atlas hook failed
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException:
java.net.ConnectException: Connection refused
at
com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at
com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at
org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:584)
at
org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:579)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:257)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:487)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:197)
at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:66)
at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:180)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at
org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:106)
at
org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:98)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at
org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:98)
at
com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at
com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 16 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:998)
at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:934)
at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:852)
at
org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at
org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at
org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:102)
... 23 more
> On Jan 20, 2016, at 2:35 AM, Shwetha Shivalingamurthy
> <sshivalingamur...@hortonworks.com> wrote:
>
> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/Ambari-2.2.0.0/bk_
> ambari_reference_guide/content/ch_integrating_atlas.html has the
> instructions on setting up 0.6 Atlas with Ambari
>
>
> Regards,
> Shwetha
>
>
>
>
>
>
> On 20/01/16 2:26 am, "Herman Yu" <herman...@teeupdata.com> wrote:
>
>> Hi everyone,
>>
>> I am having some problems with installing and configuring v0.6 on HDP
>> 2.3.2 sandbox.
>>
>> HDP 2.3.2 comes with v0.5 pre-configured with Ambari. After compiling
>> v0.6 source code, and updated the link /usr/hdp/current/atlas-server to
>> be associated with the new v0.6 folder. I also made necessary changes in
>> amber¹s (e.g. those notification related parameters in
>> application.properties). Atlas starts fine (even though with some errors
>> in log file which are kafka related), however, Hive stopped working with
>> the following errors with any operations.
>>
>> I suspect this is the Hive hook related, after several tries, I figured
>> out that if I update atlas.hook.hive.synchronous from true to false, Hive
>> starts working fine however the hook doesn¹t capture any hive queries to
>> atlas.
>>
>> Did anyone experience the same problem? Also, how does hive know where
>> the jar file of the hook is located? in HDP 2.3.2, I don¹t see
>> HIVE_AUX_PATH is appended with <Atlas_home>/bridge/hive, I had to
>> manually append this in Ambari.
>>
>>
>> org.apache.hive.service.cli.HiveSQLException: Error while processing
>> statement: FAILED: Hive Internal Error:
>> com.sun.jersey.api.client.ClientHandlerException(java.io.IOException:
>> java.net.ConnectException: Connection refused)
>> at
>> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.j
>> ava:315)
>> at
>> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.j
>> ava:156)
>> at
>> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperatio
>> n.java:183)
>> at
>> org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
>> at
>> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementIntern
>> al(HiveSessionImpl.java:388)
>> at
>> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveS
>> essionImpl.java:369)
>> at
>> org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:26
>> 1)
>> at
>> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(Thrif
>> tCLIService.java:486)
>> at
>> org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.
>> getResult(TCLIService.java:1313)
>> at
>> org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.
>> getResult(TCLIService.java:1298)
>> at
>> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>> at
>> org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>> at
>> org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressP
>> rocessor.java:56)
>> at
>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolSe
>> rver.java:285)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>> 1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>> :615)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: com.sun.jersey.api.client.ClientHandlerException:
>> java.io.IOException: java.net.ConnectException: Connection refused
>> at
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLC
>> onnectionClientHandler.java:149)
>> at com.sun.jersey.api.client.Client.handle(Client.java:648)
>> at
>> com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
>> at
>> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>> at
>> com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
>> at
>> org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:584)
>> at
>> org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:579)
>> at org.apache.atlas.AtlasClient.getType(AtlasClient.java:257)
>> at
>> org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(Hiv
>> eMetaStoreBridge.java:487)
>> at
>> org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:197)
>> at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:174)
>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1522)
>> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1054)
>> at
>> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.j
>> ava:154)
>> ... 15 more
>>
>>
>> Thanks
>> Herman.
>>
>
