[jira] [Updated] (HIVE-22626) Fix Replication related tests

2021-07-14 Thread Ayush Saxena (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ayush Saxena updated HIVE-22626:

Description: 
For TestStatsReplicationScenariosACIDNoAutogather:

this test is running "alone" because but still; it sometimes runs more than 40m 
which results in a timeout
 a jira search reveals that was pretty common: 
 
[https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestStatsReplicationScenariosACIDNoAutogather%22%20order%20by%20updated%20desc]

from the hive logs:
 * it seems like after a few minutes this test starts there is an exception:
{code:java}
2019-12-10T22:43:19,594 DEBUG [Finalizer] metastore.HiveMetaStoreClient: Unable 
to shutdown metastore client. Will try closing transport directly.
org.apache.thrift.transport.TTransportException: java.net.SocketException: 
Socket closed
at 
org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
 ~[libthrift-0.9.3-1.jar:0.9.3-1]
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) 
~[libthrift-0.9.3-1.jar:0.9.3-1]
at 
org.apache.thrift.TServiceClient.sendBaseOneway(TServiceClient.java:66) 
~[libthrift-0.9.3-1.jar:0.9.3-1]
at 
com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
 ~[libfb303-0.9.3.jar:?]
at 
com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430) 
~[libfb303-0.9.3.jar:?]
at 
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:776)
 [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[?:1.8.0_102]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_102]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
at 
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
 [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy62.close(Unknown Source) [?:?]
at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:542) 
[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.metadata.Hive.finalize(Hive.java:514) 
[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.lang.System$2.invokeFinalize(System.java:1270) [?:1.8.0_102]
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98) [?:1.8.0_102]
at java.lang.ref.Finalizer.access$100(Finalizer.java:34) [?:1.8.0_102]
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210) 
[?:1.8.0_102]
Caused by: java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116) 
~[?:1.8.0_102]
at java.net.SocketOutputStream.write(SocketOutputStream.java:153) 
~[?:1.8.0_102]
at 
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) 
~[?:1.8.0_102]
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) 
~[?:1.8.0_102]
at 
org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:159)
 ~[libthrift-0.9.3-1.jar:0.9.3-1]
{code}

 * after that some NoSuchObjectExceptions follow
 * and then some replications seems to happen

I don't fully understand this; I'll attach the logs...

  was:
this test is running "alone" because but still; it sometimes runs more than 40m 
which results in a timeout
a jira search reveals that was pretty common: 
https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestStatsReplicationScenariosACIDNoAutogather%22%20order%20by%20updated%20desc

from the hive logs:
* it seems like after a few minutes this test starts there is an exception:
{code}
2019-12-10T22:43:19,594 DEBUG [Finalizer] metastore.HiveMetaStoreClient: Unable 
to shutdown metastore client. Will try closing transport directly.
org.apache.thrift.transport.TTransportException: java.net.SocketException: 
Socket closed
at 
org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
 ~[libthrift-0.9.3-1.jar:0.9.3-1]
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) 
~[libthrift-0.9.3-1.jar:0.9.3-1]
at 
org.apache.thrift.TServiceClient.sendBaseOneway(TServiceClient.java:66) 
~[libthrift-0.9.3-1.jar:0.9.3-1]
at 
com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
 ~[libfb303-0.9.3.jar:?]
at 
com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430) 
~[libfb303-0.9.3.jar:?]
at 
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:776)
 [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at sun.refl

[jira] [Updated] (HIVE-22626) Fix Replication related tests

2021-07-14 Thread Ayush Saxena (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ayush Saxena updated HIVE-22626:

Summary: Fix Replication related tests  (was: Fix 
TestStatsReplicationScenariosACIDNoAutogather)

> Fix Replication related tests
> -
>
> Key: HIVE-22626
> URL: https://issues.apache.org/jira/browse/HIVE-22626
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Ayush Saxena
>Priority: Major
>  Labels: pull-request-available
> Attachments: qalogs.tgz
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> this test is running "alone" because but still; it sometimes runs more than 
> 40m which results in a timeout
> a jira search reveals that was pretty common: 
> https://issues.apache.org/jira/issues/?jql=text%20~%20%22TestStatsReplicationScenariosACIDNoAutogather%22%20order%20by%20updated%20desc
> from the hive logs:
> * it seems like after a few minutes this test starts there is an exception:
> {code}
> 2019-12-10T22:43:19,594 DEBUG [Finalizer] metastore.HiveMetaStoreClient: 
> Unable to shutdown metastore client. Will try closing transport directly.
> org.apache.thrift.transport.TTransportException: java.net.SocketException: 
> Socket closed
> at 
> org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
>  ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) 
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at 
> org.apache.thrift.TServiceClient.sendBaseOneway(TServiceClient.java:66) 
> ~[libthrift-0.9.3-1.jar:0.9.3-1]
> at 
> com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
>  ~[libfb303-0.9.3.jar:?]
> at 
> com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430) 
> ~[libfb303-0.9.3.jar:?]
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:776)
>  [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[?:1.8.0_102]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[?:1.8.0_102]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_102]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
> at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
>  [hive-standalone-metastore-common-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy62.close(Unknown Source) [?:?]
> at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:542) 
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.metadata.Hive.finalize(Hive.java:514) 
> [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at java.lang.System$2.invokeFinalize(System.java:1270) [?:1.8.0_102]
> at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98) 
> [?:1.8.0_102]
> at java.lang.ref.Finalizer.access$100(Finalizer.java:34) [?:1.8.0_102]
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210) 
> [?:1.8.0_102]
> Caused by: java.net.SocketException: Socket closed
> at 
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116) 
> ~[?:1.8.0_102]
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153) 
> ~[?:1.8.0_102]
> at 
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) 
> ~[?:1.8.0_102]
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) 
> ~[?:1.8.0_102]
> at 
> org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:159)
>  ~[libthrift-0.9.3-1.jar:0.9.3-1]
> {code}
> * after that some NoSuchObjectExceptions follow
> * and then some replications seems to happen
> I don't fully understand this; I'll attach the logs...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)