[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[
https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1581#comment-1581
]
Hive QA commented on HIVE-9642:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12778570/HIVE-9642.7.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2786/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2786/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2786/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-01-05 02:03:06.881
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-2786/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-01-05 02:03:06.883
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at ad335c1 HIVE-15518: Refactoring rows and range related classes
to put the window type on Window (Aihua Xu, reviewed by Yongzhi Chen)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at ad335c1 HIVE-15518: Refactoring rows and range related classes
to put the window type on Window (Aihua Xu, reviewed by Yongzhi Chen)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-01-05 02:03:07.864
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/hcatalog/core/src/test/java/org/apache/hive/hcatalog/cli/TestSemanticAnalysis.java:
No such file or directory
error:
a/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java:
No such file or directory
error:
a/metastore/src/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java:
No such file or directory
error:
a/ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java:
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12778570 - PreCommit-HIVE-Build
> Hive metastore client retries don't happen consistently for all api calls
> -
>
> Key: HIVE-9642
> URL: https://issues.apache.org/jira/browse/HIVE-9642
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Xiaobing Zhou
>Assignee: Daniel Dai
> Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch, HIVE-9642.3.patch,
> HIVE-9642.4.patch, HIVE-9642.5.patch, HIVE-9642.5.patch, HIVE-9642.6.patch,
> HIVE-9642.7.patch
>
>
> When org.apache.thrift.transport.TTransportException is thrown for issues
> like socket timeout, the retry via RetryingMetaStoreClient happens only in
> certain cases.
> Retry happens for the getDatabase call in but not for getAllDatabases().
> The reason is RetryingMetaStoreClient checks for TTransportException being
> the cause for InvocationTargetException. But in case of some calls such as
> getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a
> MetaException. We should remove this unnecessary wrapping of exceptions for
> certain functions in HMC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[ https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799892#comment-15799892 ] Anthony Hsu commented on HIVE-9642: --- Has this already been fixed by HIVE-10384? > Hive metastore client retries don't happen consistently for all api calls > - > > Key: HIVE-9642 > URL: https://issues.apache.org/jira/browse/HIVE-9642 > Project: Hive > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Xiaobing Zhou >Assignee: Daniel Dai > Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch, HIVE-9642.3.patch, > HIVE-9642.4.patch, HIVE-9642.5.patch, HIVE-9642.5.patch, HIVE-9642.6.patch, > HIVE-9642.7.patch > > > When org.apache.thrift.transport.TTransportException is thrown for issues > like socket timeout, the retry via RetryingMetaStoreClient happens only in > certain cases. > Retry happens for the getDatabase call in but not for getAllDatabases(). > The reason is RetryingMetaStoreClient checks for TTransportException being > the cause for InvocationTargetException. But in case of some calls such as > getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a > MetaException. We should remove this unnecessary wrapping of exceptions for > certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[
https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15065332#comment-15065332
]
Hive QA commented on HIVE-9642:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12778570/HIVE-9642.7.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 9953 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_stats2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_filemetadata
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_move_tbl
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_stats_filemetadata
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_columnstats_partlvl_multiple_part_clause
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testMultiSessionMultipleUse
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testSingleSessionMultipleUse
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveOperationType.checkHiveOperationTypeMatch
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarDataNucleusUnCaching
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.hive.spark.client.TestSparkClient.testAddJarsAndFiles
org.apache.hive.spark.client.TestSparkClient.testCounters
org.apache.hive.spark.client.TestSparkClient.testErrorJob
org.apache.hive.spark.client.TestSparkClient.testJobSubmission
org.apache.hive.spark.client.TestSparkClient.testMetricsCollection
org.apache.hive.spark.client.TestSparkClient.testRemoteClient
org.apache.hive.spark.client.TestSparkClient.testSimpleSparkJob
org.apache.hive.spark.client.TestSparkClient.testSyncRpc
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6405/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6405/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6405/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12778570 - PreCommit-HIVE-TRUNK-Build
> Hive metastore client retries don't happen consistently for all api calls
> -
>
> Key: HIVE-9642
> URL: https://issues.apache.org/jira/browse/HIVE-9642
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Xiaobing Zhou
>Assignee: Daniel Dai
> Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch, HIVE-9642.3.patch,
> HIVE-9642.4.patch, HIVE-9642.5.patch, HIVE-9642.5.patch, HIVE-9642.6.patch,
> HIVE-9642.7.patch
>
>
> When org.apache.thrift.transport.TTransportException is thrown for issues
> like socket timeout, the retry via RetryingMetaStoreClient happens only in
> certain cases.
> Retry happens for the getDatabase call in but not for getAllDatabases().
> The reason is RetryingMetaStoreClient checks for TTransportException being
> the cause for InvocationTargetException. But in case of some calls such as
> getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a
> MetaException. We should remove this unnecessary wrapping of exceptions for
> certain functions in HMC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[
https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15042578#comment-15042578
]
Hive QA commented on HIVE-9642:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12775676/HIVE-9642.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 9895 tests executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_udf_max
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_move_tbl
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vectorized_dynamic_partition_pruning
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mergejoin
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6242/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6242/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6242/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12775676 - PreCommit-HIVE-TRUNK-Build
> Hive metastore client retries don't happen consistently for all api calls
> -
>
> Key: HIVE-9642
> URL: https://issues.apache.org/jira/browse/HIVE-9642
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Xiaobing Zhou
>Assignee: Daniel Dai
> Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch, HIVE-9642.3.patch,
> HIVE-9642.4.patch, HIVE-9642.5.patch, HIVE-9642.5.patch, HIVE-9642.6.patch
>
>
> When org.apache.thrift.transport.TTransportException is thrown for issues
> like socket timeout, the retry via RetryingMetaStoreClient happens only in
> certain cases.
> Retry happens for the getDatabase call in but not for getAllDatabases().
> The reason is RetryingMetaStoreClient checks for TTransportException being
> the cause for InvocationTargetException. But in case of some calls such as
> getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a
> MetaException. We should remove this unnecessary wrapping of exceptions for
> certain functions in HMC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[
https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028680#comment-15028680
]
Hive QA commented on HIVE-9642:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12774299/HIVE-9642.4.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6134/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6134/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6134/
Messages:
{noformat}
This message was trimmed, see log for full details
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @
hive-hbase-handler ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-github-source-source/hbase-handler/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-hbase-handler
---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/hbase-handler/target/tmp
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/hbase-handler/target/warehouse
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/hbase-handler/target/tmp/conf
[copy] Copying 14 files to
/data/hive-ptest/working/apache-github-source-source/hbase-handler/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
hive-hbase-handler ---
[INFO] Compiling 17 source files to
/data/hive-ptest/working/apache-github-source-source/hbase-handler/target/test-classes
[WARNING]
/data/hive-ptest/working/apache-github-source-source/hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseSerDe.java:
Some input files use or override a deprecated API.
[WARNING]
/data/hive-ptest/working/apache-github-source-source/hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseSerDe.java:
Recompile with -Xlint:deprecation for details.
[WARNING]
/data/hive-ptest/working/apache-github-source-source/hbase-handler/src/test/org/apache/hadoop/hive/hbase/avro/Address.java:
Some input files use unchecked or unsafe operations.
[WARNING]
/data/hive-ptest/working/apache-github-source-source/hbase-handler/src/test/org/apache/hadoop/hive/hbase/avro/Address.java:
Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-hbase-handler
---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-hbase-handler ---
[INFO] Building jar:
/data/hive-ptest/working/apache-github-source-source/hbase-handler/target/hive-hbase-handler-2.0.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @
hive-hbase-handler ---
[INFO]
[INFO] --- maven-jar-plugin:2.2:test-jar (default) @ hive-hbase-handler ---
[INFO] Building jar:
/data/hive-ptest/working/apache-github-source-source/hbase-handler/target/hive-hbase-handler-2.0.0-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @
hive-hbase-handler ---
[INFO] Installing
/data/hive-ptest/working/apache-github-source-source/hbase-handler/target/hive-hbase-handler-2.0.0-SNAPSHOT.jar
to
/data/hive-ptest/working/maven/org/apache/hive/hive-hbase-handler/2.0.0-SNAPSHOT/hive-hbase-handler-2.0.0-SNAPSHOT.jar
[INFO] Installing
/data/hive-ptest/working/apache-github-source-source/hbase-handler/pom.xml to
/data/hive-ptest/working/maven/org/apache/hive/hive-hbase-handler/2.0.0-SNAPSHOT/hive-hbase-handler-2.0.0-SNAPSHOT.pom
[INFO] Installing
/data/hive-ptest/working/apache-github-source-source/hbase-handler/target/hive-hbase-handler-2.0.0-SNAPSHOT-tests.jar
to
/data/hive-ptest/working/maven/org/apache/hive/hive-hbase-handler/2.0.0-SNAPSHOT/hive-hbase-handler-2.0.0-SNAPSHOT-tests.jar
[INFO]
[INFO]
[INFO] Building Hive HCatalog 2.0.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-hcatalog ---
[INFO] Deleting
/data/hive-ptest/working/apache-github-source-source/hcatalog/target
[INFO] Deleting /data/hive-ptest/working/apache-github-source-source/hcatalog
(includes = [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @
hive-hcatalog ---
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-h
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[ https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025733#comment-15025733 ] Thejas M Nair commented on HIVE-9642: - +1 Thanks for also adding the test case. Note for others- this new patch also addresses the cases where MetaStoreClient constructor has errors. > Hive metastore client retries don't happen consistently for all api calls > - > > Key: HIVE-9642 > URL: https://issues.apache.org/jira/browse/HIVE-9642 > Project: Hive > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Xiaobing Zhou >Assignee: Daniel Dai > Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch, HIVE-9642.3.patch > > > When org.apache.thrift.transport.TTransportException is thrown for issues > like socket timeout, the retry via RetryingMetaStoreClient happens only in > certain cases. > Retry happens for the getDatabase call in but not for getAllDatabases(). > The reason is RetryingMetaStoreClient checks for TTransportException being > the cause for InvocationTargetException. But in case of some calls such as > getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a > MetaException. We should remove this unnecessary wrapping of exceptions for > certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[ https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14339215#comment-14339215 ] Thejas M Nair commented on HIVE-9642: - We should also add a unit test for this. ie, - # bring up two metstore servers # make an api call # bring down first server (thats one it connects to) # make the api call again # bring down 2nd server # bring up first server # make the api call again See TestHiveMetaStorePartitionSpecs.RunMS on starting metastore server. > Hive metastore client retries don't happen consistently for all api calls > - > > Key: HIVE-9642 > URL: https://issues.apache.org/jira/browse/HIVE-9642 > Project: Hive > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch > > > When org.apache.thrift.transport.TTransportException is thrown for issues > like socket timeout, the retry via RetryingMetaStoreClient happens only in > certain cases. > Retry happens for the getDatabase call in but not for getAllDatabases(). > The reason is RetryingMetaStoreClient checks for TTransportException being > the cause for InvocationTargetException. But in case of some calls such as > getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a > MetaException. We should remove this unnecessary wrapping of exceptions for > certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[ https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14339216#comment-14339216 ] Thejas M Nair commented on HIVE-9642: - Looks like the original change to add retries (HIVE-3400) didn't add a test case . > Hive metastore client retries don't happen consistently for all api calls > - > > Key: HIVE-9642 > URL: https://issues.apache.org/jira/browse/HIVE-9642 > Project: Hive > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch > > > When org.apache.thrift.transport.TTransportException is thrown for issues > like socket timeout, the retry via RetryingMetaStoreClient happens only in > certain cases. > Retry happens for the getDatabase call in but not for getAllDatabases(). > The reason is RetryingMetaStoreClient checks for TTransportException being > the cause for InvocationTargetException. But in case of some calls such as > getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a > MetaException. We should remove this unnecessary wrapping of exceptions for > certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[ https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14339176#comment-14339176 ] Xiaobing Zhou commented on HIVE-9642: - Thanks [~thejas]. Let's wait for HIVE-9582 committed so that I can make a patch based on that, otherwise, duplicate efforts. > Hive metastore client retries don't happen consistently for all api calls > - > > Key: HIVE-9642 > URL: https://issues.apache.org/jira/browse/HIVE-9642 > Project: Hive > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch > > > When org.apache.thrift.transport.TTransportException is thrown for issues > like socket timeout, the retry via RetryingMetaStoreClient happens only in > certain cases. > Retry happens for the getDatabase call in but not for getAllDatabases(). > The reason is RetryingMetaStoreClient checks for TTransportException being > the cause for InvocationTargetException. But in case of some calls such as > getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a > MetaException. We should remove this unnecessary wrapping of exceptions for > certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[ https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337492#comment-14337492 ] Thejas M Nair commented on HIVE-9642: - The interface IMetaStoreClient that HiveMetaStoreClient implements does throw TException. It would require changes only in few places. Probably just one file - SessionHiveMetaStoreClient.java, and maybe some places in hcatalog. But there is already one patch (HIVE-9582) that changes that. > Hive metastore client retries don't happen consistently for all api calls > - > > Key: HIVE-9642 > URL: https://issues.apache.org/jira/browse/HIVE-9642 > Project: Hive > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch > > > When org.apache.thrift.transport.TTransportException is thrown for issues > like socket timeout, the retry via RetryingMetaStoreClient happens only in > certain cases. > Retry happens for the getDatabase call in but not for getAllDatabases(). > The reason is RetryingMetaStoreClient checks for TTransportException being > the cause for InvocationTargetException. But in case of some calls such as > getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a > MetaException. We should remove this unnecessary wrapping of exceptions for > certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[
https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337229#comment-14337229
]
Xiaobing Zhou commented on HIVE-9642:
-
Thanks [~thejas] for comments. With patch V2, there are some compile issues
since the function signature changed, e.g.,
{noformat}
public List getAllTables(String dbname) throws MetaException {
{noformat}, and now
{noformat}
public List getAllTables(String dbname) throws TException {
{noformat}
This will cause many places to be changed, moreover, incompatible exception
declaration happened. I think we can resort back to patch V1. Any thoughts?
> Hive metastore client retries don't happen consistently for all api calls
> -
>
> Key: HIVE-9642
> URL: https://issues.apache.org/jira/browse/HIVE-9642
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch
>
>
> When org.apache.thrift.transport.TTransportException is thrown for issues
> like socket timeout, the retry via RetryingMetaStoreClient happens only in
> certain cases.
> Retry happens for the getDatabase call in but not for getAllDatabases().
> The reason is RetryingMetaStoreClient checks for TTransportException being
> the cause for InvocationTargetException. But in case of some calls such as
> getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a
> MetaException. We should remove this unnecessary wrapping of exceptions for
> certain functions in HMC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[
https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337111#comment-14337111
]
Hive QA commented on HIVE-9642:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12700844/HIVE-9642.2.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2869/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2869/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2869/
Messages:
{noformat}
This message was trimmed, see log for full details
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN KW_DATE StringLiteral" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN KW_NULL GREATERTHAN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN KW_NULL BITWISEXOR" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN KW_CASE KW_ARRAY" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN KW_NULL KW_BETWEEN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN KW_CASE KW_STRUCT" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:115:5:
Decision can match input such as "KW_CLUSTER KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:127:5:
Decision can match input such as "KW_PARTITION KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:138:5:
Decision can match input such as "KW_DISTRIBUTE KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:149:5:
Decision can match input such as "KW_SORT KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:166:7:
Decision can match input such as "STAR" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:194:5:
Decision can match input such as "KW_ARRAY" using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:194:5:
Decision can match input such as "KW_UNIONTYPE" using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:194:5:
Decision can match input such as "KW_STRUCT" using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:288:5:
Decision can match input such as "KW_FALSE" using multiple alternatives: 2, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:288:5:
Decision can match input such as "KW_TRUE" using multiple alternatives: 2, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:288:5:
Decision can match input such as "KW_NULL" using multiple alternatives: 1, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:420:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_OVERWRITE" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:420:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_CLUSTER
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:420:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:420:5:
Decision can match input such as "KW_BETWEEN KW_MAP LPAREN" using multiple
alternatives: 8, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:420:5:
Decision can match inpu
[jira] [Commented] (HIVE-9642) Hive metastore client retries don't happen consistently for all api calls
[ https://issues.apache.org/jira/browse/HIVE-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337036#comment-14337036 ] Thejas M Nair commented on HIVE-9642: - Looks good. Just a minor nit , the javadoc has "@inheritDoc" and this is adding a partial javadoc of this function "@throws" to it . Can you remove that "@throws" that got added ? (the full javadoc is available via the superclass). > Hive metastore client retries don't happen consistently for all api calls > - > > Key: HIVE-9642 > URL: https://issues.apache.org/jira/browse/HIVE-9642 > Project: Hive > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HIVE-9642.1.patch, HIVE-9642.2.patch > > > When org.apache.thrift.transport.TTransportException is thrown for issues > like socket timeout, the retry via RetryingMetaStoreClient happens only in > certain cases. > Retry happens for the getDatabase call in but not for getAllDatabases(). > The reason is RetryingMetaStoreClient checks for TTransportException being > the cause for InvocationTargetException. But in case of some calls such as > getAllDatabases in HiveMetastoreClient, all exceptions get wrapped in a > MetaException. We should remove this unnecessary wrapping of exceptions for > certain functions in HMC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
