[
https://issues.apache.org/jira/browse/HIVE-20953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16696565#comment-16696565
]
Ashutosh Bapat commented on HIVE-20953:
---------------------------------------
[~maheshk114], I tried the attached patch [^test_func_load_failure_retry.patch]
but it's failing at line 204 (in code). There it expects that only f1 will get
loaded, but show functions lists both f1 and f2.
2018-11-22T23:05:55,347 INFO [main] parse.TestReplicationScenarios: Expecting
replicated_testBootstrapReplLoadRetryAfterFailureForFunctions_1542956751333.f1
2018-11-22T23:05:55,347 INFO [main] parse.TestReplicationScenarios: Got
[replicated_testbootstrapreplloadretryafterfailureforfunctions_1542956751333.f1,
replicated_testbootstrapreplloadretryafterfailureforfunctions_1542956751333.f2]
>From logs I can see that an exception is thrown while loading f2 as expected
2018-11-22T23:05:55,270 WARN [pool-26-thread-7] parse.TestReplicationScenarios:
Verified - Function : f2
2018-11-22T23:05:55,273 DEBUG [pool-26-thread-7] pool.PoolBase: HikariPool-1 -
Reset (autoCommit) on connection
org.apache.derby.impl.jdbc.EmbedConnection@46773139 (XID = 166), (SESSIONID =
1), (DATABASE = memory:/work/hive20953/cr/itests/hive-unit/target/tmp/APP),
(DRDAID = null)
2018-11-22T23:05:55,276 ERROR [pool-26-thread-7] metastore.RetryingHMSHandler:
MetaException(message:InjectableBehaviourObjectStore: Invalid Create Function
operation on DB:
replicated_testBootstrapReplLoadRetryAfterFailureForFunctions_1542956751333
function: f2)
at
org.apache.hadoop.hive.metastore.InjectableBehaviourObjectStore.createFunction(InjectableBehaviourObjectStore.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
So far I don't have any clue as to why both the functions are seen on the
replica even when the load has failed.
> Fix testcase
> TestReplicationScenariosAcrossInstances#testBootstrapReplLoadRetryAfterFailureForPartitions
> to not depend upon the order in which objects get loaded
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-20953
> URL: https://issues.apache.org/jira/browse/HIVE-20953
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Affects Versions: 4.0.0
> Reporter: Ashutosh Bapat
> Assignee: Ashutosh Bapat
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20953.01, HIVE-20953.02,
> test_func_load_failure_retry.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The testcase is intended to test REPL LOAD with retry. The test creates a
> partitioned table and a function in the source database and loads those to
> the replica. The first attempt to load a dump is intended to fail while
> loading one of the partitions. Based on the order in which the objects get
> loaded, if the function is queued after the table, it will not be available
> in replica after the load failure. But if it's queued before the table, it
> will be available in replica even after the load failure. The test assumes
> the later case, which may not be true always.
> Hence fix the testcase to order the objects by a fixed ordering. By setting
> hive.in.repl.test.files.sorted to true, the objects are ordered by the
> directory names. This ordering is available with minimal changes for testing,
> hence we use it. With this ordering a function gets loaded before a table. So
> changed the test to not expect the function to be available after the failed
> load, but be available after the retry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
