[
https://issues.apache.org/jira/browse/HIVE-21886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16874541#comment-16874541
]
Hive QA commented on HIVE-21886:
--------------------------------
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17777/dev-support/hive-personality.sh
|
| git revision | master / e000e2f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17777/yetus.txt |
| Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
> REPL - With table list - Handle rename events during replace policy
> -------------------------------------------------------------------
>
> Key: HIVE-21886
> URL: https://issues.apache.org/jira/browse/HIVE-21886
> Project: Hive
> Issue Type: Sub-task
> Components: repl
> Reporter: mahesh kumar behera
> Assignee: mahesh kumar behera
> Priority: Major
> Labels: DR, Replication, pull-request-available
> Attachments: HIVE-21886.01.patch, HIVE-21886.02.patch,
> HIVE-21886.03.patch, HIVE-21886.04.patch, HIVE-21886.04.patch
>
> Time Spent: 11h 10m
> Remaining Estimate: 0h
>
> If some rename events are found to be dumped and replayed while replace
> policy is getting executed, it needs to take care of the policy inclusion in
> both the policy for each table name.
> 1. Create a list of tables to be bootstrapped.
> 2. During handling of alter table, if the alter type is rename
> 1. If the old table name is present in the list of table to be
> bootstrapped, remove it.
> 2. If the new table name, matches the new policy, add it to the list
> of tables to be bootstrapped.
> 3. If the old table does not match the old policy drop it, even if the
> table is not present at target.
> 3. During handling of drop table
> 1. if the table is in the list of tables to be bootstrapped, then
> remove it and ignore the event.
> 4. During other event handling
> 1. if the table is there in the list of tables to be bootstrapped,
> then ignore the event.
> 2. If the new policy does not match the table name, then ignore the
> event.
>
> Rename handling during replace policy
> # Old name not matching old policy – The old table will not be there at the
> target cluster. The table will not be returned by get-all-table.
> ## Old name is not matching new policy
> ### New name not matching old policy
> #### New name not matching new policy
> ***** Ignore the event, no need to do anything.
> #### New name matching new policy
> ***** The table will be returned by get-all-table. Replace policy handler
> will bootstrap this table as its matching new policy and not matching old
> policy.
> ***** All the future events will be ignored as part of check added by
> replace policy handling.
> ***** All the event with old table name will anyways be ignored as the old
> name is not matching the new policy.
> ### New name matching old policy
> #### New name not matching new policy
> ***** As the new name is not matching the new policy, the table need not be
> replicated.
> ***** As the old name is not matching the new policy, the rename events will
> be ignored.
> ***** So nothing to be done for this scenario.
> #### New name matching new policy
> ***** As the new name is matching both old and new policy, replace handler
> will not bootstrap the table.
> ***** Add the table to the list of tables to be bootstrapped.
> ***** Ignore all the events with new name.
> ***** If there is a drop event for the table (with new name), then remove
> the table from the the list of table to be bootstrapped.
> ***** In case of rename event (double rename)
> ****** If the new name satisfies the table pattern, then add the new name to
> the list of tables to be bootstrapped and remove the old name from the list
> of tables to be bootstrapped.
> ****** If the new name does not satisfies then just removed the table name
> from the list of tables to be bootstrapped.
> ## Old name is matching new policy – As per replace policy handler, which
> checks based on old table, the table should be bootstrapped and event should
> be ignored. But rename handler should decide based on new name.The old table
> name will not be returned by get-all-table, so replace handler will not d
> anything for the old table.
> ### New name not matching old policy
> #### New name not matching new policy
> ***** As the old table is not there at target and new name is not matching
> new policy. Ignore the event.
> ***** No need to add the table to the list of tables to be bootstrapped.
> ***** All the subsequent events will be ignored as the new name is not
> matching the new policy.
> #### New name matching new policy
> ***** As the new name is not matching old policy but matching new policy,
> the table will be bootstrapped by replace policy handler. So rename event
> need not add this table to list of table to be bootstrapped.
> ***** All the future events will be ignored by replace policy handler.
> ***** For rename event (double rename)
> ****** If there is a rename, the table (with intermittent new name) will not
> be present and thus replace handler will not bootstrap the table.
> ****** So if the new name (the latest one) is matching the new policy, then
> add it to the list of table to be bootstrapped.
> ****** And If the new name (the latest one) is not matching the new policy,
> then just ignore the event as the intermittent new name would not have added
> to the list of table to be bootstrapped.
> ### New name matching old policy
> #### New name not matching new policy
> ***** Dump the event. The table will be dropped by repl load at the target.
> #### New name matching new policy
> ***** Replace handler will not bootstrap this table as the new name is
> matching both policies.
> ***** As old name is not matching the old policy, the table will not be
> there at target. The rename event should add the new name to the list of
> table to be bootstrapped.
> ***** Subsequent events with new table name should be ignored.
> ***** Drop events should not be ignored as if the table is present during
> bootstrapped, then its a new table and thus should be dropped.
> ***** In case of rename event (double rename)
> ****** If the new name satisfies the table pattern, then add the new name to
> the list of tables to be bootstrapped and remove the old name from the list
> of tables to be bootstrapped.
> ****** If the new name does not satisfies then just removed the table name
> from the list of tables to be bootstrapped.
> # Old name is matching old policy – The old table will be there at the
> target cluster. The table will not be returned by get-all-table. Repl load
> should delete the old table as it is not matching the new policy.
> ## Old name is not matching new policy
> ### New name not matching old policy
> #### New name not matching new policy
> ***** Nothing to be done. Ignore the event.
> #### New name matching new policy
> ***** As the new name is not matching old policy but matching new policy,
> the table will be bootstrapped by replace policy handler. So rename event
> need not add this table to list of table to be bootstrapped.
> ***** All the future events will be ignored by replace policy handler.
> ***** For rename event (double rename)
> ****** If there is a rename, the table (with intermittent new name) will not
> be present and thus replace handler will not bootstrap the table.
> ****** So if the new name (the latest one) is matching the new policy, then
> add it to the list of table to be bootstrapped.
> ****** And If the new name (the latest one) is not matching the new policy,
> then just ignore the event as the intermittent new name would not have added
> to the list of table to be bootstrapped.
> ### New name matching old policy
> #### New name not matching new policy
> ***** Table with new name will be dropped by repl load
> ***** Along with other event, ignore the rename event also.
> #### New name matching new policy
> ***** As the new name is matching both old and new policy, replace handler
> will not bootstrap the table.
> ***** Add the table to the list of tables to be bootstrapped.
> ***** Ignore all the events with new name.
> ***** If there is a drop event for the table (with new name), then remove
> the table from the the list of table to be bootstrapped.
> ***** In case of rename event (double rename)
> ****** If the new name satisfies the table pattern, then add the new name to
> the list of tables to be bootstrapped and remove the old name from the list
> of tables to be bootstrapped.
> ****** If the new name does not satisfies then just removed the table name
> from the list of tables to be bootstrapped.
> ## Old name is matching new policy
> ### New name not matching old policy
> #### New name not matching new policy
> ***** The old table needs to be dropped at target. Ignore this event, as the
> old table is not matching the new policy, it will be dropped by repl load.
> #### New name matching new policy
> ***** Allow the event to dump and replayed at target.
> ***** Allow further events to be handled as usual.
> ***** In case of rename event (double rename)
> ****** If the latest new name is matching the new policy, then keep it as is
> it. Let rename event replayed at target.
> ****** If the latest new name is not matching the new policy, then change
> the rename event to drop event.
> ### New name matching old policy
> #### New name not matching new policy
> ##### Nothing to be done.
> #### New name matching new policy
> ##### Add the table name to the list of tables to be bootstrapped.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
