[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Status: Patch Available (was: Open) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, HIVE-16530.02.patch, > Incremental_ReplDump_Console_Log.png, Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: HIVE-16530.02.patch > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, HIVE-16530.02.patch, > Incremental_ReplDump_Console_Log.png, Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Status: Open (was: Patch Available) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, > Incremental_ReplDump_Console_Log.png, Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: (was: HIVE-16530.02.patch) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, > Incremental_ReplDump_Console_Log.png, Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: HIVE-16530.02.patch Added 02.patch with following update. - Rebased with master code to avoid merge conflicts. - Added database name in the log for incremental repl load case. > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, HIVE-16530.02.patch, > Incremental_ReplDump_Console_Log.png, Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Status: Patch Available (was: Open) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, HIVE-16530.02.patch, > Incremental_ReplDump_Console_Log.png, Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: Incremental_ReplLoad_Console_Log.png > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, > Incremental_ReplDump_Console_Log.png, Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Status: Open (was: Patch Available) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, > Incremental_ReplDump_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: (was: Incremental_ReplLoad_Console_Log.png) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, > Incremental_ReplDump_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: HIVE-16530.01.patch > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, > Incremental_ReplDump_Console_Log.png, Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: (was: HIVE-16530.01.patch) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, Incremental_ReplDump_Console_Log.png, > Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: Incremental_ReplLoad_Console_Log.png Incremental_ReplDump_Console_Log.png Bootstrap_ReplLoad_Console_Log.png Bootstrap_ReplDump_Console_Log.png > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, Incremental_ReplDump_Console_Log.png, > Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: (was: Bootstrap_ReplDump_Console_Log.png) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: HIVE-16530.01.patch > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: Bootstrap_ReplDump_Console_Log.png > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: Bootstrap_ReplDump_Console_Log.png, HIVE-16530.01.patch > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: HIVE-16530.01.patch > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: HIVE-16530.01.patch > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Status: Patch Available (was: Open) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: HIVE-16530.01.patch > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Status: Open (was: Patch Available) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: (was: HIVE-16530.01.patch) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: (was: Incremental_ReplLoad_Console_Log.png) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: HIVE-16530.01.patch > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: (was: Bootstrap_ReplLoad_Console_Log.png) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: HIVE-16530.01.patch > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: (was: Bootstrap_ReplDump_Console_Log.png) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: HIVE-16530.01.patch > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Labels: DR replication (was: ) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: HIVE-16530.01.patch > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: (was: Incremental_ReplDump_Console_Log.png) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Attachments: HIVE-16530.01.patch > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Status: Patch Available (was: Open) > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, > Incremental_ReplDump_Console_Log.png, Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Attachment: Incremental_ReplLoad_Console_Log.png Incremental_ReplDump_Console_Log.png Bootstrap_ReplLoad_Console_Log.png Bootstrap_ReplDump_Console_Log.png HIVE-16530.01.patch Added 01.patch: - Added progress logs for REPL DUMP and REPL LOAD. - Considered the log table level for bootstrap and event level for incremental. - Object level log is not considered as it will be taken as follow up task. - No new test added as the log comes in console and the same is attached here for review. > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Attachments: Bootstrap_ReplDump_Console_Log.png, > Bootstrap_ReplLoad_Console_Log.png, HIVE-16530.01.patch, > Incremental_ReplDump_Console_Log.png, Incremental_ReplLoad_Console_Log.png > > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Description: This is the log format that is being proposed for Hive Repl query logs For bootstrap case: Hive will log a message for each object as it is being bootstrapped and it will be in the following sequence - Tables first (views are tables for this purpose) at time including partitions (depth first), followed by functions, constraints - The ordering is based on the ordering of listStatus API of HDFS - For each object, a message at the beginning of the replication will be logged - Every partition bootstrapped will be followed by a message saying the number of partitions bootstrapped so far (for the table) and the partition name - And a message at the end of bootstrap of an object Incremental case: - We will have DB Name, event id and event type will be part of the log header (for debugging/troubleshooting) - We will have information of current event ID and total number of events to replicate for every event replicated. was: This is the log format that is being proposed for Hive Repl query logs For bootstrap case: Hive will log a message for each object as it is being bootstrapped And it will be in the following sequence - Tables first (views are tables for this purpose) at time including partitions (depth first), followed by functions, constraints - The ordering is based on the ordering of listStatus API of HDFS - For each object, a message at the beginning of the replication will be logged - Every partition bootstrapped will be followed by a message saying the number of partitions bootstrapped so far (for the table) and the partition name - And a message at the end of bootstrap of an object Incremental case: - We will have DB Name, event id and event type will be part of the log header (for debugging/troubleshooting) - We will have information of current event ID and total number of events to replicate for every event replicated. > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped and it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16530) Improve execution logs for REPL commands
[ https://issues.apache.org/jira/browse/HIVE-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-16530: Description: This is the log format that is being proposed for Hive Repl query logs For bootstrap case: Hive will log a message for each object as it is being bootstrapped And it will be in the following sequence - Tables first (views are tables for this purpose) at time including partitions (depth first), followed by functions, constraints - The ordering is based on the ordering of listStatus API of HDFS - For each object, a message at the beginning of the replication will be logged - Every partition bootstrapped will be followed by a message saying the number of partitions bootstrapped so far (for the table) and the partition name - And a message at the end of bootstrap of an object Incremental case: - We will have DB Name, event id and event type will be part of the log header (for debugging/troubleshooting) - We will have information of current event ID and total number of events to replicate for every event replicated. was: This is the log format that is being proposed for Hive Repl query logs For bootstrap case: Hive will log a message for each object as it is being bootstrapped And it will be in the following sequence - Tables first (views are tables for this purpose) at at time including partitions (depth first), followed by functions, constraints - The ordering is based on the ordering of listStatus API of HDFS - For each object, a message at the beginning of the replication will be logged - Every partition bootstrapped will be followed by a message saying the number of partitions bootstrapped so far (for the table) and the partition name - And a message at the end of bootstrap of an object Incremental case: - We will have DB Name, event id and event type will be part of the log header (for debugging/troubleshooting) - We will have information of current event ID and total number of events to replicate for every event replicated. > Improve execution logs for REPL commands > > > Key: HIVE-16530 > URL: https://issues.apache.org/jira/browse/HIVE-16530 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.2.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > > This is the log format that is being proposed for Hive Repl query logs > For bootstrap case: > Hive will log a message for each object as it is being bootstrapped And it > will be in the following sequence > - Tables first (views are tables for this purpose) at time including > partitions (depth first), > followed by functions, constraints > - The ordering is based on the ordering of listStatus API of HDFS > - For each object, a message at the beginning of the replication will be > logged > - Every partition bootstrapped will be followed by a message saying the > number of partitions bootstrapped so far (for the table) and the partition > name > - And a message at the end of bootstrap of an object > Incremental case: > - We will have DB Name, event id and event type will be part of the log > header (for debugging/troubleshooting) > - We will have information of current event ID and total number of events to > replicate for every event replicated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
