[jira] [Assigned] (HIVE-17579) repl load without providing the database name in the command fails.

2017-09-22 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-17579: -- Assignee: anishek > repl load without providing the database name in the command fails. >

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel

2017-09-22 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Attachment: HIVE-17426.6.patch rebased from master > Execution framework in hive to run tasks in parallel >

[jira] [Commented] (HIVE-17426) Execution framework in hive to run tasks in parallel

2017-09-20 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172840#comment-16172840 ] anishek commented on HIVE-17426: All tasks which share some sort of threadlocal or other variables which

[jira] [Updated] (HIVE-17512) Not use doAs if distcp privileged user same as user running hive

2017-09-18 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17512: --- Attachment: HIVE-17512.2.patch reattaching the patch so build kicks off > Not use doAs if distcp privileged

[jira] [Updated] (HIVE-17512) Not use doAs if distcp privileged user same as user running hive

2017-09-15 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17512: --- Attachment: HIVE-17512.2.patch rebased from master, test failures in the previous builds are failing on

[jira] [Commented] (HIVE-17527) Support replication for rename/move table across database

2017-09-15 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167615#comment-16167615 ] anishek commented on HIVE-17527: +1 cc [~thejas]/[~daijy] > Support replication for rename/move table

[jira] [Commented] (HIVE-16898) Validation of source file after distcp in repl load

2017-09-15 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167575#comment-16167575 ] anishek commented on HIVE-16898: [~daijy] can you please provide a pull request for the same. >

[jira] [Assigned] (HIVE-17539) User impersonation failure is not propagated by server as a failure to client

2017-09-15 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-17539: -- > User impersonation failure is not propagated by server as a failure to client >

[jira] [Updated] (HIVE-17512) Not use doAs if distcp privileged user same as user running hive

2017-09-14 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17512: --- Attachment: HIVE-17512.1.patch > Not use doAs if distcp privileged user same as user running hive >

[jira] [Updated] (HIVE-17512) Not use doAs if distcp privileged user same as user running hive

2017-09-14 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17512: --- Attachment: (was: HIVE-17512.1.patch) > Not use doAs if distcp privileged user same as user running hive

[jira] [Updated] (HIVE-17512) Not use doAs if distcp privileged user same as user running hive

2017-09-14 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17512: --- Attachment: HIVE-17512.1.patch > Not use doAs if distcp privileged user same as user running hive >

[jira] [Updated] (HIVE-17512) Not use doAs if distcp privileged user same as user running hive

2017-09-14 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17512: --- Status: Patch Available (was: Open) > Not use doAs if distcp privileged user same as user running hive >

[jira] [Commented] (HIVE-17428) REPL LOAD of ALTER_PARTITION event doesn't create import tasks if the partition doesn't exist during analyze phase.

2017-09-13 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165760#comment-16165760 ] anishek commented on HIVE-17428: +1 cc [~thejas]/[~daijy] > REPL LOAD of ALTER_PARTITION event doesn't

[jira] [Assigned] (HIVE-17512) Not use doAs if distcp privileged user same as user running hive

2017-09-12 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-17512: -- > Not use doAs if distcp privileged user same as user running hive >

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel

2017-09-11 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Description: the execution framework currently only runs MR / Spark Tasks in parallel when {{set

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel

2017-09-11 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Description: the execution framework currently only runs MR Tasks in parallel when {{set

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel

2017-09-11 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Summary: Execution framework in hive to run tasks in parallel (was: Execution framework in hive to run tasks

[jira] [Commented] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-09-11 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16160971#comment-16160971 ] anishek commented on HIVE-17426: *

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-09-10 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: HIVE-16886.8.patch rebased from master [~daijy] > HMS log notifications may have duplicated

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-09-10 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Attachment: HIVE-17426.5.patch removed graphstream and correct some comments > Execution framework in hive

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-09-08 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Attachment: HIVE-17426.4.patch docs, added test case, changed default number of tasks in hiveConf for

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-09-08 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Attachment: HIVE-17426.3.patch ability to print graphs for tasks along with fix to allow parallel task to run

[jira] [Updated] (HIVE-17410) repl load task during subsequent DAG generation does not start from the last partition processed

2017-09-07 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17410: --- Attachment: HIVE-17410.3.patch fixing comments from sankar , Please review [~sankarh] > repl load task

[jira] [Assigned] (HIVE-17484) Bootstrap load should no have metadata only operations for partitions

2017-09-07 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-17484: -- > Bootstrap load should no have metadata only operations for partitions >

[jira] [Commented] (HIVE-17183) Disable rename operations during bootstrap dump

2017-09-07 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156932#comment-16156932 ] anishek commented on HIVE-17183: +1 cc [~thejas]/[~daijy] > Disable rename operations during bootstrap

[jira] [Commented] (HIVE-17421) Clear incorrect stats after replication

2017-09-07 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156594#comment-16156594 ] anishek commented on HIVE-17421: +1 > Clear incorrect stats after replication >

[jira] [Comment Edited] (HIVE-17410) repl load task during subsequent DAG generation does not start from the last partition processed

2017-09-06 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155036#comment-16155036 ] anishek edited comment on HIVE-17410 at 9/6/17 9:07 AM: [~thejas] i think you were

[jira] [Commented] (HIVE-17410) repl load task during subsequent DAG generation does not start from the last partition processed

2017-09-06 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155036#comment-16155036 ] anishek commented on HIVE-17410: Sankar Hariappan/Thejas M Nair/Daniel Dai please review, second reminder

[jira] [Updated] (HIVE-17410) repl load task during subsequent DAG generation does not start from the last partition processed

2017-09-06 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17410: --- Attachment: HIVE-17410.2.patch rebased master > repl load task during subsequent DAG generation does not

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-09-05 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Attachment: HIVE-17426.2.patch sending in 0 as current transaction id when not acid type use case >

[jira] [Commented] (HIVE-17420) bootstrap - get replid before object dump

2017-09-05 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153376#comment-16153376 ] anishek commented on HIVE-17420: [~thejas] looks like the above patch is committed with HIVE-17426 bug

[jira] [Commented] (HIVE-17420) bootstrap - get replid before object dump

2017-09-05 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153364#comment-16153364 ] anishek commented on HIVE-17420: hey [~thejas] i couldnt find this commit on master , can you please

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-09-05 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Attachment: (was: HIVE-17426.1.patch) > Execution framework in hive to run tasks in parallel other than

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-09-05 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Attachment: HIVE-17426.1.patch attaching the patch with rebase on master > Execution framework in hive to

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-09-05 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Attachment: HIVE-17426.1.patch associating the currentTransactionId with the correct class > Execution

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-09-01 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Status: Patch Available (was: Open) > Execution framework in hive to run tasks in parallel other than MR

[jira] [Updated] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-09-01 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17426: --- Attachment: HIVE-17426.0.patch do an initial change to see, what is the impact. > Execution framework in

[jira] [Updated] (HIVE-17420) bootstrap - get replid before object dump

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17420: --- Attachment: HIVE-17420.2.patch > bootstrap - get replid before object dump >

[jira] [Assigned] (HIVE-17426) Execution framework in hive to run tasks in parallel other than MR Tasks

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-17426: -- > Execution framework in hive to run tasks in parallel other than MR Tasks >

[jira] [Commented] (HIVE-17420) bootstrap - get replid before object dump

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149518#comment-16149518 ] anishek commented on HIVE-17420: Review please [~sankarh]/[~thejas]/[~daijy] > bootstrap - get replid

[jira] [Commented] (HIVE-17367) IMPORT table doesn't load from data dump if a metadata-only dump was already imported.

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149516#comment-16149516 ] anishek commented on HIVE-17367: +1 cc [~thejas]/[~daijy] > IMPORT table doesn't load from data dump if

[jira] [Updated] (HIVE-17420) bootstrap - get replid before object dump

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17420: --- Status: Patch Available (was: In Progress) > bootstrap - get replid before object dump >

[jira] [Work started] (HIVE-17420) bootstrap - get replid before object dump

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-17420 started by anishek. -- > bootstrap - get replid before object dump > - > >

[jira] [Updated] (HIVE-17420) bootstrap - get replid before object dump

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17420: --- Attachment: HIVE-17420.1.patch > bootstrap - get replid before object dump >

[jira] [Updated] (HIVE-17420) bootstrap - get replid before object dump

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17420: --- Description: when we do the bootstrap dump, the idea is that we take note of the replication id of a object

[jira] [Assigned] (HIVE-17420) bootstrap - get replid before object dump

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-17420: -- Assignee: anishek > bootstrap - get replid before object dump >

[jira] [Commented] (HIVE-17408) replication distcp should only be invoked if number of files AND file size cross configured limits

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149290#comment-16149290 ] anishek commented on HIVE-17408: Thanks for the license Header and commit [~thejas] > replication distcp

[jira] [Commented] (HIVE-17367) IMPORT table doesn't load from data dump if a metadata-only dump was already imported.

2017-08-31 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149237#comment-16149237 ] anishek commented on HIVE-17367: [~sankarh]can you rebase it, does not apply cleanly currently. Also can

[jira] [Commented] (HIVE-16904) during repl load for large number of partitions the metadata file can be huge and can lead to out of memory

2017-08-30 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148234#comment-16148234 ] anishek commented on HIVE-16904: On internal runs we saw that for 1 partitions with one file each it

[jira] [Commented] (HIVE-17408) replication distcp should only be invoked if number of files AND file size cross configured limits

2017-08-30 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148023#comment-16148023 ] anishek commented on HIVE-17408: *

[jira] [Commented] (HIVE-17410) repl load task during subsequent DAG generation does not start from the last partition processed

2017-08-30 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147734#comment-16147734 ] anishek commented on HIVE-17410: *

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-30 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: HIVE-16886.7.patch fixing log comments > HMS log notifications may have duplicated event IDs if

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-30 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147667#comment-16147667 ] anishek commented on HIVE-16886: * org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[transform_acid]

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-30 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147592#comment-16147592 ] anishek commented on HIVE-16886: [~akolb] {{SQLGenerator}} is a artifact of the ACID implementation and we

[jira] [Commented] (HIVE-17100) Improve HS2 operation logs for REPL commands.

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146548#comment-16146548 ] anishek commented on HIVE-17100: +1 only issue left is the dependency logger added to root, which i think

[jira] [Commented] (HIVE-17410) repl load task during subsequent DAG generation does not start from the last partition processed

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146373#comment-16146373 ] anishek commented on HIVE-17410: [~sankarh]/[~thejas]/[~daijy] please review > repl load task during

[jira] [Updated] (HIVE-17410) repl load task during subsequent DAG generation does not start from the last partition processed

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17410: --- Attachment: HIVE-17410.1.patch > repl load task during subsequent DAG generation does not start from the last

[jira] [Updated] (HIVE-17410) repl load task during subsequent DAG generation does not start from the last partition processed

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17410: --- Status: Patch Available (was: In Progress) > repl load task during subsequent DAG generation does not start

[jira] [Work started] (HIVE-17410) repl load task during subsequent DAG generation does not start from the last partition processed

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-17410 started by anishek. -- > repl load task during subsequent DAG generation does not start from the last > partition

[jira] [Assigned] (HIVE-17410) repl load task during subsequent DAG generation does not start from the last partition processed

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-17410: -- Assignee: anishek > repl load task during subsequent DAG generation does not start from the last >

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: HIVE-16886.6.patch fixing configuration object reference > HMS log notifications may have

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: (was: HIVE-16886.6.patch) > HMS log notifications may have duplicated event IDs if multiple

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: HIVE-16886.6.patch fixing the Configuration object reference > HMS log notifications may have

[jira] [Resolved] (HIVE-17298) export when running distcp for large number of files should not run as privileged user

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek resolved HIVE-17298. Resolution: Duplicate > export when running distcp for large number of files should not run as >

[jira] [Commented] (HIVE-17408) replication distcp should only be invoked if number of files AND file size cross configured limits

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145975#comment-16145975 ] anishek commented on HIVE-17408: [~sankarh]/[~thejas]/[~daijy] please review > replication distcp should

[jira] [Updated] (HIVE-17408) replication distcp should only be invoked if number of files AND file size cross configured limits

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17408: --- Status: Patch Available (was: In Progress) > replication distcp should only be invoked if number of files

[jira] [Updated] (HIVE-17408) replication distcp should only be invoked if number of files AND file size cross configured limits

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17408: --- Attachment: HIVE-17408.1.patch > replication distcp should only be invoked if number of files AND file size

[jira] [Work started] (HIVE-17408) replication distcp should only be invoked if number of files AND file size cross configured limits

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-17408 started by anishek. -- > replication distcp should only be invoked if number of files AND file size > cross

[jira] [Assigned] (HIVE-17408) replication distcp should only be invoked if number of files AND file size cross configured limits

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-17408: -- > replication distcp should only be invoked if number of files AND file size > cross configured limits >

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: HIVE-16886.5.patch configurations for sleep and retry intervals for notification sequence lock >

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: (was: HIVE-16886.5.patch) > HMS log notifications may have duplicated event IDs if multiple

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: HIVE-16886.5.patch adding configurations to manage retry limit and sleep time for acquiring the

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145845#comment-16145845 ] anishek commented on HIVE-16886: add {{NOTIFICATION_SEQUENCE_LOCK_MAX_RETRIES}} and

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-29 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145772#comment-16145772 ] anishek commented on HIVE-16886: [~akolb] the transaction for handling the dbnotification update is a

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-28 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144587#comment-16144587 ] anishek commented on HIVE-16886: Failing tests are from older builds. [~thejas]/[~daijy]/[~sankarh]

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-28 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: HIVE-16886.4.patch fixing oracle schema upgrades > HMS log notifications may have duplicated

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-28 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: HIVE-16886.3.patch fixing errors for derby schema and removing the java lock for updating

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-28 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143952#comment-16143952 ] anishek commented on HIVE-16886: [~lina.li] changes to notification log are in nested transactions with

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-25 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: HIVE-16886.2.patch > HMS log notifications may have duplicated event IDs if multiple HMS are >

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-24 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140650#comment-16140650 ] anishek commented on HIVE-16886: [~akolb] the {{select for update}} will be on {{notification_sequence}}

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-23 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139346#comment-16139346 ] anishek commented on HIVE-16886: Thinking more on this, might be we just do "select for update' for

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-23 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139246#comment-16139246 ] anishek commented on HIVE-16886: link seems to be discussing sequences and like i mentioned earlier the

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-23 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138921#comment-16138921 ] anishek commented on HIVE-16886: 1. this will not happen as the id will *not* be allocated before commit,

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-23 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138741#comment-16138741 ] anishek commented on HIVE-16886: auto increments can have holes if * a transaction was aborted. * or the

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-22 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137596#comment-16137596 ] anishek commented on HIVE-16886: [~spena]/[~akolb] I agree to the points you have made and i have been

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-21 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Status: Patch Available (was: Open) > HMS log notifications may have duplicated event IDs if multiple HMS

[jira] [Updated] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-21 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-16886: --- Attachment: HIVE-16886.1.patch attaching patch for test run. > HMS log notifications may have duplicated

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-21 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135840#comment-16135840 ] anishek commented on HIVE-16886: [~spena] thanks for your insights. * Sequence is not present in Mysql

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-21 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135433#comment-16135433 ] anishek commented on HIVE-16886: I am just going to provide some more detail information so that we all

[jira] [Comment Edited] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-16 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128384#comment-16128384 ] anishek edited comment on HIVE-16886 at 8/16/17 8:35 AM: - Yeh we can do that

[jira] [Comment Edited] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-16 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128384#comment-16128384 ] anishek edited comment on HIVE-16886 at 8/16/17 8:35 AM: - Yeh we can do that

[jira] [Comment Edited] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-16 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128384#comment-16128384 ] anishek edited comment on HIVE-16886 at 8/16/17 6:21 AM: - Yeh we can do that

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-16 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128384#comment-16128384 ] anishek commented on HIVE-16886: Yeh we can do that though we have to explicitly parse and typecast the

[jira] [Assigned] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-11 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-16886: -- Assignee: anishek > HMS log notifications may have duplicated event IDs if multiple HMS are > running

[jira] [Commented] (HIVE-16990) REPL LOAD should update last repl ID only after successful copy of data files.

2017-08-11 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122985#comment-16122985 ] anishek commented on HIVE-16990: +1 [~thejas]/[~daijy] please commit > REPL LOAD should update last repl

[jira] [Updated] (HIVE-17298) export when running distcp for large number of files should not run as privileged user

2017-08-10 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17298: --- Attachment: HIVE-17298.1.patch > export when running distcp for large number of files should not run as >

[jira] [Work started] (HIVE-17298) export when running distcp for large number of files should not run as privileged user

2017-08-10 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-17298 started by anishek. -- > export when running distcp for large number of files should not run as > privileged user

[jira] [Updated] (HIVE-17298) export when running distcp for large number of files should not run as privileged user

2017-08-10 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-17298: --- Summary: export when running distcp for large number of files should not run as privileged user (was:

[jira] [Assigned] (HIVE-17298) export when running distcp for large number of files runs as privileged user from hiveconf

2017-08-10 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek reassigned HIVE-17298: -- > export when running distcp for large number of files runs as privileged user > from hiveconf >

[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently

2017-08-10 Thread anishek (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121179#comment-16121179 ] anishek commented on HIVE-16886: for the supported database NL_ID will be generated either using the

<    1   2   3   4   5   6   7   8   9   >