[jira] [Commented] (HIVE-16573) In-place update for HoS can't be disabled

2017-05-03 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994393#comment-15994393 ] Rui Li commented on HIVE-16573: --- In {{InPlaceUpdate.canRenderInPlace}}, we don't check the c

[jira] [Assigned] (HIVE-16573) In-place update for HoS can't be disabled

2017-05-03 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned HIVE-16573: - > In-place update for HoS can't be disabled > - > > K

[jira] [Commented] (HIVE-14412) Add a timezone-aware timestamp

2017-05-02 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994346#comment-15994346 ] Rui Li commented on HIVE-14412: --- Latest failure are not related. I've updated the v11 patch

[jira] [Commented] (HIVE-16047) Shouldn't try to get KeyProvider unless encryption is enabled

2017-05-02 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994247#comment-15994247 ] Rui Li commented on HIVE-16047: --- Patch reverted in master and branch-2.3. > Shouldn't try t

[jira] [Commented] (HIVE-16047) Shouldn't try to get KeyProvider unless encryption is enabled

2017-05-02 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994215#comment-15994215 ] Rui Li commented on HIVE-16047: --- Hi [~spena], I'm not aware the patch is not in branch-2.2.

[jira] [Updated] (HIVE-14412) Add a timezone-aware timestamp

2017-05-02 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-14412: -- Attachment: HIVE-14412.11.patch Rebase > Add a timezone-aware timestamp > -- > >

[jira] [Commented] (HIVE-16552) Limit the number of tasks a Spark job may contain

2017-05-02 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992478#comment-15992478 ] Rui Li commented on HIVE-16552: --- [~xuefuz], thanks for the explanations. I agree this propos

[jira] [Commented] (HIVE-16552) Limit the number of tasks a Spark job may contain

2017-05-01 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992308#comment-15992308 ] Rui Li commented on HIVE-16552: --- [~xuefuz], my point is a large number of tasks doesn't mean

[jira] [Updated] (HIVE-15859) HoS: Write RPC messages in event loop

2017-05-01 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-15859: -- Fix Version/s: 1.3.0 > HoS: Write RPC messages in event loop > - > >

[jira] [Commented] (HIVE-15859) HoS: Write RPC messages in event loop

2017-05-01 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992299#comment-15992299 ] Rui Li commented on HIVE-15859: --- Pushed to branch-1. Sorry about the delay. > HoS: Write RP

[jira] [Commented] (HIVE-16552) Limit the number of tasks a Spark job may contain

2017-05-01 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992287#comment-15992287 ] Rui Li commented on HIVE-16552: --- Hi [~xuefuz], could you give some example when we want to p

[jira] [Commented] (HIVE-16456) Kill spark job when InterruptedException happens or driverContext.isShutdown is true.

2017-05-01 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992120#comment-15992120 ] Rui Li commented on HIVE-16456: --- Thanks [~zxu] for working on this. One question is could yo

[jira] [Commented] (HIVE-14412) Add a timezone-aware timestamp

2017-04-26 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985892#comment-15985892 ] Rui Li commented on HIVE-14412: --- As discussed in HIVE-16418, we won't store the TZ part and

[jira] [Updated] (HIVE-14412) Add a timezone-aware timestamp

2017-04-26 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-14412: -- Attachment: HIVE-14412.10.patch > Add a timezone-aware timestamp > -- > >

[jira] [Updated] (HIVE-14412) Add a timezone-aware timestamp

2017-04-25 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-14412: -- Attachment: HIVE-14412.9.patch > Add a timezone-aware timestamp > -- > >

[jira] [Commented] (HIVE-11133) Support hive.explain.user for Spark

2017-04-25 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-11133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15984125#comment-15984125 ] Rui Li commented on HIVE-11133: --- OK I'm +1 to commit this and fix the bugs in follow-ups. [~

[jira] [Commented] (HIVE-16288) Add blobstore tests for ORC and RCFILE file formats

2017-04-25 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15982724#comment-15982724 ] Rui Li commented on HIVE-16288: --- Seems the events.txt also goes to the wrong location. > Ad

[jira] [Commented] (HIVE-16516) Set storage-api.version to 3.0.0-SNAPSHOT

2017-04-25 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15982584#comment-15982584 ] Rui Li commented on HIVE-16516: --- Is this same as HIVE-16396? > Set storage-api.version to 3

[jira] [Commented] (HIVE-16287) Alter table partition rename with location - moves partition back to hive warehouse

2017-04-23 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15980660#comment-15980660 ] Rui Li commented on HIVE-16287: --- QA on branch-1 has been broken for a long time. I think we

[jira] [Commented] (HIVE-16047) Shouldn't try to get KeyProvider unless encryption is enabled

2017-04-23 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15980654#comment-15980654 ] Rui Li commented on HIVE-16047: --- The 2.2 branch is still building with Hadoop-2.7.2. My conc

[jira] [Commented] (HIVE-11133) Support hive.explain.user for Spark

2017-04-23 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-11133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15980630#comment-15980630 ] Rui Li commented on HIVE-11133: --- Hi [~stakiar], in the query plan of your [comment|https://

[jira] [Commented] (HIVE-16047) Shouldn't try to get KeyProvider unless encryption is enabled

2017-04-20 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15978044#comment-15978044 ] Rui Li commented on HIVE-16047: --- [~andrew.wang] thanks for the information. Is there any pub

[jira] [Commented] (HIVE-11133) Support hive.explain.user for Spark

2017-04-20 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-11133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977939#comment-15977939 ] Rui Li commented on HIVE-11133: --- Another example of non-root stage having vertex dependency

[jira] [Commented] (HIVE-11133) Support hive.explain.user for Spark

2017-04-20 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-11133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977931#comment-15977931 ] Rui Li commented on HIVE-11133: --- [~stakiar], the plan in your last comment doesn't seem corr

[jira] [Commented] (HIVE-15165) Flaky test: TestSparkNegativeCliDriver

2017-04-19 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976164#comment-15976164 ] Rui Li commented on HIVE-15165: --- The code throwing this exception is {code} val newD

[jira] [Updated] (HIVE-16459) Forward channelInactive to RpcDispatcher

2017-04-19 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Resolution: Fixed Fix Version/s: 2.4.0 3.0.0 2.3.0 Status

[jira] [Updated] (HIVE-16459) Forward channelInactive to RpcDispatcher

2017-04-19 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Summary: Forward channelInactive to RpcDispatcher (was: Cancel outstanding RPCs when channel closes) > Forward

[jira] [Commented] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-19 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15975965#comment-15975965 ] Rui Li commented on HIVE-16459: --- The TestSparkCliDriver is not related and tracked by HIVE-1

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

2017-04-19 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15975959#comment-15975959 ] Rui Li commented on HIVE-15302: --- Thanks [~xuefuz] for the suggestions. I'll investigate how

[jira] [Commented] (HIVE-11133) Support hive.explain.user for Spark

2017-04-19 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-11133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15975956#comment-15975956 ] Rui Li commented on HIVE-11133: --- I haven't used the explain user feature before. Got one que

[jira] [Updated] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-19 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Attachment: HIVE-16459.2.patch > Cancel outstanding RPCs when channel closes > -

[jira] [Comment Edited] (HIVE-16418) Allow HiveKey to skip some bytes for comparison

2017-04-18 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972158#comment-15972158 ] Rui Li edited comment on HIVE-16418 at 4/18/17 12:48 PM: - [~ashuto

[jira] [Updated] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-18 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Attachment: HIVE-16459.2.patch > Cancel outstanding RPCs when channel closes > -

[jira] [Updated] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-18 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Attachment: (was: HIVE-16459.2.patch) > Cancel outstanding RPCs when channel closes > --

[jira] [Updated] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-18 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Attachment: (was: HIVE-16459.2.patch) > Cancel outstanding RPCs when channel closes > --

[jira] [Commented] (HIVE-16418) Allow HiveKey to skip some bytes for comparison

2017-04-17 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972158#comment-15972158 ] Rui Li commented on HIVE-16418: --- [~ashutoshc], thanks for your advice. bq. use LocalTimeZone

[jira] [Updated] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-17 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Attachment: HIVE-16459.2.patch Failures can't be reproduced. Try again > Cancel outstanding RPCs when channel c

[jira] [Updated] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-17 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Attachment: HIVE-16459.2.patch Just noticed we're already cancelling RPCs in {{RpcDispatcher::channelInactive}}

[jira] [Commented] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-17 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970867#comment-15970867 ] Rui Li commented on HIVE-16459: --- Failures not related. [~xuefuz] could you take a look? Than

[jira] [Updated] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-17 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Status: Patch Available (was: Open) > Cancel outstanding RPCs when channel closes > ---

[jira] [Updated] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-17 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Attachment: HIVE-16459.1.patch > Cancel outstanding RPCs when channel closes > -

[jira] [Updated] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-17 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16459: -- Description: In SparkTask, we try to get job infos after the query finishes. Assume the job finishes due to remo

[jira] [Assigned] (HIVE-16459) Cancel outstanding RPCs when channel closes

2017-04-16 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned HIVE-16459: - > Cancel outstanding RPCs when channel closes > --- > >

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

2017-04-16 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970629#comment-15970629 ] Rui Li commented on HIVE-15302: --- Hi [~vanzin], the goal here is just to figure out what to s

[jira] [Commented] (HIVE-16287) Alter table partition rename with location - moves partition back to hive warehouse

2017-04-15 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970230#comment-15970230 ] Rui Li commented on HIVE-16287: --- [~vihangk1], since this issue exists in 1.x, could you prov

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

2017-04-15 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970228#comment-15970228 ] Rui Li commented on HIVE-15302: --- Thanks [~vanzin] for the suggestions. I'm trying to figure

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

2017-04-14 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969031#comment-15969031 ] Rui Li commented on HIVE-15302: --- Following is the identified min set of required jars: {nofo

[jira] [Commented] (HIVE-16418) Allow HiveKey to skip some bytes for comparison

2017-04-14 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968955#comment-15968955 ] Rui Li commented on HIVE-16418: --- [~xuefuz] has talked about this with me offline. Let me use

[jira] [Commented] (HIVE-16418) Allow HiveKey to skip some bytes for comparison

2017-04-13 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968551#comment-15968551 ] Rui Li commented on HIVE-16418: --- Thanks guys for your inputs! To [~gopalv]'s question: bq. T

[jira] [Commented] (HIVE-14412) Add a timezone-aware timestamp

2017-04-13 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968546#comment-15968546 ] Rui Li commented on HIVE-14412: --- Hi [~ashutoshc], thanks for the suggestions. bq. Timestamp

[jira] [Commented] (HIVE-16418) Allow HiveKey to skip some bytes for comparison

2017-04-13 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967464#comment-15967464 ] Rui Li commented on HIVE-16418: --- [~gopalv] - thanks for the review. My plan is to only allow

[jira] [Updated] (HIVE-16418) Allow HiveKey to skip some bytes for comparison

2017-04-13 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16418: -- Status: Patch Available (was: Open) > Allow HiveKey to skip some bytes for comparison > ---

[jira] [Updated] (HIVE-16418) Allow HiveKey to skip some bytes for comparison

2017-04-13 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16418: -- Attachment: HIVE-16418.1.patch > Allow HiveKey to skip some bytes for comparison > -

[jira] [Assigned] (HIVE-16418) Allow HiveKey to skip some bytes for comparison

2017-04-11 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned HIVE-16418: - > Allow HiveKey to skip some bytes for comparison > --- > >

[jira] [Assigned] (HIVE-15104) Hive on Spark generate more shuffle data than hive on mr

2017-04-09 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned HIVE-15104: - Assignee: Rui Li (was: Aihua Xu) > Hive on Spark generate more shuffle data than hive on mr > --

[jira] [Commented] (HIVE-15104) Hive on Spark generate more shuffle data than hive on mr

2017-04-06 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15960146#comment-15960146 ] Rui Li commented on HIVE-15104: --- Hi [~aihuaxu], are you still working on this? If not, do yo

[jira] [Commented] (HIVE-16394) HoS does not support queue name change in middle of session

2017-04-05 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958326#comment-15958326 ] Rui Li commented on HIVE-16394: --- +1. I think configs starting with "spark" has already been

[jira] [Updated] (HIVE-16315) Describe table doesn't show num of partitions

2017-03-30 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16315: -- Resolution: Fixed Fix Version/s: 3.0.0 2.3.0 Status: Resolved (was: Patch A

[jira] [Commented] (HIVE-16336) Rename hive.spark.use.file.size.for.mapjoin to hive.spark.use.ts.stats.for.mapjoin

2017-03-30 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950166#comment-15950166 ] Rui Li commented on HIVE-16336: --- +1, thanks Chao for the rename. > Rename hive.spark.use.fi

[jira] [Commented] (HIVE-16328) HoS: more aggressive mapjoin optimization when hive.spark.use.file.size.for.mapjoin is true

2017-03-30 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948689#comment-15948689 ] Rui Li commented on HIVE-16328: --- Hi [~csun], this is off the topic here, but IIRC, HoS uses

[jira] [Commented] (HIVE-16315) Describe table doesn't show num of partitions

2017-03-29 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948328#comment-15948328 ] Rui Li commented on HIVE-16315: --- [~ashutoshc], the latest failures are of age 15 and not rel

[jira] [Commented] (HIVE-16315) Describe table doesn't show num of partitions

2017-03-29 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947029#comment-15947029 ] Rui Li commented on HIVE-16315: --- [~ashutoshc], I've put patch v2 on RB. Could you take a loo

[jira] [Updated] (HIVE-16315) Describe table doesn't show num of partitions

2017-03-29 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16315: -- Attachment: HIVE-16315.2.patch Update golden files. All the diff is adding the numPartition stats, which is exp

[jira] [Updated] (HIVE-16315) Describe table doesn't show num of partitions

2017-03-28 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16315: -- Status: Patch Available (was: Open) > Describe table doesn't show num of partitions > -

[jira] [Updated] (HIVE-16315) Describe table doesn't show num of partitions

2017-03-28 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16315: -- Attachment: HIVE-16315.1.patch > Describe table doesn't show num of partitions > ---

[jira] [Assigned] (HIVE-16315) Describe table doesn't show num of partitions

2017-03-28 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned HIVE-16315: - > Describe table doesn't show num of partitions > - > >

[jira] [Commented] (HIVE-16292) SparkUtilities upload to HDFS doesn't work with viewfs

2017-03-26 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15942604#comment-15942604 ] Rui Li commented on HIVE-16292: --- +1, thanks for the fix Jimmy. > SparkUtilities upload to H

[jira] [Commented] (HIVE-14919) Improve the performance of Hive on Spark 2.0.0

2017-03-21 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15935686#comment-15935686 ] Rui Li commented on HIVE-14919: --- [~stakiar], [~kellyzly] we're using RDD with type like . Th

[jira] [Commented] (HIVE-14919) Improve the performance of Hive on Spark 2.0.0

2017-03-16 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929448#comment-15929448 ] Rui Li commented on HIVE-14919: --- Yeah that's what I meant. > Improve the performance of Hiv

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-03-16 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929424#comment-15929424 ] Rui Li commented on HIVE-16071: --- +1 > Spark remote driver misuses the timeout in RPC handsh

[jira] [Commented] (HIVE-14919) Improve the performance of Hive on Spark 2.0.0

2017-03-16 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927700#comment-15927700 ] Rui Li commented on HIVE-14919: --- I mean we can set Xms, not Xmx. > Improve the performance

[jira] [Commented] (HIVE-14557) Nullpointer When both SkewJoin and Mapjoin Enabled

2017-03-16 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927693#comment-15927693 ] Rui Li commented on HIVE-14557: --- Change looks good to me. +1 pending test > Nullpointer Whe

[jira] [Commented] (HIVE-16183) Fix potential thread safety issues with static variables

2017-03-15 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927367#comment-15927367 ] Rui Li commented on HIVE-16183: --- Thanks for the update. +1 > Fix potential thread safety is

[jira] [Commented] (HIVE-16183) Fix potential thread safety issues with static variables

2017-03-14 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925457#comment-15925457 ] Rui Li commented on HIVE-16183: --- Thanks Xuefu for working on this. I also left some minor co

[jira] [Commented] (HIVE-15912) Executor kill task and Failed to get spark memory/core info

2017-03-13 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15907035#comment-15907035 ] Rui Li commented on HIVE-15912: --- [~yiyao], is that all the redirector log you have? Usually

[jira] [Updated] (HIVE-16155) No need for ConditionalTask if no conditional map join is created

2017-03-12 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16155: -- Affects Version/s: 2.1.1 > No need for ConditionalTask if no conditional map join is created > -

[jira] [Updated] (HIVE-16155) No need for ConditionalTask if no conditional map join is created

2017-03-12 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16155: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to m

[jira] [Commented] (HIVE-15912) Executor kill task and Failed to get spark memory/core info

2017-03-12 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15906777#comment-15906777 ] Rui Li commented on HIVE-15912: --- Can you see any output from the RemoteDriver in your hive.l

[jira] [Commented] (HIVE-16155) No need for ConditionalTask if no conditional map join is created

2017-03-10 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904708#comment-15904708 ] Rui Li commented on HIVE-16155: --- The failure can't be reproduced locally. [~xuefuz] would yo

[jira] [Commented] (HIVE-15912) Executor kill task and Failed to get spark memory/core info

2017-03-10 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904694#comment-15904694 ] Rui Li commented on HIVE-15912: --- [~yiyao], you can see the log indicates the Rpc channel is

[jira] [Commented] (HIVE-15912) Executor kill task and Failed to get spark memory/core info

2017-03-09 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904593#comment-15904593 ] Rui Li commented on HIVE-15912: --- Hi [~yiyao], for yarn-client mode, you should check the hiv

[jira] [Updated] (HIVE-16155) No need for ConditionalTask if no conditional map join is created

2017-03-09 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16155: -- Status: Patch Available (was: Open) > No need for ConditionalTask if no conditional map join is created > -

[jira] [Updated] (HIVE-16155) No need for ConditionalTask if no conditional map join is created

2017-03-09 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16155: -- Attachment: HIVE-16155.1.patch > No need for ConditionalTask if no conditional map join is created > ---

[jira] [Assigned] (HIVE-16155) No need for ConditionalTask if no conditional map join is created

2017-03-09 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned HIVE-16155: - > No need for ConditionalTask if no conditional map join is created > ---

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-03-09 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902756#comment-15902756 ] Rui Li commented on HIVE-16071: --- Yeah that makes sense to me. > Spark remote driver misuses

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-03-08 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902425#comment-15902425 ] Rui Li commented on HIVE-16071: --- Hi [~xuefuz], your understanding is inline with mine. Not s

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-03-08 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900942#comment-15900942 ] Rui Li commented on HIVE-16071: --- I looked more into the code and it turned out to be trickie

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-03-07 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900728#comment-15900728 ] Rui Li commented on HIVE-16071: --- Hi [~xuefuz], let me summarise my point: here we're talking

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-03-07 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900627#comment-15900627 ] Rui Li commented on HIVE-16071: --- Hi [~xuefuz], in your example, if the SASL handshake doesn'

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-03-07 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898962#comment-15898962 ] Rui Li commented on HIVE-16071: --- Hi [~xuefuz] and [~ctang.ma], I'd prefer removing the cance

[jira] [Commented] (HIVE-16131) Hive building with Hadoop 3 - additional stuff broken recently

2017-03-06 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898677#comment-15898677 ] Rui Li commented on HIVE-16131: --- +1 to the changes to {{Hadoop23Shims}} :) > Hive building

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-03-02 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893801#comment-15893801 ] Rui Li commented on HIVE-16071: --- Thanks [~ctang.ma] for the detailed analysis! bq. it was ca

[jira] [Comment Edited] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-03-01 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891686#comment-15891686 ] Rui Li edited comment on HIVE-16071 at 3/2/17 6:50 AM: --- Hi [~ctang.m

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-03-01 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891686#comment-15891686 ] Rui Li commented on HIVE-16071: --- Hi [~ctang.ma], for the issue mentioned in this JIRA, I thi

[jira] [Updated] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-03-01 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-15882: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to m

[jira] [Commented] (HIVE-15882) HS2 generating high memory pressure with many partitions and concurrent queries

2017-03-01 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891439#comment-15891439 ] Rui Li commented on HIVE-15882: --- I'll commit shortly if there's no further comments. > HS2

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-02-28 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889695#comment-15889695 ] Rui Li commented on HIVE-16071: --- My understanding is the timeout on SparkClient side is long

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

2017-02-28 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889474#comment-15889474 ] Rui Li commented on HIVE-16071: --- [~ctang.ma], you might want to take a look at HIVE-15671. I

[jira] [Updated] (HIVE-16047) Shouldn't try to get KeyProvider unless encryption is enabled

2017-02-28 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HIVE-16047: -- Resolution: Fixed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Committed to m

[jira] [Commented] (HIVE-16047) Shouldn't try to get KeyProvider unless encryption is enabled

2017-02-28 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889415#comment-15889415 ] Rui Li commented on HIVE-16047: --- Failure with age 1 can't be reproduced. > Shouldn't try to

<    2   3   4   5   6   7   8   9   10   11   >