[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Status: Patch Available (was: Open)
> Avoid using SPARK_JAVA_OPTS [Spark Branch]
> ---
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Attachment: HIVE-9019.1-spark.patch
> Avoid using SPARK_JAVA_OPTS [Spark Branch]
>
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Attachment: (was: HIVE-9019.1-spark.patch)
> Avoid using SPARK_JAVA_OPTS [Spark Branch]
> -
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Attachment: HIVE-9019.1-spark.patch
> Avoid using SPARK_JAVA_OPTS [Spark Branch]
>
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Description:
SPARK_JAVA_OPTS has been deprecated, see {{SparkConf.validateSettings}}.
Using it together with {{spar
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-9019:
Assignee: Rui Li
> Avoid using SPARK_JAVA_OPTS [Spark Branch]
> --
>
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8536:
-
Attachment: HIVE-8536.4-spark.patch
Disable for now.
> Enable SkewJoinResolver for spark [Spark Branch]
>
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237359#comment-14237359
]
Rui Li commented on HIVE-8536:
--
Hi [~xuefuz], my uncertainty is mainly about the # map join ta
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234082#comment-14234082
]
Rui Li commented on HIVE-8991:
--
Not sure if it's because how we add hive-exec:
If added dynami
[
https://issues.apache.org/jira/browse/HIVE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9019:
-
Issue Type: Sub-task (was: Test)
Parent: HIVE-7292
> Avoid using SPARK_JAVA_OPTS [Spark Branch]
>
Rui Li created HIVE-9019:
Summary: Avoid using SPARK_JAVA_OPTS [Spark Branch]
Key: HIVE-9019
URL: https://issues.apache.org/jira/browse/HIVE-9019
Project: Hive
Issue Type: Test
Components:
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233993#comment-14233993
]
Rui Li commented on HIVE-8991:
--
I looked a little more into this. It seems hive-exec is proper
[
https://issues.apache.org/jira/browse/HIVE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9017:
-
Issue Type: Sub-task (was: Test)
Parent: HIVE-7292
> Clean up temp files after unit tests [Spark Branch]
>
Rui Li created HIVE-9017:
Summary: Clean up temp files after unit tests [Spark Branch]
Key: HIVE-9017
URL: https://issues.apache.org/jira/browse/HIVE-9017
Project: Hive
Issue Type: Test
Com
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Attachment: (was: HIVE-8972.1-spark.patch)
> Implement more fine-grained remote client-level events [Spark Bran
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233870#comment-14233870
]
Rui Li commented on HIVE-8536:
--
The failures and diff in query plan is expected.
We can disabl
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Attachment: HIVE-8972.3-spark.patch
Try again as I can't reproduce the failures on my side.
> Implement more fine-
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Attachment: HIVE-8972.2-spark.patch
Fix some bugs and try again.
> Implement more fine-grained remote client-level
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8536:
-
Attachment: HIVE-8536.3-spark.patch
Now {{skewjoin_noskew.q}} should be able to run, with diff in query plan.
> En
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Attachment: HIVE-8972.1-spark.patch
Reload the same patch to run test again.
> Implement more fine-grained remote
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14231528#comment-14231528
]
Rui Li commented on HIVE-8995:
--
Further verified by monitoring the surefirebooter JVM. Seems i
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8995:
-
Attachment: with-patch.PNG
> Find thread leak in RSC Tests
> -
>
> Key:
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8995:
-
Attachment: without-patch.PNG
> Find thread leak in RSC Tests
> -
>
> K
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Status: Patch Available (was: Open)
> Implement more fine-grained remote client-level events [Spark Branch]
>
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8972:
-
Attachment: HIVE-8972.1-spark.patch
Submit a patch to let the test run
> Implement more fine-grained remote client
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8995:
-
Status: Patch Available (was: Open)
> Find thread leak in RSC Tests
> -
>
>
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8995:
-
Attachment: HIVE-8995.1-spark.patch
I tried the tests [~brocknoland] mentioned with this patch. All the tests
pass
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230872#comment-14230872
]
Rui Li commented on HIVE-8995:
--
Hi [~brocknoland], I checked the code. It seems we initialize
[
https://issues.apache.org/jira/browse/HIVE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230812#comment-14230812
]
Rui Li commented on HIVE-8995:
--
OK I'll have a look.
> Find thread leak in RSC Tests
> --
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230811#comment-14230811
]
Rui Li commented on HIVE-8991:
--
Hi [~vanzin], just as [~xuefuz] said, this JIRA is only meant
[
https://issues.apache.org/jira/browse/HIVE-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-8972:
Assignee: Rui Li
> Implement more fine-grained remote client-level events [Spark Branch]
> -
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228335#comment-14228335
]
Rui Li commented on HIVE-8991:
--
[~vanzin] could you help look at this? Thanks!
> Fix custom_i
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8991:
-
Attachment: HIVE-8991.1-spark.patch
This patch can fix the test on my machine.
Strange thing is that, if I add hive
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8991:
-
Status: Patch Available (was: Open)
> Fix custom_input_output_format [Spark Branch]
>
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-8991:
Assignee: Rui Li
> Fix custom_input_output_format [Spark Branch]
> -
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8991:
-
Component/s: Spark
> Fix custom_input_output_format [Spark Branch]
> -
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8991:
-
Summary: Fix custom_input_output_format [Spark Branch] (was: Fix
custom_input_output_format)
> Fix custom_input_o
[
https://issues.apache.org/jira/browse/HIVE-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8991:
-
Issue Type: Sub-task (was: Bug)
Parent: HIVE-8548
> Fix custom_input_output_format
> -
Rui Li created HIVE-8991:
Summary: Fix custom_input_output_format
Key: HIVE-8991
URL: https://issues.apache.org/jira/browse/HIVE-8991
Project: Hive
Issue Type: Bug
Reporter: Rui Li
Afte
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14228039#comment-14228039
]
Rui Li commented on HIVE-8984:
--
Thanks [~xuefuz] for the review!
> Enable job metrics/statist
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14227708#comment-14227708
]
Rui Li commented on HIVE-8984:
--
[~xuefuz] - sorry for the lack of descriptions. The patch shou
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8984:
-
Status: Patch Available (was: Open)
> Enable job metrics/statistics gathering for remote spark context [Spark
> B
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8984:
-
Attachment: HIVE-8984.1-spark.patch
The refactor in this patch is mainly moving spark counter related classes to
s
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8984:
-
Description: stats_counter fails due to this issue
> Enable job metrics/statistics gathering for remote spark conte
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-8984:
Assignee: Rui Li
> Enable job metrics/statistics gathering for remote spark context [Spark
> Branch]
>
Rui Li created HIVE-8984:
Summary: Enable job metrics/statistics gathering for remote spark
context [Spark Branch]
Key: HIVE-8984
URL: https://issues.apache.org/jira/browse/HIVE-8984
Project: Hive
I
[
https://issues.apache.org/jira/browse/HIVE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8984:
-
Issue Type: Sub-task (was: Improvement)
Parent: HIVE-8548
> Enable job metrics/statistics gathering for re
[
https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14227118#comment-14227118
]
Rui Li commented on HIVE-8836:
--
Thanks [~brocknoland] and [~xuefuz] for the work. Maybe we can
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14227107#comment-14227107
]
Rui Li commented on HIVE-8956:
--
OK I see. Thanks [~vanzin] for the explanation.
> Hive hangs
[
https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225860#comment-14225860
]
Rui Li commented on HIVE-8836:
--
Shall we use same #reducers as before for now? So that we can
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225826#comment-14225826
]
Rui Li commented on HIVE-8961:
--
Thanks [~xuefuz] for the work.
> Remove unnecessary dependenc
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225763#comment-14225763
]
Rui Li commented on HIVE-8956:
--
Yeah I created HIVE-8972 as a follow up.
As for retry failed j
Rui Li created HIVE-8972:
Summary: Implement more fine-grained remote client-level events
[Spark Branch]
Key: HIVE-8972
URL: https://issues.apache.org/jira/browse/HIVE-8972
Project: Hive
Issue Type:
[
https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225751#comment-14225751
]
Rui Li commented on HIVE-8836:
--
OK I'll try out the failed tests.
> Enable automatic tests wi
[
https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225721#comment-14225721
]
Rui Li commented on HIVE-8836:
--
Hi [~brocknoland], this may be related to HIVE-8956. Do you kn
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225677#comment-14225677
]
Rui Li commented on HIVE-8536:
--
Thanks [~csun], I can verify this later. For now, let's stick
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225614#comment-14225614
]
Rui Li commented on HIVE-8956:
--
Thanks [~vanzin] for your input!
Adding {{JobReceived}} and {{
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225570#comment-14225570
]
Rui Li commented on HIVE-8961:
--
Thanks [~xuefuz] for taking care of this. I'll rebase and run
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225565#comment-14225565
]
Rui Li commented on HIVE-8536:
--
Hi [~csun] thanks for your input.
I'm removing the dependency
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8961:
-
Attachment: HIVE-8961.1-spark.patch
Changes to golden files are all in query plan. No results changed.
> Remove un
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8961:
-
Status: Patch Available (was: Open)
> Remove unnecessary dependency collection task [Spark Branch]
> -
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8961:
-
Component/s: Spark
> Remove unnecessary dependency collection task [Spark Branch]
> ---
[
https://issues.apache.org/jira/browse/HIVE-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8961:
-
Issue Type: Sub-task (was: Test)
Parent: HIVE-7292
> Remove unnecessary dependency collection task [Spark
Rui Li created HIVE-8961:
Summary: Remove unnecessary dependency collection task [Spark
Branch]
Key: HIVE-8961
URL: https://issues.apache.org/jira/browse/HIVE-8961
Project: Hive
Issue Type: Test
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224299#comment-14224299
]
Rui Li commented on HIVE-8536:
--
It seems the dependency task is created in {{GenSparkProcConte
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8956:
-
Status: Patch Available (was: Open)
> Hive hangs while some error/exception happens beyond job execution[Spark
>
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8956:
-
Attachment: HIVE-8956.1-spark.patch
Add timeout for waiting a spark job to be submitted.
> Hive hangs while some e
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224182#comment-14224182
]
Rui Li commented on HIVE-8536:
--
OK I'll take a look.
> Enable SkewJoinResolver for spark [Spa
Rui Li created HIVE-8958:
Summary: Make sure map join tasks created by runtime skew join can
fit into memory [Spark Branch]
Key: HIVE-8958
URL: https://issues.apache.org/jira/browse/HIVE-8958
Project: Hive
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224122#comment-14224122
]
Rui Li commented on HIVE-8536:
--
Hi [~szehon],
{{skewjoin_noskew}} failed to run because our r
[
https://issues.apache.org/jira/browse/HIVE-8956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-8956:
Assignee: Rui Li
> Hive hangs while some error/exception happens beyond job execution[Spark
> Branch]
> ---
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224045#comment-14224045
]
Rui Li commented on HIVE-8834:
--
Thanks guys for the review!
> enable job progress monitoring
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Attachment: HIVE-8834.6-spark.patch
Address RB comments. Thanks [~vanzin] for the review!
> enable job progress mo
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222964#comment-14222964
]
Rui Li commented on HIVE-8536:
--
With the latest patch, the previous example query plan is:
{no
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8536:
-
Attachment: HIVE-8536.2-spark.patch
Address RB comments
> Enable SkewJoinResolver for spark [Spark Branch]
> -
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Attachment: HIVE-8834.5-spark.patch
Sorry I missed some commits in last patch, update it.
> enable job progress mo
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222573#comment-14222573
]
Rui Li commented on HIVE-8834:
--
[~xuefuz] - for the serializable mark, there's some discussion
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Attachment: HIVE-8834.4-spark.patch
The new patch wraps requesting job/stage info into a {{Job}} so we can use the
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221726#comment-14221726
]
Rui Li commented on HIVE-8536:
--
Got it. Thanks [~csun] for the explanation.
> Enable SkewJoin
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221696#comment-14221696
]
Rui Li commented on HIVE-8536:
--
[~szehon] - thanks for the review! Stage 3 is not split becaus
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220851#comment-14220851
]
Rui Li commented on HIVE-8834:
--
Hold on this, I'll try if I can wrap all these in a Job to avo
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8536:
-
Attachment: (was: plan)
> Enable SkewJoinResolver for spark [Spark Branch]
> --
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220838#comment-14220838
]
Rui Li commented on HIVE-8536:
--
{noformat}
Listening for transport dt_socket at address: 5005
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8536:
-
Attachment: plan
This is the plan for a simple join query.
[~csun] please take a look and let me know if this is th
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8536:
-
Status: Patch Available (was: Open)
> Enable SkewJoinResolver for spark [Spark Branch]
> -
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8536:
-
Attachment: HIVE-8536.1-spark.patch
The SkewJoinResolver part patch.
> Enable SkewJoinResolver for spark [Spark Br
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220566#comment-14220566
]
Rui Li commented on HIVE-8834:
--
cc [~xuefuz]
You may want to look at this since it involves so
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Status: Patch Available (was: Open)
> enable job progress monitoring of Remote Spark Context [Spark Branch]
>
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Attachment: HIVE-8834.3-spark.patch
The latest patch adds more refactor:
1. Simplify JobHandle to encapsulate too d
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220478#comment-14220478
]
Rui Li commented on HIVE-8834:
--
Hold on the patch. Will do more refactor.
> enable job progre
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Status: Open (was: Patch Available)
> enable job progress monitoring of Remote Spark Context [Spark Branch]
>
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Attachment: HIVE-8834.2-spark.patch
Address RB comments. Thanks [~szehon] for review!
> enable job progress monito
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Attachment: HIVE-8834.1-spark.patch
This patch takes care of job progress monitor. Leave job metrics/statistics to
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Status: Patch Available (was: Open)
> enable job progress monitoring of Remote Spark Context [Spark Branch]
>
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8834:
-
Summary: enable job progress monitoring of Remote Spark Context [Spark
Branch] (was: enable job progress monitorin
Rui Li created HIVE-8913:
Summary: Make SparkMapJoinResolver handle runtime skew join [Spark
Branch]
Key: HIVE-8913
URL: https://issues.apache.org/jira/browse/HIVE-8913
Project: Hive
Issue Type: Imp
[
https://issues.apache.org/jira/browse/HIVE-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8536:
-
Summary: Enable SkewJoinResolver for spark [Spark Branch] (was: Enable
runtime skew join optimization for spark [S
[
https://issues.apache.org/jira/browse/HIVE-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li reassigned HIVE-8834:
Assignee: Rui Li
> enable job progress monitoring of Remote Spark Context[Spark Branch]
> --
[
https://issues.apache.org/jira/browse/HIVE-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14214356#comment-14214356
]
Rui Li commented on HIVE-8852:
--
Thanks a lot [~brocknoland] for taking care of all this! I'll
[
https://issues.apache.org/jira/browse/HIVE-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-8852:
-
Attachment: HIVE-8852.3-spark.patch
> Update new spark progress API for local submitted job monitoring[Spark Branch
301 - 400 of 679 matches
Mail list logo
