[jira] [Updated] (SPARK-32616) Window operators should be added determinedly

2020-08-14 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32616: - Issue Type: Bug (was: Improvement) > Window operators should be added determinedly >

[jira] [Created] (SPARK-32616) Window operators should be added determinedly

2020-08-14 Thread wuyi (Jira)
wuyi created SPARK-32616: Summary: Window operators should be added determinedly Key: SPARK-32616 URL: https://issues.apache.org/jira/browse/SPARK-32616 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-32600) Unify task name in some logs between driver and executor

2020-08-12 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32600: - Summary: Unify task name in some logs between driver and executor (was: Unify task name in some places between

[jira] [Created] (SPARK-32600) Unify task name in some places between driver and executor

2020-08-12 Thread wuyi (Jira)
wuyi created SPARK-32600: Summary: Unify task name in some places between driver and executor Key: SPARK-32600 URL: https://issues.apache.org/jira/browse/SPARK-32600 Project: Spark Issue Type:

[jira] [Created] (SPARK-32518) CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

2020-08-02 Thread wuyi (Jira)
wuyi created SPARK-32518: Summary: CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources Key: SPARK-32518 URL: https://issues.apache.org/jira/browse/SPARK-32518 Project:

[jira] [Created] (SPARK-32466) Add support to catch SparkPlan regression base on TPC-DS queries

2020-07-28 Thread wuyi (Jira)
wuyi created SPARK-32466: Summary: Add support to catch SparkPlan regression base on TPC-DS queries Key: SPARK-32466 URL: https://issues.apache.org/jira/browse/SPARK-32466 Project: Spark Issue

[jira] [Created] (SPARK-32459) UDF regression of WrappedArray supporting caused by SPARK-31826

2020-07-27 Thread wuyi (Jira)
wuyi created SPARK-32459: Summary: UDF regression of WrappedArray supporting caused by SPARK-31826 Key: SPARK-32459 URL: https://issues.apache.org/jira/browse/SPARK-32459 Project: Spark Issue Type:

[jira] [Created] (SPARK-32372) "Resolved attribute(s) XXX missing" after dudup conflict references

2020-07-20 Thread wuyi (Jira)
wuyi created SPARK-32372: Summary: "Resolved attribute(s) XXX missing" after dudup conflict references Key: SPARK-32372 URL: https://issues.apache.org/jira/browse/SPARK-32372 Project: Spark Issue

[jira] [Commented] (SPARK-32307) Aggression that use map type input UDF as group expression can fail

2020-07-16 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159020#comment-17159020 ] wuyi commented on SPARK-32307: -- Hi [~dongjoon] Please see my response here

[jira] [Updated] (SPARK-32307) Aggression that use map type input UDF as group expression can fail

2020-07-14 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32307: - Description: {code:java} spark.udf.register("key", udf((m: Map[String, String]) => m.keys.head.toInt))

[jira] [Created] (SPARK-32307) Aggression that use map type input UDF as group expression can fail

2020-07-14 Thread wuyi (Jira)
wuyi created SPARK-32307: Summary: Aggression that use map type input UDF as group expression can fail Key: SPARK-32307 URL: https://issues.apache.org/jira/browse/SPARK-32307 Project: Spark Issue

[jira] [Updated] (SPARK-32287) Flaky test: ExecutorAllocationManagerSuite.add executors default profile

2020-07-12 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32287: - Component/s: Tests > Flaky test: ExecutorAllocationManagerSuite.add executors default profile >

[jira] [Commented] (SPARK-32287) Flaky test: ExecutorAllocationManagerSuite.add executors default profile

2020-07-12 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17156441#comment-17156441 ] wuyi commented on SPARK-32287: -- cc [~tgraves] [~hyukjin.kwon] > Flaky test:

[jira] [Updated] (SPARK-32287) Flaky test: ExecutorAllocationManagerSuite.add executors default profile

2020-07-12 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32287: - Description: This test becomes flaky in Github Actions, see:

[jira] [Updated] (SPARK-32287) Flaky test: ExecutorAllocationManagerSuite.add executors default profile

2020-07-12 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32287: - Description: This test becomes flaky in Github Actions, see:

[jira] [Created] (SPARK-32287) Flaky test: ExecutorAllocationManagerSuite.add executors default profile

2020-07-12 Thread wuyi (Jira)
wuyi created SPARK-32287: Summary: Flaky test: ExecutorAllocationManagerSuite.add executors default profile Key: SPARK-32287 URL: https://issues.apache.org/jira/browse/SPARK-32287 Project: Spark

[jira] [Commented] (SPARK-32250) Reenable MasterSuite's "Master should avoid dead loop while launching executor failed in Worker"

2020-07-10 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155201#comment-17155201 ] wuyi commented on SPARK-32250: -- [~hyukjin.kwon]Thanks for ping me. I'll take a look later. > Reenable

[jira] [Updated] (SPARK-32238) Use Utils.getSimpleName to avoid hitting Malformed class name in ScalaUDF

2020-07-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32238: - Issue Type: Bug (was: Improvement) > Use Utils.getSimpleName to avoid hitting Malformed class name in ScalaUDF

[jira] [Created] (SPARK-32238) "Malformed class name" error from ScalaUDF

2020-07-09 Thread wuyi (Jira)
wuyi created SPARK-32238: Summary: "Malformed class name" error from ScalaUDF Key: SPARK-32238 URL: https://issues.apache.org/jira/browse/SPARK-32238 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32120) Single GPU is allocated multiple times

2020-07-08 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153579#comment-17153579 ] wuyi commented on SPARK-32120: -- oh yeah, we've documented it. Thanks for reminding. > Single GPU is

[jira] [Commented] (SPARK-32120) Single GPU is allocated multiple times

2020-07-05 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151555#comment-17151555 ] wuyi commented on SPARK-32120: -- We've removed coordination logic in SPARK-30969. It should not be a problem

[jira] [Commented] (SPARK-32120) Single GPU is allocated multiple times

2020-07-05 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151550#comment-17151550 ] wuyi commented on SPARK-32120: -- [~EnricoMi] Thanks for reporting! I am looking into this. > Single GPU is

[jira] [Updated] (SPARK-32154) Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

2020-07-02 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32154: - Summary: Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type (was: Use

[jira] [Updated] (SPARK-32154) Use ExpressionEncoder for the return type of ScalaUDF to serialize to catalyst type

2020-07-02 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32154: - Summary: Use ExpressionEncoder for the return type of ScalaUDF to serialize to catalyst type (was: Use

[jira] [Created] (SPARK-32154) Use ExpressionEncoder to serialize to catalyst type for the return type of ScalaUDF

2020-07-02 Thread wuyi (Jira)
wuyi created SPARK-32154: Summary: Use ExpressionEncoder to serialize to catalyst type for the return type of ScalaUDF Key: SPARK-32154 URL: https://issues.apache.org/jira/browse/SPARK-32154 Project: Spark

[jira] [Updated] (SPARK-32091) Ignore timeout error when remove blocks on the lost executor

2020-06-24 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32091: - Description: When removing blocks(e.g. RDD, broadcast, shuffle), BlockManagerMaserEndpoint will make RPC calls

[jira] [Created] (SPARK-32091) Ignore timeout error when remove blocks on the lost executor

2020-06-24 Thread wuyi (Jira)
wuyi created SPARK-32091: Summary: Ignore timeout error when remove blocks on the lost executor Key: SPARK-32091 URL: https://issues.apache.org/jira/browse/SPARK-32091 Project: Spark Issue Type:

[jira] [Created] (SPARK-32090) UserDefinedType.equal() does not have symmetry

2020-06-24 Thread wuyi (Jira)
wuyi created SPARK-32090: Summary: UserDefinedType.equal() does not have symmetry Key: SPARK-32090 URL: https://issues.apache.org/jira/browse/SPARK-32090 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-32087) Allow UserDefinedType to use encoder to deserialize rows in ScalaUDF as well

2020-06-24 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-32087: - Summary: Allow UserDefinedType to use encoder to deserialize rows in ScalaUDF as well (was: Allow

[jira] [Created] (SPARK-32087) Allow UserDenfinedType to use encoder to deserialize rows in ScalaUDF as well

2020-06-24 Thread wuyi (Jira)
wuyi created SPARK-32087: Summary: Allow UserDenfinedType to use encoder to deserialize rows in ScalaUDF as well Key: SPARK-32087 URL: https://issues.apache.org/jira/browse/SPARK-32087 Project: Spark

[jira] [Created] (SPARK-32077) Support host-local shuffle data reading with external shuffle service disabled

2020-06-23 Thread wuyi (Jira)
wuyi created SPARK-32077: Summary: Support host-local shuffle data reading with external shuffle service disabled Key: SPARK-32077 URL: https://issues.apache.org/jira/browse/SPARK-32077 Project: Spark

[jira] [Created] (SPARK-32055) Unify getReader and getReaderForRange in ShuffleManager

2020-06-22 Thread wuyi (Jira)
wuyi created SPARK-32055: Summary: Unify getReader and getReaderForRange in ShuffleManager Key: SPARK-32055 URL: https://issues.apache.org/jira/browse/SPARK-32055 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32037) Rename blacklisting feature to avoid language with racist connotation

2020-06-22 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17141742#comment-17141742 ] wuyi commented on SPARK-32037: -- +1 for healthy/unhealthy > Rename blacklisting feature to avoid language

[jira] [Created] (SPARK-32031) Fix the wrong references of PartialMerge/Final AggregateExpression

2020-06-19 Thread wuyi (Jira)
wuyi created SPARK-32031: Summary: Fix the wrong references of PartialMerge/Final AggregateExpression Key: SPARK-32031 URL: https://issues.apache.org/jira/browse/SPARK-32031 Project: Spark Issue

[jira] [Updated] (SPARK-31970) Make MDC configuration step be consistent between setLocalProperty and log4j.properties

2020-06-11 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31970: - Summary: Make MDC configuration step be consistent between setLocalProperty and log4j.properties (was: Make

[jira] [Created] (SPARK-31970) Make MDC configuration step be consistent between setLocalProperty and log4j

2020-06-11 Thread wuyi (Jira)
wuyi created SPARK-31970: Summary: Make MDC configuration step be consistent between setLocalProperty and log4j Key: SPARK-31970 URL: https://issues.apache.org/jira/browse/SPARK-31970 Project: Spark

[jira] [Commented] (SPARK-31946) Failed to register SIGPWR handler on MacOS

2020-06-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130015#comment-17130015 ] wuyi commented on SPARK-31946: -- oh, I mean, for the **decommission** feature, we should register someone

[jira] [Comment Edited] (SPARK-31946) Failed to register SIGPWR handler on MacOS

2020-06-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130015#comment-17130015 ] wuyi edited comment on SPARK-31946 at 6/10/20, 3:27 AM: oh, I mean, for the

[jira] [Commented] (SPARK-31946) Failed to register SIGPWR handler on MacOS

2020-06-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129982#comment-17129982 ] wuyi commented on SPARK-31946: -- The message is good. But the failure should not be expected, IIUC. >

[jira] [Updated] (SPARK-31946) Failed to register SIGPWR handler on MacOS

2020-06-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31946: - Description:   {code:java} 20/06/09 22:54:54 WARN SignalUtils: Failed to register SIGPWR handler - disabling

[jira] [Commented] (SPARK-31946) Failed to register SIGPWR handler on MacOS

2020-06-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129947#comment-17129947 ] wuyi commented on SPARK-31946: -- cc [~holden] > Failed to register SIGPWR handler on MacOS >

[jira] [Created] (SPARK-31946) Failed to register SIGPWR handler on MacOS

2020-06-09 Thread wuyi (Jira)
wuyi created SPARK-31946: Summary: Failed to register SIGPWR handler on MacOS Key: SPARK-31946 URL: https://issues.apache.org/jira/browse/SPARK-31946 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-31922) "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

2020-06-07 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31922: - Summary: "RpcEnv already stopped" error when exit spark-shell with local-cluster mode (was: "RpcEnv already

[jira] [Updated] (SPARK-31922) "RpcEnv already stopped" Error when exit spark-shell with local-cluster mode

2020-06-07 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31922: - Summary: "RpcEnv already stopped" Error when exit spark-shell with local-cluster mode (was:

[jira] [Updated] (SPARK-31922) TransportRequestHandler Error when exit spark-shell with local-cluster mode

2020-06-06 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31922: - Affects Version/s: 2.4.6 > TransportRequestHandler Error when exit spark-shell with local-cluster mode >

[jira] [Created] (SPARK-31922) TransportRequestHandler Error when exit spark-shell with local-cluster mode

2020-06-06 Thread wuyi (Jira)
wuyi created SPARK-31922: Summary: TransportRequestHandler Error when exit spark-shell with local-cluster mode Key: SPARK-31922 URL: https://issues.apache.org/jira/browse/SPARK-31922 Project: Spark

[jira] [Commented] (SPARK-31922) TransportRequestHandler Error when exit spark-shell with local-cluster mode

2020-06-06 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17127370#comment-17127370 ] wuyi commented on SPARK-31922: -- I am working on this. > TransportRequestHandler Error when exit

[jira] [Updated] (SPARK-31921) Wrong warning of "WARN Master: App app-xxx requires more resource than any of Workers could have."

2020-06-06 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31921: - Description: When starting spark-shell using local cluster mode, e.g. ./bin/spark-shell --master

[jira] [Created] (SPARK-31921) Wrong warning of "WARN Master: App app-xxx requires more resource than any of Workers could have."

2020-06-06 Thread wuyi (Jira)
wuyi created SPARK-31921: Summary: Wrong warning of "WARN Master: App app-xxx requires more resource than any of Workers could have." Key: SPARK-31921 URL: https://issues.apache.org/jira/browse/SPARK-31921

[jira] [Created] (SPARK-31837) Shift to the new highest locality level if there is when recomputeLocality

2020-05-27 Thread wuyi (Jira)
wuyi created SPARK-31837: Summary: Shift to the new highest locality level if there is when recomputeLocality Key: SPARK-31837 URL: https://issues.apache.org/jira/browse/SPARK-31837 Project: Spark

[jira] [Created] (SPARK-31826) Support composed type of case class for typed Scala UDF

2020-05-26 Thread wuyi (Jira)
wuyi created SPARK-31826: Summary: Support composed type of case class for typed Scala UDF Key: SPARK-31826 URL: https://issues.apache.org/jira/browse/SPARK-31826 Project: Spark Issue Type:

[jira] [Updated] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

2020-05-21 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31784: - Description: {code:java} test("share messages with allGather() call") { val conf = new SparkConf()

[jira] [Updated] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

2020-05-21 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31784: - Description: {code:java} test("share messages with allGather() call") { val conf = new SparkConf()

[jira] [Updated] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

2020-05-21 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31784: - Description: {code:java} test("share messages with allGather() call") { val conf = new SparkConf()

[jira] [Updated] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

2020-05-21 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31784: - Description: {code:java} test("share messages with allGather() call") { val conf = new SparkConf()

[jira] [Updated] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

2020-05-21 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31784: - Environment:   {code:java}  {code}   was:   {code:java} test("share messages with allGather() call") {

[jira] [Created] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

2020-05-21 Thread wuyi (Jira)
wuyi created SPARK-31784: Summary: Fix test BarrierTaskContextSuite."share messages with allGather() call" Key: SPARK-31784 URL: https://issues.apache.org/jira/browse/SPARK-31784 Project: Spark

[jira] [Updated] (SPARK-31750) Eliminate UpCast if child's dataType is DecimalType

2020-05-18 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31750: - Summary: Eliminate UpCast if child's dataType is DecimalType (was: Eliminate UpCast if chid's dataType is

[jira] [Created] (SPARK-31750) Eliminate UpCast if chid's dataType is DecimalType

2020-05-18 Thread wuyi (Jira)
wuyi created SPARK-31750: Summary: Eliminate UpCast if chid's dataType is DecimalType Key: SPARK-31750 URL: https://issues.apache.org/jira/browse/SPARK-31750 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-31682) Turn on spark.sql.legacy.createHiveTableByDefault.enabled by default

2020-05-11 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31682: - Parent: (was: SPARK-30098) Issue Type: Improvement (was: Sub-task) > Turn on

[jira] [Updated] (SPARK-31682) Turn on spark.sql.legacy.createHiveTableByDefault.enabled by default

2020-05-11 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31682: - Parent: SPARK-31085 Issue Type: Sub-task (was: Improvement) > Turn on

[jira] [Created] (SPARK-31682) Turn on spark.sql.legacy.createHiveTableByDefault.enabled by default

2020-05-11 Thread wuyi (Jira)
wuyi created SPARK-31682: Summary: Turn on spark.sql.legacy.createHiveTableByDefault.enabled by default Key: SPARK-31682 URL: https://issues.apache.org/jira/browse/SPARK-31682 Project: Spark Issue

[jira] [Commented] (SPARK-31620) TreeNodeException: Binding attribute, tree: sum#19L

2020-05-10 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17104008#comment-17104008 ] wuyi commented on SPARK-31620: -- I'm working on this now. > TreeNodeException: Binding attribute, tree:

[jira] [Commented] (SPARK-20628) Keep track of nodes which are going to be shut down & avoid scheduling new tasks

2020-05-06 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100793#comment-17100793 ] wuyi commented on SPARK-20628: -- Hi [~holden] is this ticket resolved by 

[jira] [Updated] (SPARK-31651) Improve handling the case where different barrier sync types in a single sync

2020-05-06 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31651: - Summary: Improve handling the case where different barrier sync types in a single sync (was: Improve handling

[jira] [Updated] (SPARK-31651) Improve handling for the case of different barrier sync types in a single sync

2020-05-06 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31651: - Description: Currently, we use cleanupBarrierStage when detecting different barrier sync types in a single

[jira] [Created] (SPARK-31651) Improve handling for the case of different barrier sync types in a single sync

2020-05-06 Thread wuyi (Jira)
wuyi created SPARK-31651: Summary: Improve handling for the case of different barrier sync types in a single sync Key: SPARK-31651 URL: https://issues.apache.org/jira/browse/SPARK-31651 Project: Spark

[jira] [Updated] (SPARK-31650) SQL UI doesn't show metrics and whole stage codegen in AQE

2020-05-06 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31650: - Issue Type: Bug (was: Test) > SQL UI doesn't show metrics and whole stage codegen in AQE >

[jira] [Created] (SPARK-31650) SQL UI doesn't show metrics and whole stage codegen in AQE

2020-05-06 Thread wuyi (Jira)
wuyi created SPARK-31650: Summary: SQL UI doesn't show metrics and whole stage codegen in AQE Key: SPARK-31650 URL: https://issues.apache.org/jira/browse/SPARK-31650 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-31650) SQL UI doesn't show metrics and whole stage codegen in AQE

2020-05-06 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31650: - Attachment: before_aqe_ui.png > SQL UI doesn't show metrics and whole stage codegen in AQE >

[jira] [Updated] (SPARK-31643) Flaky o.a.s.scheduler.BarrierTaskContextSuite.barrier task killed, interrupt

2020-05-05 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31643: - Description:

[jira] [Created] (SPARK-31643) Flaky o.a.s.scheduler.BarrierTaskContextSuite.barrier task killed, interrupt

2020-05-05 Thread wuyi (Jira)
wuyi created SPARK-31643: Summary: Flaky o.a.s.scheduler.BarrierTaskContextSuite.barrier task killed, interrupt Key: SPARK-31643 URL: https://issues.apache.org/jira/browse/SPARK-31643 Project: Spark

[jira] [Resolved] (SPARK-31572) Improve task logs at executor side

2020-04-27 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-31572. -- Resolution: Won't Fix we shall use debug level to improve logs. > Improve task logs at executor side >

[jira] [Created] (SPARK-31572) Improve task logs at executor side

2020-04-27 Thread wuyi (Jira)
wuyi created SPARK-31572: Summary: Improve task logs at executor side Key: SPARK-31572 URL: https://issues.apache.org/jira/browse/SPARK-31572 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-31529) Remove extra whitespaces in the formatted explain

2020-04-23 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31529: - Description: The formatted explain included extra whitespaces. And even the number of spaces are different

[jira] [Updated] (SPARK-31529) Remove extra whitespaces in the formatted explain

2020-04-23 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31529: - Summary: Remove extra whitespaces in the formatted explain (was: Remove redundant whitespaces in the formatted

[jira] [Created] (SPARK-31529) Remove redundant whitespaces in the formatted explain

2020-04-23 Thread wuyi (Jira)
wuyi created SPARK-31529: Summary: Remove redundant whitespaces in the formatted explain Key: SPARK-31529 URL: https://issues.apache.org/jira/browse/SPARK-31529 Project: Spark Issue Type:

[jira] [Created] (SPARK-31521) The fetch size is not correct when merging blocks into a merged block

2020-04-22 Thread wuyi (Jira)
wuyi created SPARK-31521: Summary: The fetch size is not correct when merging blocks into a merged block Key: SPARK-31521 URL: https://issues.apache.org/jira/browse/SPARK-31521 Project: Spark Issue

[jira] [Updated] (SPARK-31509) Recommend user to disable delay scheduling for barrier taskset

2020-04-21 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31509: - Affects Version/s: (was: 3.1.0) 2.4.0 > Recommend user to disable delay scheduling

[jira] [Updated] (SPARK-31509) Recommend user to disable delay scheduling for barrier taskset

2020-04-21 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31509: - Description: Currently, barrier taskset can only be scheduled when all tasks are launched at the same time. As

[jira] [Updated] (SPARK-31509) Recommend user to disable delay scheduling for barrier taskset

2020-04-21 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31509: - Description: Currently, barrier taskset can only be scheduled when all tasks are launched at the same time. As

[jira] [Updated] (SPARK-31509) Recommend user to disable delay scheduling for barrier taskset

2020-04-21 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31509: - Description: Currently, barrier taskset can only be scheduled when all tasks are launched at same time. As a

[jira] [Created] (SPARK-31509) Recommend user to disable delay scheduling for barrier taskset

2020-04-21 Thread wuyi (Jira)
wuyi created SPARK-31509: Summary: Recommend user to disable delay scheduling for barrier taskset Key: SPARK-31509 URL: https://issues.apache.org/jira/browse/SPARK-31509 Project: Spark Issue Type:

[jira] [Created] (SPARK-31504) Output fields in formatted Explain should have determined order.

2020-04-21 Thread wuyi (Jira)
wuyi created SPARK-31504: Summary: Output fields in formatted Explain should have determined order. Key: SPARK-31504 URL: https://issues.apache.org/jira/browse/SPARK-31504 Project: Spark Issue

[jira] [Created] (SPARK-31495) Support formatted explain for Adaptive Query Execution

2020-04-20 Thread wuyi (Jira)
wuyi created SPARK-31495: Summary: Support formatted explain for Adaptive Query Execution Key: SPARK-31495 URL: https://issues.apache.org/jira/browse/SPARK-31495 Project: Spark Issue Type:

[jira] [Created] (SPARK-31487) Move slots check of barrier job from DAGScheduler to TaskSchedulerImpl

2020-04-19 Thread wuyi (Jira)
wuyi created SPARK-31487: Summary: Move slots check of barrier job from DAGScheduler to TaskSchedulerImpl Key: SPARK-31487 URL: https://issues.apache.org/jira/browse/SPARK-31487 Project: Spark

[jira] [Created] (SPARK-31485) Barrier stage can hang if only partial tasks launched

2020-04-19 Thread wuyi (Jira)
wuyi created SPARK-31485: Summary: Barrier stage can hang if only partial tasks launched Key: SPARK-31485 URL: https://issues.apache.org/jira/browse/SPARK-31485 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-31472) allGather() may return null messages

2020-04-17 Thread wuyi (Jira)
wuyi created SPARK-31472: Summary: allGather() may return null messages Key: SPARK-31472 URL: https://issues.apache.org/jira/browse/SPARK-31472 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-30385) WebUI occasionally throw IOException on stop()

2020-04-16 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-30385: - Environment: MacOS 10.14.6 Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_231 Scala version 2.12.10 > WebUI

[jira] [Updated] (SPARK-30385) WebUI occasionally throw IOException on stop()

2020-04-16 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-30385: - Description: While using ./bin/spark-shell, recently, I occasionally see IOException when I try to quit:

[jira] [Updated] (SPARK-30385) WebUI occasionally throw IOException on stop()

2020-04-16 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-30385: - Description: While using ./bin/spark-shell, recently, I occasionally see IOException when I try to quit:

[jira] [Updated] (SPARK-30385) WebUI occasionally throw IOException on stop()

2020-04-16 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-30385: - Description: While using ./bin/spark-shell, recently, I occasionally see IOException when I try to quit:

[jira] [Updated] (SPARK-31425) UnsafeKVExternalSorter/VariableLengthRowBasedKeyValueBatch should also respect UnsafeAlignedOffset

2020-04-14 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31425: - Summary: UnsafeKVExternalSorter/VariableLengthRowBasedKeyValueBatch should also respect UnsafeAlignedOffset

[jira] [Updated] (SPARK-31407) Fix hive/SQLQuerySuite.derived from Hive query file: drop_database_removes_partition_dirs.q

2020-04-13 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31407: - Description: Test "derived from Hive query file: drop_database_removes_partition_dirs.q" can fail if we run it

[jira] [Updated] (SPARK-31407) Fix hive/SQLQuerySuite.derived from Hive query file: drop_database_removes_partition_dirs.q

2020-04-13 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31407: - Description: Test "derived from Hive query file: drop_database_removes_partition_dirs.q" can fail if we run it

[jira] [Created] (SPARK-31425) UnsafeKVExternalSorter should also respect UnsafeAlignedOffset

2020-04-12 Thread wuyi (Jira)
wuyi created SPARK-31425: Summary: UnsafeKVExternalSorter should also respect UnsafeAlignedOffset Key: SPARK-31425 URL: https://issues.apache.org/jira/browse/SPARK-31425 Project: Spark Issue Type:

[jira] [Updated] (SPARK-30953) InsertAdaptiveSparkPlan should apply AQE on child plan of write commands

2020-04-10 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-30953: - Description: Apply AQE on write commands with child plan will expose  {{LogicalQueryStage}} to {{Analyzer}} 

[jira] [Updated] (SPARK-30953) InsertAdaptiveSparkPlan should apply AQE on child plan of write commands

2020-04-10 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-30953: - Summary: InsertAdaptiveSparkPlan should apply AQE on child plan of write commands (was:

[jira] [Created] (SPARK-31409) Fix failed tests due to result order changing when we enable AQE

2020-04-10 Thread wuyi (Jira)
wuyi created SPARK-31409: Summary: Fix failed tests due to result order changing when we enable AQE Key: SPARK-31409 URL: https://issues.apache.org/jira/browse/SPARK-31409 Project: Spark Issue

[jira] [Created] (SPARK-31407) Fix hive/SQLQuerySuite.derived from Hive query file: drop_database_removes_partition_dirs.q

2020-04-10 Thread wuyi (Jira)
wuyi created SPARK-31407: Summary: Fix hive/SQLQuerySuite.derived from Hive query file: drop_database_removes_partition_dirs.q Key: SPARK-31407 URL: https://issues.apache.org/jira/browse/SPARK-31407 Project:

<    1   2   3   4   5   >