[jira] [Updated] (SPARK-26730) Strip redundant AssertNotNull expression for ExpressionEncoder's serializer

2019-01-25 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-26730: - Description: For types like Product, we've already add AssertNotNull when we construct serializer(pls see the

[jira] [Updated] (SPARK-26730) Strip redundant AssertNotNull expression for ExpressionEncoder's serializer

2019-01-25 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-26730: - Description: For types like Product, we've already add AssertNotNull when we construct serializer(pls see the

[jira] [Created] (SPARK-26730) Strip redundant AssertNotNull expression for ExpressionEncoder's serializer

2019-01-25 Thread wuyi (JIRA)
wuyi created SPARK-26730: Summary: Strip redundant AssertNotNull expression for ExpressionEncoder's serializer Key: SPARK-26730 URL: https://issues.apache.org/jira/browse/SPARK-26730 Project: Spark

[jira] [Commented] (SPARK-26439) Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-01-07 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736687#comment-16736687 ] wuyi commented on SPARK-26439: -- [~srowen] ok. > Introduce WorkerOffer reservation mechanism for Barrier

[jira] [Updated] (SPARK-26439) Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2018-12-26 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-26439: - Description: Currently, Barrier TaskSet has a hard requirement that tasks can only be launched in a single

[jira] [Updated] (SPARK-26439) Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2018-12-26 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-26439: - Summary: Introduce WorkerOffer reservation mechanism for Barrier TaskSet (was: Introduce WorkOffer reservation

[jira] [Updated] (SPARK-26439) Introduce WorkOffer reservation mechanism for Barrier TaskSet

2018-12-25 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-26439: - Description: Currently, Barrier TaskSet has a hard requirement that tasks can only be launched in a single

[jira] [Created] (SPARK-26439) Introduce WorkOffer reservation mechanism for Barrier TaskSet

2018-12-25 Thread wuyi (JIRA)
wuyi created SPARK-26439: Summary: Introduce WorkOffer reservation mechanism for Barrier TaskSet Key: SPARK-26439 URL: https://issues.apache.org/jira/browse/SPARK-26439 Project: Spark Issue Type:

[jira] [Created] (SPARK-26431) Update availableSlots by availableCpus for barrier taskset

2018-12-23 Thread wuyi (JIRA)
wuyi created SPARK-26431: Summary: Update availableSlots by availableCpus for barrier taskset Key: SPARK-26431 URL: https://issues.apache.org/jira/browse/SPARK-26431 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-26392) Cancel pending allocate requests by taking locality preference into account

2018-12-18 Thread wuyi (JIRA)
wuyi created SPARK-26392: Summary: Cancel pending allocate requests by taking locality preference into account Key: SPARK-26392 URL: https://issues.apache.org/jira/browse/SPARK-26392 Project: Spark

[jira] [Created] (SPARK-26269) YarnAllocator should have same blacklist behaviour with YARN to maxmize use of cluster resource

2018-12-04 Thread wuyi (JIRA)
wuyi created SPARK-26269: Summary: YarnAllocator should have same blacklist behaviour with YARN to maxmize use of cluster resource Key: SPARK-26269 URL: https://issues.apache.org/jira/browse/SPARK-26269

[jira] [Created] (SPARK-24141) Fix bug in CoarseGrainedSchedulerBackend.killExecutors

2018-05-01 Thread wuyi (JIRA)
wuyi created SPARK-24141: Summary: Fix bug in CoarseGrainedSchedulerBackend.killExecutors Key: SPARK-24141 URL: https://issues.apache.org/jira/browse/SPARK-24141 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24021) Fix bug in BlacklistTracker's updateBlacklistForFetchFailure

2018-04-18 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-24021: - Description: There's a miswrite in BlacklistTracker's updateBlacklistForFetchFailure:   {code:java} val

[jira] [Created] (SPARK-24021) Fix bug in BlacklistTracker's updateBlacklistForFetchFailure

2018-04-18 Thread wuyi (JIRA)
wuyi created SPARK-24021: Summary: Fix bug in BlacklistTracker's updateBlacklistForFetchFailure Key: SPARK-24021 URL: https://issues.apache.org/jira/browse/SPARK-24021 Project: Spark Issue Type:

[jira] [Updated] (SPARK-24011) Cache rdd's immediate parent ShuffleDependencies to accelerate getShuffleDependencies()

2018-04-18 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-24011: - Summary: Cache rdd's immediate parent ShuffleDependencies to accelerate getShuffleDependencies() (was: Cache

[jira] [Created] (SPARK-24011) Cache rdd's immediate parent ShuffleDependency to accelerate getShuffleDependencies()

2018-04-18 Thread wuyi (JIRA)
wuyi created SPARK-24011: Summary: Cache rdd's immediate parent ShuffleDependency to accelerate getShuffleDependencies() Key: SPARK-24011 URL: https://issues.apache.org/jira/browse/SPARK-24011 Project: Spark

[jira] [Updated] (SPARK-23888) speculative task should not run on a given host where another attempt is already running on

2018-04-17 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-23888: - Description:   There's a bug in: {code:java} /** Check whether a task is currently running an attempt on a

[jira] [Commented] (SPARK-23884) hasLaunchedTask should be true when launchedAnyTask be true

2018-04-10 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432149#comment-16432149 ] wuyi commented on SPARK-23884: -- [~gentlewang] why? > hasLaunchedTask should be true when launchedAnyTask be

[jira] [Created] (SPARK-23888) speculative task should not run on a given host where another attempt is already running on

2018-04-06 Thread wuyi (JIRA)
wuyi created SPARK-23888: Summary: speculative task should not run on a given host where another attempt is already running on Key: SPARK-23888 URL: https://issues.apache.org/jira/browse/SPARK-23888 Project:

[jira] [Created] (SPARK-23884) hasLaunchedTask should be true when launchedAnyTask be true

2018-04-06 Thread wuyi (JIRA)
wuyi created SPARK-23884: Summary: hasLaunchedTask should be true when launchedAnyTask be true Key: SPARK-23884 URL: https://issues.apache.org/jira/browse/SPARK-23884 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-22967) VersionSuite failed on Windows caused by Windows format path

2018-01-08 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-22967: - Summary: VersionSuite failed on Windows caused by Windows format path (was: VersionSuite failed on Windows

[jira] [Comment Edited] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-08 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316368#comment-16316368 ] wuyi edited comment on SPARK-22967 at 1/8/18 2:27 PM: -- Maybe, we can give a warning

[jira] [Commented] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-08 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316368#comment-16316368 ] wuyi commented on SPARK-22967: -- Maybe, we can give a warning rather than an exception, since the main goal

[jira] [Commented] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-08 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316262#comment-16316262 ] wuyi commented on SPARK-22967: -- Yeah, it fails the test. > VersionSuite failed on Windows caused by

[jira] [Commented] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-08 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316161#comment-16316161 ] wuyi commented on SPARK-22967: -- I understand what you mean now, and things with Hive go well after I tried

[jira] [Commented] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-05 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314380#comment-16314380 ] wuyi commented on SPARK-22967: -- I'd like to open a PR, but I'm not 100% sure how to fix this bug yet. As you

[jira] [Comment Edited] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-04 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312579#comment-16312579 ] wuyi edited comment on SPARK-22967 at 1/5/18 6:51 AM: -- @[~srowen] was (Author:

[jira] [Commented] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-04 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312579#comment-16312579 ] wuyi commented on SPARK-22967: -- [~srowen] > VersionSuite failed on Windows caused by unescapeSQLString() >

[jira] [Comment Edited] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-04 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312579#comment-16312579 ] wuyi edited comment on SPARK-22967 at 1/5/18 6:51 AM: -- @[~srowen] was (Author:

[jira] [Updated] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-04 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-22967: - Shepherd: Sean Owen > VersionSuite failed on Windows caused by unescapeSQLString() >

[jira] [Updated] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-04 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-22967: - Description: On Windows system, two unit test case would fail while running VersionSuite ("A simple set of

[jira] [Updated] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-04 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-22967: - Description: On Windows system, two unit test case would fail while running VersionSuite ("A simple set of

[jira] [Created] (SPARK-22967) VersionSuite failed on Windows caused by unescapeSQLString()

2018-01-04 Thread wuyi (JIRA)
wuyi created SPARK-22967: Summary: VersionSuite failed on Windows caused by unescapeSQLString() Key: SPARK-22967 URL: https://issues.apache.org/jira/browse/SPARK-22967 Project: Spark Issue Type:

[jira] [Created] (SPARK-22878) Count totalDroppedEvents for LiveListenerBus

2017-12-22 Thread wuyi (JIRA)
wuyi created SPARK-22878: Summary: Count totalDroppedEvents for LiveListenerBus Key: SPARK-22878 URL: https://issues.apache.org/jira/browse/SPARK-22878 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-22873) Init lastReportTimestamp with system current time when start() called in AsyncEventQueue

2017-12-21 Thread wuyi (JIRA)
wuyi created SPARK-22873: Summary: Init lastReportTimestamp with system current time when start() called in AsyncEventQueue Key: SPARK-22873 URL: https://issues.apache.org/jira/browse/SPARK-22873 Project:

[jira] [Commented] (SPARK-22847) Remove the duplicate code in AppStatusListener while assigning schedulingPool for stage

2017-12-20 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16299474#comment-16299474 ] wuyi commented on SPARK-22847: -- [~srowen] So, how can I handle these little mistake properly next time I

[jira] [Created] (SPARK-22847) Remove the duplicate code in AppStatusListener while assigning schedulingPool for stage

2017-12-20 Thread wuyi (JIRA)
wuyi created SPARK-22847: Summary: Remove the duplicate code in AppStatusListener while assigning schedulingPool for stage Key: SPARK-22847 URL: https://issues.apache.org/jira/browse/SPARK-22847 Project:

<    1   2   3   4   5