[jira] [Updated] (SPARK-23788) Race condition in StreamingQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23788: - Fix Version/s: 2.2.2 > Race condition in StreamingQuerySuite >

[jira] [Resolved] (SPARK-23788) Race condition in StreamingQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23788. -- Resolution: Fixed Assignee: Jose Torres Fix Version/s: 2.4.0

[jira] [Updated] (SPARK-23623) Avoid concurrent use of cached KafkaConsumer in CachedKafkaConsumer (kafka-0-10-sql)

[ https://issues.apache.org/jira/browse/SPARK-23623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23623: - Fix Version/s: 2.3.1 > Avoid concurrent use of cached KafkaConsumer in CachedKafkaConsumer >

[jira] [Resolved] (SPARK-23623) Avoid concurrent use of cached KafkaConsumer in CachedKafkaConsumer (kafka-0-10-sql)

[ https://issues.apache.org/jira/browse/SPARK-23623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23623. -- Resolution: Fixed Fix Version/s: 2.4.0 > Avoid concurrent use of cached KafkaConsumer

[jira] [Assigned] (SPARK-23533) Add support for changing ContinuousDataReader's startOffset

[ https://issues.apache.org/jira/browse/SPARK-23533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-23533: Assignee: Li Yuanjian > Add support for changing ContinuousDataReader's startOffset >

[jira] [Resolved] (SPARK-23533) Add support for changing ContinuousDataReader's startOffset

[ https://issues.apache.org/jira/browse/SPARK-23533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23533. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20689

[jira] [Updated] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 1.1.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18057: - Summary: Update structured streaming kafka from 0.10.0.1 to 1.1.0 (was: Update structured

[jira] [Updated] (SPARK-23475) The "stages" page doesn't show any completed stages

[ https://issues.apache.org/jira/browse/SPARK-23475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23475: - Target Version/s: 2.3.0 > The "stages" page doesn't show any completed stages >

[jira] [Resolved] (SPARK-23481) The job page shows wrong stages when some of stages are evicted

[ https://issues.apache.org/jira/browse/SPARK-23481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23481. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20654

[jira] [Updated] (SPARK-23475) The "stages" page doesn't show any completed stages

[ https://issues.apache.org/jira/browse/SPARK-23475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23475: - Attachment: (was: Screen Shot 2018-02-21 at 12.39.46 AM.png) > The "stages" page doesn't

[jira] [Commented] (SPARK-23481) The job page shows wrong stages when some of stages are evicted

[ https://issues.apache.org/jira/browse/SPARK-23481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371902#comment-16371902 ] Shixiong Zhu commented on SPARK-23481: -- [~vanzin] Yep. Here is the fix with a regression test:

[jira] [Created] (SPARK-23481) The job page shows wrong stages when some of stages are evicted

Shixiong Zhu created SPARK-23481: Summary: The job page shows wrong stages when some of stages are evicted Key: SPARK-23481 URL: https://issues.apache.org/jira/browse/SPARK-23481 Project: Spark

[jira] [Commented] (SPARK-23475) The "stages" page doesn't show any completed stages

[ https://issues.apache.org/jira/browse/SPARK-23475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371897#comment-16371897 ] Shixiong Zhu commented on SPARK-23475: -- The job page issue is a separated issue. Created SPARK-23481

[jira] [Updated] (SPARK-23481) The job page shows wrong stages when some of stages are evicted

[ https://issues.apache.org/jira/browse/SPARK-23481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23481: - Description: Run "bin/spark-shell --conf spark.ui.retainedJobs=10 --conf

[jira] [Updated] (SPARK-23475) The "stages" page doesn't show any completed stages

[ https://issues.apache.org/jira/browse/SPARK-23475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23475: - Description: Run "bin/spark-shell --conf spark.ui.retainedJobs=10 --conf

[jira] [Updated] (SPARK-23481) The job page shows wrong stages when some of stages are evicted

[ https://issues.apache.org/jira/browse/SPARK-23481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23481: - Description: Run "bin/spark-shell --conf spark.ui.retainedJobs=10 --conf

[jira] [Updated] (SPARK-23481) The job page shows wrong stages when some of stages are evicted

[ https://issues.apache.org/jira/browse/SPARK-23481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23481: - Attachment: Screen Shot 2018-02-21 at 12.39.46 AM.png > The job page shows wrong stages when

[jira] [Updated] (SPARK-23406) Stream-stream self joins does not work

[ https://issues.apache.org/jira/browse/SPARK-23406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23406: - Fix Version/s: (was: 3.0.0) 2.4.0 > Stream-stream self joins does not

[jira] [Commented] (SPARK-23475) The "stages" page doesn't show any completed stages

[ https://issues.apache.org/jira/browse/SPARK-23475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371098#comment-16371098 ] Shixiong Zhu commented on SPARK-23475: -- cc [~vanzin] > The "stages" page doesn't show any completed

[jira] [Created] (SPARK-23475) The "stages" page doesn't show any completed stages

Shixiong Zhu created SPARK-23475: Summary: The "stages" page doesn't show any completed stages Key: SPARK-23475 URL: https://issues.apache.org/jira/browse/SPARK-23475 Project: Spark Issue

[jira] [Updated] (SPARK-23475) The "stages" page doesn't show any completed stages

[ https://issues.apache.org/jira/browse/SPARK-23475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23475: - Attachment: Screen Shot 2018-02-21 at 12.39.46 AM.png Screen Shot 2018-02-21 at

[jira] [Assigned] (SPARK-23434) Spark should not warn `metadata directory` for a HDFS file path

[ https://issues.apache.org/jira/browse/SPARK-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-23434: Assignee: Dongjoon Hyun > Spark should not warn `metadata directory` for a HDFS file path

[jira] [Resolved] (SPARK-23434) Spark should not warn `metadata directory` for a HDFS file path

[ https://issues.apache.org/jira/browse/SPARK-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23434. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20616

[jira] [Updated] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23470: - Description: I was testing 2.3.0 RC3 and found that it's easy to hit "read timeout" when

[jira] [Updated] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23470: - Target Version/s: 2.3.0 > org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too

[jira] [Commented] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370531#comment-16370531 ] Shixiong Zhu commented on SPARK-23470: -- [~vanzin] could you make a PR to fix it? I can help you test

[jira] [Updated] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23470: - Priority: Blocker (was: Major) > org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription

[jira] [Commented] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370496#comment-16370496 ] Shixiong Zhu commented on SPARK-23470: -- [~vanzin] [~cloud_fan] >

[jira] [Created] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

Shixiong Zhu created SPARK-23470: Summary: org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow Key: SPARK-23470 URL: https://issues.apache.org/jira/browse/SPARK-23470 Project:

[jira] [Commented] (SPARK-23433) java.lang.IllegalStateException: more than one active taskSet for stage

[ https://issues.apache.org/jira/browse/SPARK-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367728#comment-16367728 ] Shixiong Zhu commented on SPARK-23433: -- [~irashid] I'm busy with other stuff and not working on

[jira] [Commented] (SPARK-23433) java.lang.IllegalStateException: more than one active taskSet for stage

[ https://issues.apache.org/jira/browse/SPARK-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366421#comment-16366421 ] Shixiong Zhu commented on SPARK-23433: -- cc [~irashid] > java.lang.IllegalStateException: more than

[jira] [Commented] (SPARK-23433) java.lang.IllegalStateException: more than one active taskSet for stage

[ https://issues.apache.org/jira/browse/SPARK-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366417#comment-16366417 ] Shixiong Zhu commented on SPARK-23433: -- {code} 18/02/11 13:22:20 INFO TaskSetManager: Finished task

[jira] [Resolved] (SPARK-23430) Cannot sort "Executor ID" or "Host" columns in the task table

[ https://issues.apache.org/jira/browse/SPARK-23430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23430. -- Resolution: Duplicate > Cannot sort "Executor ID" or "Host" columns in the task table >

[jira] [Created] (SPARK-23433) java.lang.IllegalStateException: more than one active taskSet for stage

Shixiong Zhu created SPARK-23433: Summary: java.lang.IllegalStateException: more than one active taskSet for stage Key: SPARK-23433 URL: https://issues.apache.org/jira/browse/SPARK-23433 Project:

[jira] [Created] (SPARK-23430) Cannot sort "Executor ID" or "Host" columns in the task table

Shixiong Zhu created SPARK-23430: Summary: Cannot sort "Executor ID" or "Host" columns in the task table Key: SPARK-23430 URL: https://issues.apache.org/jira/browse/SPARK-23430 Project: Spark

[jira] [Commented] (SPARK-23351) checkpoint corruption in long running application

[ https://issues.apache.org/jira/browse/SPARK-23351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363115#comment-16363115 ] Shixiong Zhu commented on SPARK-23351: -- [~davidahern] It's better to ask the vendor for support.

[jira] [Updated] (SPARK-23400) Add the extra constructors for ScalaUDF

[ https://issues.apache.org/jira/browse/SPARK-23400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23400: - Fix Version/s: 2.4.0 > Add the extra constructors for ScalaUDF >

[jira] [Resolved] (SPARK-23400) Add the extra constructors for ScalaUDF

[ https://issues.apache.org/jira/browse/SPARK-23400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23400. -- Resolution: Fixed Fix Version/s: 2.3.1 Issue resolved by pull request 20591

[jira] [Commented] (SPARK-23351) checkpoint corruption in long running application

[ https://issues.apache.org/jira/browse/SPARK-23351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357632#comment-16357632 ] Shixiong Zhu commented on SPARK-23351: -- I believe this should be resolved in

[jira] [Commented] (SPARK-23351) checkpoint corruption in long running application

[ https://issues.apache.org/jira/browse/SPARK-23351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357629#comment-16357629 ] Shixiong Zhu commented on SPARK-23351: -- What's your file system? HDFS? > checkpoint corruption in

[jira] [Updated] (SPARK-23326) "Scheduler Delay" of a task is confusing

[ https://issues.apache.org/jira/browse/SPARK-23326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23326: - Description: Run the following code and check the UI {code} sc.makeRDD(1 to 1, 1).foreach { i =>

[jira] [Updated] (SPARK-23326) "Scheduler Delay" of a task is confusing

[ https://issues.apache.org/jira/browse/SPARK-23326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23326: - Environment: (was: Run the following code and check the UI {code} sc.makeRDD(1 to 1,

[jira] [Created] (SPARK-23326) "Scheduler Delay" of a task is confusing

Shixiong Zhu created SPARK-23326: Summary: "Scheduler Delay" of a task is confusing Key: SPARK-23326 URL: https://issues.apache.org/jira/browse/SPARK-23326 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23294) Spark Streaming + Rate source + Console Sink : Receiver MaxRate is violated

[ https://issues.apache.org/jira/browse/SPARK-23294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349411#comment-16349411 ] Shixiong Zhu commented on SPARK-23294: -- [~rmatte] the configurations you posted in the ticket is for

[jira] [Resolved] (SPARK-23294) Spark Streaming + Rate source + Console Sink : Receiver MaxRate is violated

[ https://issues.apache.org/jira/browse/SPARK-23294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23294. -- Resolution: Not A Problem > Spark Streaming + Rate source + Console Sink : Receiver MaxRate is

[jira] [Assigned] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

[ https://issues.apache.org/jira/browse/SPARK-23307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-23307: Assignee: Shixiong Zhu > Spark UI should sort jobs/stages with the completed timestamp

[jira] [Updated] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

[ https://issues.apache.org/jira/browse/SPARK-23307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23307: - Description: When you have a long running job, it may be deleted from UI quickly when it

[jira] [Commented] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

[ https://issues.apache.org/jira/browse/SPARK-23307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349288#comment-16349288 ] Shixiong Zhu commented on SPARK-23307: -- cc [~vanzin] [~cloud_fan] > Spark UI should sort

[jira] [Created] (SPARK-23307) Spark UI should sort jobs/stages with the completed timestamp before cleaning up them

Shixiong Zhu created SPARK-23307: Summary: Spark UI should sort jobs/stages with the completed timestamp before cleaning up them Key: SPARK-23307 URL: https://issues.apache.org/jira/browse/SPARK-23307

[jira] [Created] (SPARK-23289) OneForOneBlockFetcher.DownloadCallback.onData may write just a part of data

Shixiong Zhu created SPARK-23289: Summary: OneForOneBlockFetcher.DownloadCallback.onData may write just a part of data Key: SPARK-23289 URL: https://issues.apache.org/jira/browse/SPARK-23289 Project:

[jira] [Assigned] (SPARK-23245) KafkaContinuousSourceSuite may hang forever

[ https://issues.apache.org/jira/browse/SPARK-23245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-23245: Assignee: Jose Torres > KafkaContinuousSourceSuite may hang forever >

[jira] [Resolved] (SPARK-23245) KafkaContinuousSourceSuite may hang forever

[ https://issues.apache.org/jira/browse/SPARK-23245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23245. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20413

[jira] [Created] (SPARK-23245) KafkaContinuousSourceSuite may hang forever

Shixiong Zhu created SPARK-23245: Summary: KafkaContinuousSourceSuite may hang forever Key: SPARK-23245 URL: https://issues.apache.org/jira/browse/SPARK-23245 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-23242) Don't run tests in KafkaSourceSuiteBase twice

[ https://issues.apache.org/jira/browse/SPARK-23242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23242. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20412

[jira] [Updated] (SPARK-23242) Don't run tests in KafkaSourceSuiteBase twice

[ https://issues.apache.org/jira/browse/SPARK-23242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23242: - Description: KafkaSourceSuiteBase should be abstract class, otherwise KafkaSourceSuiteBase will

[jira] [Updated] (SPARK-23242) Don't run tests in KafkaSourceSuiteBase twice

[ https://issues.apache.org/jira/browse/SPARK-23242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23242: - Component/s: Tests > Don't run tests in KafkaSourceSuiteBase twice >

[jira] [Updated] (SPARK-23242) Don't run tests in KafkaSourceSuiteBase twice

[ https://issues.apache.org/jira/browse/SPARK-23242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23242: - Environment: (was: KafkaSourceSuiteBase should be abstract class, otherwise

[jira] [Created] (SPARK-23242) Don't run tests in KafkaSourceSuiteBase twice

Shixiong Zhu created SPARK-23242: Summary: Don't run tests in KafkaSourceSuiteBase twice Key: SPARK-23242 URL: https://issues.apache.org/jira/browse/SPARK-23242 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338427#comment-16338427 ] Shixiong Zhu commented on SPARK-23206: -- We can also just add more information to metrics system and

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338421#comment-16338421 ] Shixiong Zhu commented on SPARK-23206: -- Also cc [~jerryshao] since you were working on metrics

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338396#comment-16338396 ] Shixiong Zhu commented on SPARK-23206: -- cc [~vanzin]  > Additional Memory Tuning Metrics >

[jira] [Assigned] (SPARK-23198) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution

[ https://issues.apache.org/jira/browse/SPARK-23198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-23198: Assignee: Dongjoon Hyun > Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to

[jira] [Resolved] (SPARK-23198) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution

[ https://issues.apache.org/jira/browse/SPARK-23198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23198. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20374

[jira] [Resolved] (SPARK-23184) All jobs page is broken when some stage is missing

[ https://issues.apache.org/jira/browse/SPARK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23184. -- Resolution: Duplicate Fix Version/s: 2.3.0 > All jobs page is broken when some stage is

[jira] [Commented] (SPARK-23184) All jobs page is broken when some stage is missing

[ https://issues.apache.org/jira/browse/SPARK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335155#comment-16335155 ] Shixiong Zhu commented on SPARK-23184: -- Not yet. But I see it's duplicated after reading the patch.

[jira] [Commented] (SPARK-23184) All jobs page is broken when some stage is missing

[ https://issues.apache.org/jira/browse/SPARK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335131#comment-16335131 ] Shixiong Zhu commented on SPARK-23184: -- cc [~vanzin] [~smurakozi] [~cloud_fan] > All jobs page is

[jira] [Commented] (SPARK-23184) All jobs page is broken when some stage is missing

[ https://issues.apache.org/jira/browse/SPARK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335128#comment-16335128 ] Shixiong Zhu commented on SPARK-23184: -- This seems caused by the fix for SPARK-23051. > All jobs

[jira] [Created] (SPARK-23184) All jobs page is broken when some stage is missing

Shixiong Zhu created SPARK-23184: Summary: All jobs page is broken when some stage is missing Key: SPARK-23184 URL: https://issues.apache.org/jira/browse/SPARK-23184 Project: Spark Issue

[jira] [Assigned] (SPARK-21996) Streaming ignores files with spaces in the file names

[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-21996: Assignee: Xiayun Sun > Streaming ignores files with spaces in the file names >

[jira] [Resolved] (SPARK-23064) Add documentation for stream-stream joins

[ https://issues.apache.org/jira/browse/SPARK-23064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23064. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20255

[jira] [Resolved] (SPARK-21996) Streaming ignores files with spaces in the file names

[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21996. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19247

[jira] [Resolved] (SPARK-23119) Fix API annotation in DataSource V2 for streaming

[ https://issues.apache.org/jira/browse/SPARK-23119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23119. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20286

[jira] [Resolved] (SPARK-23093) don't modify run id

[ https://issues.apache.org/jira/browse/SPARK-23093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23093. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20282

[jira] [Assigned] (SPARK-23093) don't modify run id

[ https://issues.apache.org/jira/browse/SPARK-23093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-23093: Assignee: Jose Torres > don't modify run id > --- > >

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21996: - Component/s: (was: SQL) Structured Streaming > Streaming ignores files with

[jira] [Commented] (SPARK-23050) Structured Streaming with S3 file source duplicates data because of eventual consistency.

[ https://issues.apache.org/jira/browse/SPARK-23050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327632#comment-16327632 ] Shixiong Zhu commented on SPARK-23050: -- [~ste...@apache.org] Yeah, that's a good improvement for S3.

[jira] [Updated] (SPARK-22956) Union Stream Failover Cause `IllegalStateException`

[ https://issues.apache.org/jira/browse/SPARK-22956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-22956: - Fix Version/s: (was: 2.3.0) 2.4.0 2.3.1 > Union Stream

[jira] [Resolved] (SPARK-22956) Union Stream Failover Cause `IllegalStateException`

[ https://issues.apache.org/jira/browse/SPARK-22956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-22956. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20150

[jira] [Assigned] (SPARK-22956) Union Stream Failover Cause `IllegalStateException`

[ https://issues.apache.org/jira/browse/SPARK-22956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-22956: Assignee: Li Yuanjian > Union Stream Failover Cause `IllegalStateException` >

[jira] [Commented] (SPARK-23050) Structured Streaming with S3 file source duplicates data because of eventual consistency.

[ https://issues.apache.org/jira/browse/SPARK-23050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324717#comment-16324717 ] Shixiong Zhu commented on SPARK-23050: -- [~yash...@gmail.com] Spark SQL should handle it. Yeah,

[jira] [Assigned] (SPARK-22975) MetricsReporter producing NullPointerException when there was no progress reported

[ https://issues.apache.org/jira/browse/SPARK-22975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-22975: Assignee: Marco Gaido > MetricsReporter producing NullPointerException when there was no

[jira] [Resolved] (SPARK-22975) MetricsReporter producing NullPointerException when there was no progress reported

[ https://issues.apache.org/jira/browse/SPARK-22975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-22975. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.2 Issue resolved by pull

[jira] [Commented] (SPARK-21475) Change to use NIO's Files API for external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-21475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323617#comment-16323617 ] Shixiong Zhu commented on SPARK-21475: -- Fixed > Change to use NIO's Files API for external shuffle

[jira] [Updated] (SPARK-21475) Change to use NIO's Files API for external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-21475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21475: - Fix Version/s: (was: 3.0.0) > Change to use NIO's Files API for external shuffle service >

[jira] [Commented] (SPARK-23051) job description in Spark UI is broken

[ https://issues.apache.org/jira/browse/SPARK-23051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323360#comment-16323360 ] Shixiong Zhu commented on SPARK-23051: -- I marked this is a blocker since it's a regression. > job

[jira] [Updated] (SPARK-23051) job description in Spark UI is broken

[ https://issues.apache.org/jira/browse/SPARK-23051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23051: - Labels: regression (was: ) > job description in Spark UI is broken >

[jira] [Created] (SPARK-23051) job description in Spark UI is broken

Shixiong Zhu created SPARK-23051: Summary: job description in Spark UI is broken Key: SPARK-23051 URL: https://issues.apache.org/jira/browse/SPARK-23051 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-23050) Structured Streaming with S3 file source duplicates data because of eventual consistency.

[ https://issues.apache.org/jira/browse/SPARK-23050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323320#comment-16323320 ] Shixiong Zhu edited comment on SPARK-23050 at 1/12/18 12:56 AM: How do

[jira] [Commented] (SPARK-23050) Structured Streaming with S3 file source duplicates data because of eventual consistency.

[ https://issues.apache.org/jira/browse/SPARK-23050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323320#comment-16323320 ] Shixiong Zhu commented on SPARK-23050: -- How do you read the output? If you use Spark to read the

[jira] [Resolved] (SPARK-22989) sparkstreaming ui show 0 records when spark-streaming-kafka application restore from checkpoint

[ https://issues.apache.org/jira/browse/SPARK-22989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-22989. -- Resolution: Duplicate > sparkstreaming ui show 0 records when spark-streaming-kafka

[jira] [Updated] (SPARK-22991) High read latency with spark streaming 2.2.1 and kafka 0.10.0.1

[ https://issues.apache.org/jira/browse/SPARK-22991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-22991: - Component/s: (was: Structured Streaming) (was: Spark Core)

[jira] [Updated] (SPARK-22975) MetricsReporter producing NullPointerException when there was no progress reported

[ https://issues.apache.org/jira/browse/SPARK-22975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-22975: - Component/s: (was: SQL) Structured Streaming > MetricsReporter producing

[jira] [Commented] (SPARK-21760) Structured streaming terminates with Exception

[ https://issues.apache.org/jira/browse/SPARK-21760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312089#comment-16312089 ] Shixiong Zhu commented on SPARK-21760: -- Could you try 2.2.1? It's probably fixed by this line:

[jira] [Resolved] (SPARK-21475) Change the usage of FileInputStream/OutputStream to Files.newInput/OutputStream in the critical path

[ https://issues.apache.org/jira/browse/SPARK-21475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21475. -- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by pull

[jira] [Updated] (SPARK-21475) Change to use NIO's Files API for external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-21475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21475: - Summary: Change to use NIO's Files API for external shuffle service (was: Change the usage of

[jira] [Updated] (SPARK-21475) Change the usage of FileInputStream/OutputStream to Files.newInput/OutputStream in the critical path

[ https://issues.apache.org/jira/browse/SPARK-21475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21475: - Fix Version/s: (was: 2.3.0) > Change the usage of FileInputStream/OutputStream to >

[jira] [Resolved] (SPARK-22863) Make MicroBatchExecution also support MicroBatchRead/WriteSupport

[ https://issues.apache.org/jira/browse/SPARK-22863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-22863. -- Resolution: Duplicate > Make MicroBatchExecution also support MicroBatchRead/WriteSupport >

[jira] [Reopened] (SPARK-21475) Change the usage of FileInputStream/OutputStream to Files.newInput/OutputStream in the critical path

[ https://issues.apache.org/jira/browse/SPARK-21475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reopened SPARK-21475: -- I sent https://github.com/apache/spark/pull/20119 to revert this due to a performance regression.

[jira] [Updated] (SPARK-22909) Move Structured Streaming v2 APIs to streaming package

[ https://issues.apache.org/jira/browse/SPARK-22909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-22909: - Target Version/s: 2.3.0 > Move Structured Streaming v2 APIs to streaming package >

[jira] [Updated] (SPARK-22909) Move Structured Streaming v2 APIs to streaming package

[ https://issues.apache.org/jira/browse/SPARK-22909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-22909: - Priority: Blocker (was: Major) > Move Structured Streaming v2 APIs to streaming package >

<    1   2   3   4   5   6   7   8   9   10   >