[jira] [Created] (SPARK-18143) History Server is broken because of the refactoring work in Structured Streaming

Shixiong Zhu created SPARK-18143: Summary: History Server is broken because of the refactoring work in Structured Streaming Key: SPARK-18143 URL: https://issues.apache.org/jira/browse/SPARK-18143

[jira] [Resolved] (SPARK-17813) Maximum data per trigger

[ https://issues.apache.org/jira/browse/SPARK-17813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17813. -- Resolution: Fixed Assignee: Cody Koeninger Fix Version/s: 2.1.0

[jira] [Updated] (SPARK-16963) Change Source API so that sources do not need to keep unbounded state

[ https://issues.apache.org/jira/browse/SPARK-16963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-16963: - Affects Version/s: 2.0.1 > Change Source API so that sources do not need to keep unbounded state

[jira] [Resolved] (SPARK-16963) Change Source API so that sources do not need to keep unbounded state

[ https://issues.apache.org/jira/browse/SPARK-16963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-16963. -- Resolution: Fixed Assignee: Frederick Reiss Fix Version/s: 2.1.0

[jira] [Updated] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-13747: - Affects Version/s: 2.0.1 > Concurrent execution in SQL doesn't work with Scala ForkJoinPool >

[jira] [Resolved] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-13747. -- Resolution: Fixed Fix Version/s: 2.0.2 > Concurrent execution in SQL doesn't work with

[jira] [Resolved] (SPARK-18104) Don't build KafkaSource doc

[ https://issues.apache.org/jira/browse/SPARK-18104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-18104. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Don't build

[jira] [Updated] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-13747: - Fix Version/s: (was: 2.0.0) 2.1.0 > Concurrent execution in SQL doesn't

[jira] [Updated] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-13747: - Target Version/s: 2.0.2, 2.1.0 > Concurrent execution in SQL doesn't work with Scala

[jira] [Updated] (SPARK-16304) LinkageError should not crash Spark executor

[ https://issues.apache.org/jira/browse/SPARK-16304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-16304: - Fix Version/s: 2.0.2 > LinkageError should not crash Spark executor >

[jira] [Created] (SPARK-18104) Don't build KafkaSource doc

Shixiong Zhu created SPARK-18104: Summary: Don't build KafkaSource doc Key: SPARK-18104 URL: https://issues.apache.org/jira/browse/SPARK-18104 Project: Spark Issue Type: Documentation

[jira] [Resolved] (SPARK-17624) Flaky test? StateStoreSuite maintenance

[ https://issues.apache.org/jira/browse/SPARK-17624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17624. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Flaky test?

[jira] [Updated] (SPARK-18044) FileStreamSource should not infer partitions in every batch

[ https://issues.apache.org/jira/browse/SPARK-18044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18044: - Fix Version/s: 2.0.2 > FileStreamSource should not infer partitions in every batch >

[jira] [Updated] (SPARK-17153) [Structured streams] readStream ignores partition columns

[ https://issues.apache.org/jira/browse/SPARK-17153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17153: - Fix Version/s: 2.0.2 > [Structured streams] readStream ignores partition columns >

[jira] [Resolved] (SPARK-17812) More granular control of starting offsets (assign)

[ https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17812. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > More granular control

[jira] [Resolved] (SPARK-18044) FileStreamSource should not infer partitions in every batch

[ https://issues.apache.org/jira/browse/SPARK-18044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-18044. -- Resolution: Fixed Fix Version/s: 2.1.0 > FileStreamSource should not infer partitions

[jira] [Resolved] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17850. -- Resolution: Fixed Assignee: Shixiong Zhu Target Version/s: 2.0.2, 2.1.0

[jira] [Resolved] (SPARK-17929) Deadlock when AM restart and send RemoveExecutor on reset

[ https://issues.apache.org/jira/browse/SPARK-17929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17929. -- Resolution: Fixed Assignee: Fei Wang Fix Version/s: 2.1.0

[jira] [Updated] (SPARK-17999) Add getPreferredLocations for KafkaSourceRDD

[ https://issues.apache.org/jira/browse/SPARK-17999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17999: - Assignee: Saisai Shao > Add getPreferredLocations for KafkaSourceRDD >

[jira] [Resolved] (SPARK-17999) Add getPreferredLocations for KafkaSourceRDD

[ https://issues.apache.org/jira/browse/SPARK-17999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17999. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Add

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589729#comment-15589729 ] Shixiong Zhu commented on SPARK-17463: -- I could not figure out the cause. Is it easy to reproduce in

[jira] [Commented] (SPARK-17630) jvm-exit-on-fatal-error handler for spark.rpc.netty like there is available for akka

[ https://issues.apache.org/jira/browse/SPARK-17630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589463#comment-15589463 ] Shixiong Zhu commented on SPARK-17630: -- Just set up SparkUncaughtExceptionHandler as the default

[jira] [Commented] (SPARK-17630) jvm-exit-on-fatal-error handler for spark.rpc.netty like there is available for akka

[ https://issues.apache.org/jira/browse/SPARK-17630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586979#comment-15586979 ] Shixiong Zhu commented on SPARK-17630: -- Yeah, I think we can set up SparkUncaughtExceptionHandler

[jira] [Resolved] (SPARK-17930) The SerializerInstance instance used when deserializing a TaskResult is not reused

[ https://issues.apache.org/jira/browse/SPARK-17930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17930. -- Resolution: Fixed Assignee: Guoqiang Li Fix Version/s: 2.1.0 > The

[jira] [Commented] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583454#comment-15583454 ] Shixiong Zhu commented on SPARK-13747: -- [~chinwei] Could you test

[jira] [Reopened] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reopened SPARK-13747: -- Assignee: Shixiong Zhu (was: Andrew Or) There are other places need to be fixed. >

[jira] [Resolved] (SPARK-17839) Use Nio's directbuffer instead of BufferedInputStream in order to avoid additional copy from os buffer cache to user buffer

[ https://issues.apache.org/jira/browse/SPARK-17839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17839. -- Resolution: Fixed Assignee: Sital Kedia Fix Version/s: 2.1.0 > Use Nio's

[jira] [Commented] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15581094#comment-15581094 ] Shixiong Zhu commented on SPARK-13747: -- Could you post the full stack track, please? It would be

[jira] [Commented] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15579284#comment-15579284 ] Shixiong Zhu commented on SPARK-13747: -- [~chinwei] could you post the stack trace here? >

[jira] [Updated] (SPARK-17678) Spark 1.6 Scala-2.11 repl doesn't honor "spark.replClassServer.port"

[ https://issues.apache.org/jira/browse/SPARK-17678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17678: - Affects Version/s: (was: 1.6.3) 1.6.2 > Spark 1.6 Scala-2.11 repl

[jira] [Resolved] (SPARK-17678) Spark 1.6 Scala-2.11 repl doesn't honor "spark.replClassServer.port"

[ https://issues.apache.org/jira/browse/SPARK-17678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17678. -- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 1.6.3 > Spark 1.6

[jira] [Resolved] (SPARK-17834) Fetch the earliest offsets manually in KafkaSource instead of counting on KafkaConsumer

[ https://issues.apache.org/jira/browse/SPARK-17834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17834. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Fetch the earliest

[jira] [Resolved] (SPARK-17876) Write StructuredStreaming WAL to a stream instead of materializing all at once

[ https://issues.apache.org/jira/browse/SPARK-17876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17876. -- Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 2.1.0

[jira] [Updated] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17850: - Fix Version/s: 2.1.0 2.0.2 > HadoopRDD should not swallow EOFException >

[jira] [Resolved] (SPARK-17782) Kafka 010 test is flaky

[ https://issues.apache.org/jira/browse/SPARK-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17782. -- Resolution: Fixed Assignee: Cody Koeninger Resolved by

[jira] [Updated] (SPARK-17782) Kafka 010 test is flaky

[ https://issues.apache.org/jira/browse/SPARK-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17782: - Fix Version/s: 2.1.0 2.0.2 > Kafka 010 test is flaky >

[jira] [Updated] (SPARK-17782) Kafka 010 test is flaky

[ https://issues.apache.org/jira/browse/SPARK-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17782: - Affects Version/s: 2.0.0 2.0.1 > Kafka 010 test is flaky >

[jira] [Updated] (SPARK-17834) Fetch the earliest offsets manually in KafkaSource instead of counting on KafkaConsumer

[ https://issues.apache.org/jira/browse/SPARK-17834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17834: - Target Version/s: 2.0.2, 2.1.0 > Fetch the earliest offsets manually in KafkaSource instead of

[jira] [Updated] (SPARK-17816) Json serialzation of accumulators are failing with ConcurrentModificationException

[ https://issues.apache.org/jira/browse/SPARK-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17816: - Fix Version/s: 2.0.2 > Json serialzation of accumulators are failing with >

[jira] [Updated] (SPARK-17816) Json serialzation of accumulators are failing with ConcurrentModificationException

[ https://issues.apache.org/jira/browse/SPARK-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17816: - Affects Version/s: 2.0.1 > Json serialzation of accumulators are failing with >

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566391#comment-15566391 ] Shixiong Zhu commented on SPARK-17463: -- Do you have a reproducer? I saw `at

[jira] [Updated] (SPARK-17858) Provide option for Spark SQL to skip corrupt files

[ https://issues.apache.org/jira/browse/SPARK-17858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17858: - Description: In Spark 2.0, corrupt files will fail a SQL query. However, the user may just want

[jira] [Resolved] (SPARK-17816) Json serialzation of accumulators are failing with ConcurrentModificationException

[ https://issues.apache.org/jira/browse/SPARK-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17816. -- Resolution: Fixed Assignee: Ergin Seyfe Fix Version/s: 2.1.0 > Json

[jira] [Updated] (SPARK-15814) Aggregator can return null result

[ https://issues.apache.org/jira/browse/SPARK-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-15814: - Fix Version/s: 2.0.1 2.1.0 > Aggregator can return null result >

[jira] [Updated] (SPARK-17738) Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP append/extract

[ https://issues.apache.org/jira/browse/SPARK-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17738: - Component/s: Tests > Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP >

[jira] [Updated] (SPARK-17738) Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP append/extract

[ https://issues.apache.org/jira/browse/SPARK-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17738: - Labels: flaky-test (was: ) > Flaky test:

[jira] [Updated] (SPARK-17738) Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP append/extract

[ https://issues.apache.org/jira/browse/SPARK-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17738: - Issue Type: Test (was: Bug) > Flaky test:

[jira] [Resolved] (SPARK-17738) Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP append/extract

[ https://issues.apache.org/jira/browse/SPARK-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17738. -- Resolution: Fixed Fix Version/s: 2.0.2 > Flaky test:

[jira] [Updated] (SPARK-17858) Provide option for Spark SQL to skip corrupt files

[ https://issues.apache.org/jira/browse/SPARK-17858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17858: - Description: In Spark 2.0, corrupt files will fail a SQL query. However, the user may just want

[jira] [Updated] (SPARK-17858) Provide option for Spark SQL to skip corrupt files

[ https://issues.apache.org/jira/browse/SPARK-17858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17858: - Description: In Spark 2.0, corrupt files will fail a job. However, the user may just want to

[jira] [Updated] (SPARK-17858) Provide option for Spark SQL to skip corrupt files

[ https://issues.apache.org/jira/browse/SPARK-17858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17858: - Issue Type: Improvement (was: Bug) > Provide option for Spark SQL to skip corrupt files >

[jira] [Updated] (SPARK-17858) Provide option for Spark SQL to skip corrupt files

[ https://issues.apache.org/jira/browse/SPARK-17858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17858: - Description: In Spark 2.0, corrupt files will fail a job. However, the user may not > Provide

[jira] [Created] (SPARK-17858) Provide option for Spark SQL to skip corrupt files

Shixiong Zhu created SPARK-17858: Summary: Provide option for Spark SQL to skip corrupt files Key: SPARK-17858 URL: https://issues.apache.org/jira/browse/SPARK-17858 Project: Spark Issue

[jira] [Updated] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17850: - Description: The code in

[jira] [Updated] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17850: - Description: The code in

[jira] [Updated] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17850: - Description: The code in

[jira] [Updated] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17850: - Affects Version/s: 1.4.1 1.5.2 > HadoopRDD should not swallow

[jira] [Updated] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17850: - Labels: correctness (was: ) > HadoopRDD should not swallow EOFException >

[jira] [Updated] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17850: - Description: The code in

[jira] [Updated] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17850: - Description: The code in

[jira] [Updated] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17850: - Description: The code in

[jira] [Created] (SPARK-17850) HadoopRDD should not swallow EOFException

Shixiong Zhu created SPARK-17850: Summary: HadoopRDD should not swallow EOFException Key: SPARK-17850 URL: https://issues.apache.org/jira/browse/SPARK-17850 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-17850) HadoopRDD should not swallow EOFException

[ https://issues.apache.org/jira/browse/SPARK-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17850: - Affects Version/s: 1.6.2 2.0.1 > HadoopRDD should not swallow

[jira] [Updated] (SPARK-17834) Fetch the earliest offsets manually in KafkaSource instead of counting on KafkaConsumer

[ https://issues.apache.org/jira/browse/SPARK-17834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17834: - Issue Type: Sub-task (was: Bug) Parent: SPARK-15406 > Fetch the earliest offsets

[jira] [Updated] (SPARK-17834) Fetch the earliest offsets manually in KafkaSource instead of counting on KafkaConsumer

[ https://issues.apache.org/jira/browse/SPARK-17834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17834: - Summary: Fetch the earliest offsets manually in KafkaSource instead of counting on KafkaConsumer

[jira] [Created] (SPARK-17834) Fetch the initial offsets manually in KafkaSource instead of counting on KafkaConsumer

Shixiong Zhu created SPARK-17834: Summary: Fetch the initial offsets manually in KafkaSource instead of counting on KafkaConsumer Key: SPARK-17834 URL: https://issues.apache.org/jira/browse/SPARK-17834

[jira] [Updated] (SPARK-17346) Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17346: - Fix Version/s: 2.0.2 > Kafka 0.10 support in Structured Streaming >

[jira] [Assigned] (SPARK-17346) Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-17346: Assignee: Shixiong Zhu > Kafka 0.10 support in Structured Streaming >

[jira] [Reopened] (SPARK-17738) Flaky test: org.apache.spark.sql.execution.columnar.ColumnTypeSuite MAP append/extract

[ https://issues.apache.org/jira/browse/SPARK-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reopened SPARK-17738: -- It's still flaky. See:

[jira] [Resolved] (SPARK-17707) Web UI prevents spark-submit application to be finished

[ https://issues.apache.org/jira/browse/SPARK-17707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17707. -- Resolution: Fixed Assignee: Sean Owen Fix Version/s: 2.1.0

[jira] [Resolved] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17463. -- Resolution: Fixed It's already fixed. The issue in the comment is another one. >

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553134#comment-15553134 ] Shixiong Zhu commented on SPARK-17463: -- [~eseyfe] It's a different issue. Could you create a new

[jira] [Resolved] (SPARK-17798) Remove redundant Experimental annotations in sql.streaming package

[ https://issues.apache.org/jira/browse/SPARK-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17798. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Remove redundant

[jira] [Updated] (SPARK-17643) Remove comparable requirement from Offset

[ https://issues.apache.org/jira/browse/SPARK-17643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17643: - Fix Version/s: 2.0.2 > Remove comparable requirement from Offset >

[jira] [Resolved] (SPARK-17778) Mock SparkContext to reduce memory usage of BlockManagerSuite

[ https://issues.apache.org/jira/browse/SPARK-17778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17778. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Mock SparkContext to

[jira] [Updated] (SPARK-17780) Report NoClassDefFoundError in StreamExecution

[ https://issues.apache.org/jira/browse/SPARK-17780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17780: - Component/s: SQL > Report NoClassDefFoundError in StreamExecution >

[jira] [Created] (SPARK-17780) Report NoClassDefFoundError in StreamExecution

Shixiong Zhu created SPARK-17780: Summary: Report NoClassDefFoundError in StreamExecution Key: SPARK-17780 URL: https://issues.apache.org/jira/browse/SPARK-17780 Project: Spark Issue Type:

[jira] [Updated] (SPARK-17780) Report NoClassDefFoundError in StreamExecution

[ https://issues.apache.org/jira/browse/SPARK-17780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17780: - Issue Type: Improvement (was: Test) > Report NoClassDefFoundError in StreamExecution >

[jira] [Updated] (SPARK-17780) Report NoClassDefFoundError in StreamExecution

[ https://issues.apache.org/jira/browse/SPARK-17780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17780: - Affects Version/s: 2.0.1 > Report NoClassDefFoundError in StreamExecution >

[jira] [Created] (SPARK-17778) Mock SparkContext to reduce memory usage of BlockManagerSuite

Shixiong Zhu created SPARK-17778: Summary: Mock SparkContext to reduce memory usage of BlockManagerSuite Key: SPARK-17778 URL: https://issues.apache.org/jira/browse/SPARK-17778 Project: Spark

[jira] [Updated] (SPARK-17696) Race in CoarseGrainedExecutorBackend shutdown can lead to wrong exit status

[ https://issues.apache.org/jira/browse/SPARK-17696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17696: - Assignee: Charles Allen > Race in CoarseGrainedExecutorBackend shutdown can lead to wrong exit

[jira] [Resolved] (SPARK-17696) Race in CoarseGrainedExecutorBackend shutdown can lead to wrong exit status

[ https://issues.apache.org/jira/browse/SPARK-17696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17696. -- Resolution: Fixed Fix Version/s: 2.0.0 1.6.3 > Race in

[jira] [Commented] (SPARK-17649) Log how many Spark events got dropped in LiveListenerBus

[ https://issues.apache.org/jira/browse/SPARK-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530301#comment-15530301 ] Shixiong Zhu commented on SPARK-17649: -- My motivation of logging the dropped events is for helping

[jira] [Updated] (SPARK-17638) Stop JVM StreamingContext when the Python process is dead

[ https://issues.apache.org/jira/browse/SPARK-17638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17638: - Fix Version/s: (was: 2.0.2) 2.0.1 > Stop JVM StreamingContext when the

[jira] [Updated] (SPARK-17649) Log how many Spark events got dropped in LiveListenerBus

[ https://issues.apache.org/jira/browse/SPARK-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17649: - Fix Version/s: 1.6.3 > Log how many Spark events got dropped in LiveListenerBus >

[jira] [Resolved] (SPARK-17649) Log how many Spark events got dropped in LiveListenerBus

[ https://issues.apache.org/jira/browse/SPARK-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17649. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Log how many Spark

[jira] [Resolved] (SPARK-17650) Adding a malformed URL to sc.addJar and/or sc.addFile bricks Executors

[ https://issues.apache.org/jira/browse/SPARK-17650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17650. -- Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 2.1.0

[jira] [Updated] (SPARK-15703) Make ListenerBus event queue size configurable

[ https://issues.apache.org/jira/browse/SPARK-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-15703: - Fix Version/s: 2.0.2 > Make ListenerBus event queue size configurable >

[jira] [Updated] (SPARK-17649) Log how many Spark events got dropped in LiveListenerBus

[ https://issues.apache.org/jira/browse/SPARK-17649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17649: - Priority: Minor (was: Major) > Log how many Spark events got dropped in LiveListenerBus >

[jira] [Created] (SPARK-17649) Log how many Spark events got dropped in LiveListenerBus

Shixiong Zhu created SPARK-17649: Summary: Log how many Spark events got dropped in LiveListenerBus Key: SPARK-17649 URL: https://issues.apache.org/jira/browse/SPARK-17649 Project: Spark

[jira] [Resolved] (SPARK-17640) Avoid using -1 as the default batchId for FileStreamSource.FileEntry

[ https://issues.apache.org/jira/browse/SPARK-17640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17640. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Avoid using -1 as the

[jira] [Created] (SPARK-17640) Avoid using -1 as the default batchId for FileStreamSource.FileEntry

Shixiong Zhu created SPARK-17640: Summary: Avoid using -1 as the default batchId for FileStreamSource.FileEntry Key: SPARK-17640 URL: https://issues.apache.org/jira/browse/SPARK-17640 Project: Spark

[jira] [Resolved] (SPARK-17638) Stop JVM StreamingContext when the Python process is dead

[ https://issues.apache.org/jira/browse/SPARK-17638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17638. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Stop JVM

[jira] [Created] (SPARK-17638) Stop JVM StreamingContext when the Python process is dead

Shixiong Zhu created SPARK-17638: Summary: Stop JVM StreamingContext when the Python process is dead Key: SPARK-17638 URL: https://issues.apache.org/jira/browse/SPARK-17638 Project: Spark

[jira] [Resolved] (SPARK-17569) Don't recheck existence of files when generating File Relation resolution in StructuredStreaming

[ https://issues.apache.org/jira/browse/SPARK-17569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17569. -- Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 2.1.0 > Don't recheck

[jira] [Resolved] (SPARK-4563) Allow spark driver to bind to different ip then advertise ip

[ https://issues.apache.org/jira/browse/SPARK-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-4563. - Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.1.0 > Allow spark

[jira] [Commented] (SPARK-14849) shuffle broken when accessing standalone cluster through NAT

[ https://issues.apache.org/jira/browse/SPARK-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510745#comment-15510745 ] Shixiong Zhu commented on SPARK-14849: -- [~skyluc] do you still see the error in Spark 2.0.0? >

[jira] [Resolved] (SPARK-15698) Ability to remove old metadata for structure streaming MetadataLog

[ https://issues.apache.org/jira/browse/SPARK-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-15698. -- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-15698) Ability to remove old metadata for structure streaming MetadataLog

[ https://issues.apache.org/jira/browse/SPARK-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507313#comment-15507313 ] Shixiong Zhu commented on SPARK-15698: -- I just merged the PR and sent a backport for branch 2.0.

[jira] [Commented] (SPARK-15472) Add support for writing partitioned `csv`, `json`, `text` formats in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502305#comment-15502305 ] Shixiong Zhu commented on SPARK-15472: -- Not yet. Right now we don't use partition information of

<    9   10   11   12   13   14   15   16   17   18   >