[jira] [Updated] (SPARK-19874) Hide API docs for "org.apache.spark.sql.internal"

[ https://issues.apache.org/jira/browse/SPARK-19874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19874: - Priority: Minor (was: Major) > Hide API docs for "org.apache.spark.sql.internal" > -

[jira] [Assigned] (SPARK-19540) Add ability to clone SparkSession with an identical copy of the SessionState

[ https://issues.apache.org/jira/browse/SPARK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-19540: Assignee: Kunal Khamar > Add ability to clone SparkSession with an identical copy of the S

[jira] [Resolved] (SPARK-19540) Add ability to clone SparkSession with an identical copy of the SessionState

[ https://issues.apache.org/jira/browse/SPARK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19540. -- Resolution: Fixed Fix Version/s: 2.2.0 > Add ability to clone SparkSession with an ident

[jira] [Created] (SPARK-19874) Hide API docs for "org.apache.spark.sql.internal"

Shixiong Zhu created SPARK-19874: Summary: Hide API docs for "org.apache.spark.sql.internal" Key: SPARK-19874 URL: https://issues.apache.org/jira/browse/SPARK-19874 Project: Spark Issue Type:

[jira] [Updated] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19858: - Affects Version/s: (was: 2.1.1) > Add output mode to flatMapGroupsWithState and disallow inva

[jira] [Resolved] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19858. -- Resolution: Fixed Fix Version/s: 2.2.0 > Add output mode to flatMapGroupsWithState and d

[jira] [Updated] (SPARK-19413) Basic mapGroupsWithState API

[ https://issues.apache.org/jira/browse/SPARK-19413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19413: - Fix Version/s: (was: 2.1.1) > Basic mapGroupsWithState API > > >

[jira] [Commented] (SPARK-19413) Basic mapGroupsWithState API

[ https://issues.apache.org/jira/browse/SPARK-19413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902105#comment-15902105 ] Shixiong Zhu commented on SPARK-19413: -- Reverted the patch from branch 2.1. This fea

[jira] [Updated] (SPARK-19413) Basic mapGroupsWithState API

[ https://issues.apache.org/jira/browse/SPARK-19413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19413: - Target Version/s: 2.2.0 (was: 2.1.1, 2.2.0) > Basic mapGroupsWithState API > ---

[jira] [Issue Comment Deleted] (SPARK-19413) Basic mapGroupsWithState API

[ https://issues.apache.org/jira/browse/SPARK-19413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19413: - Comment: was deleted (was: User 'tdas' has created a pull request for this issue: https://github.

[jira] [Updated] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19858: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-19067 > Add output mode to flatMap

[jira] [Updated] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19858: - Affects Version/s: 2.1.1 > Add output mode to flatMapGroupsWithState and disallow invalid cases >

[jira] [Updated] (SPARK-19481) Fix flaky test: o.a.s.repl.ReplSuite should clone and clean line object in ClosureCleaner

[ https://issues.apache.org/jira/browse/SPARK-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19481: - Fix Version/s: 2.0.3 > Fix flaky test: o.a.s.repl.ReplSuite should clone and clean line object in

[jira] [Resolved] (SPARK-19859) The new watermark should override the old one

[ https://issues.apache.org/jira/browse/SPARK-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19859. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > The new watermark shou

[jira] [Resolved] (SPARK-19841) StreamingDeduplicateExec.watermarkPredicate should filter rows based on keys

[ https://issues.apache.org/jira/browse/SPARK-19841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19841. -- Resolution: Fixed Fix Version/s: 2.2.0 > StreamingDeduplicateExec.watermarkPredicate sho

[jira] [Created] (SPARK-19859) The new watermark should override the old one

Shixiong Zhu created SPARK-19859: Summary: The new watermark should override the old one Key: SPARK-19859 URL: https://issues.apache.org/jira/browse/SPARK-19859 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

Shixiong Zhu created SPARK-19858: Summary: Add output mode to flatMapGroupsWithState and disallow invalid cases Key: SPARK-19858 URL: https://issues.apache.org/jira/browse/SPARK-19858 Project: Spark

[jira] [Updated] (SPARK-19853) Uppercase Kafka topics fail when startingOffsets are SpecificOffsets

[ https://issues.apache.org/jira/browse/SPARK-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19853: - Target Version/s: 2.2.0 > Uppercase Kafka topics fail when startingOffsets are SpecificOffsets >

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900127#comment-15900127 ] Shixiong Zhu commented on SPARK-19764: -- So you don't set an UncaughtExceptionHandler

[jira] [Updated] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19851: - Component/s: (was: Spark Core) > Add support for EVERY and ANY (SOME) aggregates > --

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899980#comment-15899980 ] Shixiong Zhu commented on SPARK-19764: -- [~agesher] Do you have the OOM stack trace?

[jira] [Created] (SPARK-19841) StreamingDeduplicateExec.watermarkPredicate should filter rows based on keys

Shixiong Zhu created SPARK-19841: Summary: StreamingDeduplicateExec.watermarkPredicate should filter rows based on keys Key: SPARK-19841 URL: https://issues.apache.org/jira/browse/SPARK-19841 Project:

[jira] [Commented] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897789#comment-15897789 ] Shixiong Zhu commented on SPARK-19831: -- Cores running in the receive method should b

[jira] [Resolved] (SPARK-19822) CheckpointSuite.testCheckpointedOperation: should not check checkpointFilesOfLatestTime by the PATH string.

[ https://issues.apache.org/jira/browse/SPARK-19822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19822. -- Resolution: Fixed Assignee: Genmao Yu Fix Version/s: 2.2.0 2.

[jira] [Updated] (SPARK-19798) Query returns stale results when tables are modified on other sessions

[ https://issues.apache.org/jira/browse/SPARK-19798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19798: - Component/s: (was: Spark Core) SQL > Query returns stale results when tables

[jira] [Updated] (SPARK-19821) Throw out the Read-only disk information when create file for Shuffle

[ https://issues.apache.org/jira/browse/SPARK-19821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19821: - Priority: Minor (was: Major) > Throw out the Read-only disk information when create file for Shu

[jira] [Commented] (SPARK-19821) Throw out the Read-only disk information when create file for Shuffle

[ https://issues.apache.org/jira/browse/SPARK-19821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896079#comment-15896079 ] Shixiong Zhu commented on SPARK-19821: -- This is more like a Java issue. > Throw out

[jira] [Updated] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

[ https://issues.apache.org/jira/browse/SPARK-19816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19816: - Affects Version/s: (was: 2.2.0) > DataFrameCallbackSuite doesn't recover the log level >

[jira] [Updated] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

[ https://issues.apache.org/jira/browse/SPARK-19816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19816: - Affects Version/s: 2.1.0 > DataFrameCallbackSuite doesn't recover the log level > ---

[jira] [Resolved] (SPARK-19718) Fix flaky test: org.apache.spark.sql.kafka010.KafkaSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false

[ https://issues.apache.org/jira/browse/SPARK-19718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19718. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.2.0 > Fix flaky test:

[jira] [Created] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

Shixiong Zhu created SPARK-19816: Summary: DataFrameCallbackSuite doesn't recover the log level Key: SPARK-19816 URL: https://issues.apache.org/jira/browse/SPARK-19816 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-19774) StreamExecution should call stop() on sources when a stream fails

[ https://issues.apache.org/jira/browse/SPARK-19774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19774. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > StreamExecution should

[jira] [Assigned] (SPARK-19779) structured streaming exist needless tmp file

[ https://issues.apache.org/jira/browse/SPARK-19779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-19779: Assignee: Feng Gui > structured streaming exist needless tmp file > -

[jira] [Resolved] (SPARK-19779) structured streaming exist needless tmp file

[ https://issues.apache.org/jira/browse/SPARK-19779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19779. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 2.0.3

[jira] [Updated] (SPARK-19779) structured streaming exist needless tmp file

[ https://issues.apache.org/jira/browse/SPARK-19779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19779: - Affects Version/s: 2.1.1 2.0.3 > structured streaming exist needless tmp f

[jira] [Updated] (SPARK-19779) structured streaming exist needless tmp file

[ https://issues.apache.org/jira/browse/SPARK-19779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19779: - Affects Version/s: (was: 2.1.0) 2.2.0 > structured streaming exist nee

[jira] [Comment Edited] (SPARK-19788) DataStreamReader/DataStreamWriter.option shall accept user-defined type

[ https://issues.apache.org/jira/browse/SPARK-19788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891722#comment-15891722 ] Shixiong Zhu edited comment on SPARK-19788 at 3/2/17 7:04 AM: -

[jira] [Commented] (SPARK-19788) DataStreamReader/DataStreamWriter.option shall accept user-defined type

[ https://issues.apache.org/jira/browse/SPARK-19788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891722#comment-15891722 ] Shixiong Zhu commented on SPARK-19788: -- I remember that we want to support both Scal

[jira] [Commented] (SPARK-19768) Error for both aggregate and non-aggregate queries in Structured Streaming - "This query does not support recovering from checkpoint location"

[ https://issues.apache.org/jira/browse/SPARK-19768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890546#comment-15890546 ] Shixiong Zhu commented on SPARK-19768: -- Yeah, just recalled that I fixed the error m

[jira] [Resolved] (SPARK-19633) FileSource read from FileSink

[ https://issues.apache.org/jira/browse/SPARK-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19633. -- Resolution: Fixed Assignee: Liwei Lin Fix Version/s: 2.2.0 > FileSource read fr

[jira] [Commented] (SPARK-19768) Error for both aggregate and non-aggregate queries in Structured Streaming - "This query does not support recovering from checkpoint location"

[ https://issues.apache.org/jira/browse/SPARK-19768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889639#comment-15889639 ] Shixiong Zhu commented on SPARK-19768: -- It should work for both aggregate and non-ag

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889611#comment-15889611 ] Shixiong Zhu commented on SPARK-19764: -- These are master and workers. From the maste

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889026#comment-15889026 ] Shixiong Zhu commented on SPARK-19764: -- [~agesher] driver-log-stderr.log is actually

[jira] [Updated] (SPARK-19772) Flaky test: pyspark.streaming.tests.WindowFunctionTests

[ https://issues.apache.org/jira/browse/SPARK-19772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19772: - Component/s: (was: Structured Streaming) DStreams > Flaky test: pyspark.stre

[jira] [Commented] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888939#comment-15888939 ] Shixiong Zhu commented on SPARK-19767: -- Make sure you installed all required librari

[jira] [Commented] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888670#comment-15888670 ] Shixiong Zhu commented on SPARK-19767: -- You can use {{SKIP_API=1 jekyll build}} to b

[jira] [Issue Comment Deleted] (SPARK-19617) Fix the race condition when starting and stopping a query quickly

[ https://issues.apache.org/jira/browse/SPARK-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19617: - Comment: was deleted (was: User 'gf53520' has created a pull request for this issue: https://gith

[jira] [Resolved] (SPARK-19645) structured streaming job restart bug

[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19645. -- Resolution: Duplicate > structured streaming job restart bug >

[jira] [Updated] (SPARK-19677) HDFSBackedStateStoreProvider fails to overwrite existing file

[ https://issues.apache.org/jira/browse/SPARK-19677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19677: - Affects Version/s: 2.0.0 2.0.1 2.0.2 > HDFSBackedSt

[jira] [Resolved] (SPARK-19677) HDFSBackedStateStoreProvider fails to overwrite existing file

[ https://issues.apache.org/jira/browse/SPARK-19677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19677. -- Resolution: Fixed Assignee: Roberto Agostino Vitillo Fix Version/s: 2.2.0

[jira] [Resolved] (SPARK-19768) Error for both aggregate and non-aggregate queries in Structured Streaming - "This query does not support recovering from checkpoint location"

[ https://issues.apache.org/jira/browse/SPARK-19768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19768. -- Resolution: Not A Bug > Error for both aggregate and non-aggregate queries in Structured Str

[jira] [Commented] (SPARK-19768) Error for both aggregate and non-aggregate queries in Structured Streaming - "This query does not support recovering from checkpoint location"

[ https://issues.apache.org/jira/browse/SPARK-19768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888632#comment-15888632 ] Shixiong Zhu commented on SPARK-19768: -- You are using the console sink which doesn't

[jira] [Updated] (SPARK-19768) Error for both aggregate and non-aggregate queries in Structured Streaming - "This query does not support recovering from checkpoint location"

[ https://issues.apache.org/jira/browse/SPARK-19768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19768: - Issue Type: Question (was: Bug) > Error for both aggregate and non-aggregate queries in Struc

[jira] [Commented] (SPARK-19738) Consider adding error handler to DataStreamWriter

[ https://issues.apache.org/jira/browse/SPARK-19738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887092#comment-15887092 ] Shixiong Zhu commented on SPARK-19738: -- [~gaaldornick] could you check if SPARK-1869

[jira] [Updated] (SPARK-19738) Consider adding error handler to DataStreamWriter

[ https://issues.apache.org/jira/browse/SPARK-19738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19738: - Component/s: SQL > Consider adding error handler to DataStreamWriter > --

[jira] [Updated] (SPARK-19751) Create Data frame API fails with a self referencing bean

[ https://issues.apache.org/jira/browse/SPARK-19751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19751: - Component/s: (was: Spark Core) SQL > Create Data frame API fails with a self

[jira] [Resolved] (SPARK-19749) Name socket source with a meaningful name

[ https://issues.apache.org/jira/browse/SPARK-19749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19749. -- Resolution: Fixed Fix Version/s: 2.2.0 > Name socket source with a meaningful name > ---

[jira] [Assigned] (SPARK-19594) StreamingQueryListener fails to handle QueryTerminatedEvent if more then one listeners exists

[ https://issues.apache.org/jira/browse/SPARK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-19594: Assignee: Eyal Zituny > StreamingQueryListener fails to handle QueryTerminatedEvent if mor

[jira] [Resolved] (SPARK-19594) StreamingQueryListener fails to handle QueryTerminatedEvent if more then one listeners exists

[ https://issues.apache.org/jira/browse/SPARK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19594. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > StreamingQueryListener

[jira] [Updated] (SPARK-19718) Fix flaky test: org.apache.spark.sql.kafka010.KafkaSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false

[ https://issues.apache.org/jira/browse/SPARK-19718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19718: - Description: SPARK-19617 changed HDFSMetadataLog to enable interrupts when using the local file

[jira] [Created] (SPARK-19718) Fix flaky test: org.apache.spark.sql.kafka010.KafkaSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false

Shixiong Zhu created SPARK-19718: Summary: Fix flaky test: org.apache.spark.sql.kafka010.KafkaSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false Key: SPARK-19718 URL: https://issues.apac

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879397#comment-15879397 ] Shixiong Zhu commented on SPARK-19644: -- [~deenbandhu] Do you use Scala 2.10 or Scala

[jira] [Resolved] (SPARK-19617) Fix the race condition when starting and stopping a query quickly

[ https://issues.apache.org/jira/browse/SPARK-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19617. -- Resolution: Fixed Fix Version/s: 2.1.1 > Fix the race condition when starting and stoppi

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877276#comment-15877276 ] Shixiong Zhu commented on SPARK-19675: -- Just to clarify one thing: Executors in your

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877262#comment-15877262 ] Shixiong Zhu edited comment on SPARK-19675 at 2/22/17 1:59 AM:

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877262#comment-15877262 ] Shixiong Zhu commented on SPARK-19675: -- [~taroplus] ExecutorClassLoader does try to

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877255#comment-15877255 ] Shixiong Zhu edited comment on SPARK-19675 at 2/22/17 1:53 AM:

[jira] [Updated] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19675: - Issue Type: Improvement (was: Bug) > ExecutorClassLoader loads classes from SystemClassLoader >

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877255#comment-15877255 ] Shixiong Zhu commented on SPARK-19675: -- [~taroplus] Yeah, I should have checked `sbt

[jira] [Created] (SPARK-19690) Join a streaming DataFrame with a batch DataFrame may not work

Shixiong Zhu created SPARK-19690: Summary: Join a streaming DataFrame with a batch DataFrame may not work Key: SPARK-19690 URL: https://issues.apache.org/jira/browse/SPARK-19690 Project: Spark

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876854#comment-15876854 ] Shixiong Zhu edited comment on SPARK-19675 at 2/21/17 10:17 PM: ---

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876854#comment-15876854 ] Shixiong Zhu commented on SPARK-19675: -- [~taroplus] If I understand correctly, SBT l

[jira] [Issue Comment Deleted] (SPARK-19617) Fix the race condition when starting and stopping a query quickly

[ https://issues.apache.org/jira/browse/SPARK-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19617: - Comment: was deleted (was: User 'gf53520' has created a pull request for this issue: https://gith

[jira] [Updated] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19680: - Component/s: (was: Structured Streaming) DStreams > Offsets out of range wit

[jira] [Updated] (SPARK-19497) dropDuplicates with watermark

[ https://issues.apache.org/jira/browse/SPARK-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19497: - Labels: release_notes (was: ) > dropDuplicates with watermark > - >

[jira] [Updated] (SPARK-19617) Fix the race condition when starting and stopping a query quickly

[ https://issues.apache.org/jira/browse/SPARK-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19617: - Fix Version/s: 2.2.0 > Fix the race condition when starting and stopping a query quickly > --

[jira] [Commented] (SPARK-19525) Enable Compression of RDD Checkpoints

[ https://issues.apache.org/jira/browse/SPARK-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872717#comment-15872717 ] Shixiong Zhu commented on SPARK-19525: -- I see. This is RDD checkpointing. Sounds a g

[jira] [Updated] (SPARK-19525) Enable Compression of Spark Streaming Checkpoints

[ https://issues.apache.org/jira/browse/SPARK-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19525: - Component/s: (was: Structured Streaming) Spark Core > Enable Compression of

[jira] [Updated] (SPARK-19525) Enable Compression of RDD Checkpoints

[ https://issues.apache.org/jira/browse/SPARK-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19525: - Summary: Enable Compression of RDD Checkpoints (was: Enable Compression of Spark Streaming Check

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872672#comment-15872672 ] Shixiong Zhu commented on SPARK-19644: -- [~deenbandhu] Could you check the GC root, p

[jira] [Commented] (SPARK-19525) Enable Compression of Spark Streaming Checkpoints

[ https://issues.apache.org/jira/browse/SPARK-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872652#comment-15872652 ] Shixiong Zhu commented on SPARK-19525: -- Hm, Spark should support compression for dat

[jira] [Assigned] (SPARK-19517) KafkaSource fails to initialize partition offsets

[ https://issues.apache.org/jira/browse/SPARK-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-19517: Assignee: Roberto Agostino Vitillo > KafkaSource fails to initialize partition offsets > -

[jira] [Resolved] (SPARK-19517) KafkaSource fails to initialize partition offsets

[ https://issues.apache.org/jira/browse/SPARK-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19517. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > KafkaSource fails to i

[jira] [Commented] (SPARK-19645) structured streaming job restart bug

[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872319#comment-15872319 ] Shixiong Zhu commented on SPARK-19645: -- [~guifengl...@gmail.com] Thanks for reporti

[jira] [Commented] (SPARK-19497) dropDuplicates with watermark

[ https://issues.apache.org/jira/browse/SPARK-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871016#comment-15871016 ] Shixiong Zhu commented on SPARK-19497: -- [~samelamin] Thanks! I just submitted a PR.

[jira] [Updated] (SPARK-19628) Duplicate Spark jobs in 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19628: - Component/s: (was: Spark Core) SQL > Duplicate Spark jobs in 2.1.0 > ---

[jira] [Updated] (SPARK-19617) Fix the race condition when starting and stopping a query quickly

[ https://issues.apache.org/jira/browse/SPARK-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19617: - Description: The streaming thread in StreamExecution uses the following ways to check if it shou

[jira] [Updated] (SPARK-19617) Fix the race condition when starting and stopping a query quickly

[ https://issues.apache.org/jira/browse/SPARK-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19617: - Summary: Fix the race condition when starting and stopping a query quickly (was: Fix a case that

[jira] [Resolved] (SPARK-19603) Fix StreamingQuery explain command

[ https://issues.apache.org/jira/browse/SPARK-19603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19603. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > Fix StreamingQuery exp

[jira] [Created] (SPARK-19617) Fix a case that a query may not stop due to HADOOP-14084

Shixiong Zhu created SPARK-19617: Summary: Fix a case that a query may not stop due to HADOOP-14084 Key: SPARK-19617 URL: https://issues.apache.org/jira/browse/SPARK-19617 Project: Spark Issu

[jira] [Resolved] (SPARK-19599) Clean up HDFSMetadataLog

[ https://issues.apache.org/jira/browse/SPARK-19599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19599. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.2.0

[jira] [Updated] (SPARK-19599) Clean up HDFSMetadataLog

[ https://issues.apache.org/jira/browse/SPARK-19599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19599: - Description: SPARK-19464 removed support for Hadoop 2.5 and earlier, so we can do some cleanup f

[jira] [Updated] (SPARK-19599) Clean up HDFSMetadataLog

[ https://issues.apache.org/jira/browse/SPARK-19599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19599: - Description: SPARK-19464 removed support for Hadoop 2.5 and earlier, so we can do some cleanup f

[jira] [Updated] (SPARK-19599) Clean up HDFSMetadataLog

[ https://issues.apache.org/jira/browse/SPARK-19599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19599: - Description: SPARK-19464 removed support for Hadoop 2.5 and earlier, so we can do some cleanup f

[jira] [Updated] (SPARK-19599) Clean up HDFSMetadataLog

[ https://issues.apache.org/jira/browse/SPARK-19599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19599: - Summary: Clean up HDFSMetadataLog (was: Clean up HDFSMetadataLog for Hadoop 2.6+) > Clean up HD

[jira] [Updated] (SPARK-19599) Clean up HDFSMetadataLog for Hadoop 2.6+

[ https://issues.apache.org/jira/browse/SPARK-19599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19599: - Description: SPARK-19464 removed support for Hadoop 2.5 and earlier, so we can do some cleanup f

[jira] [Commented] (SPARK-19594) StreamingQueryListener fails to handle QueryTerminatedEvent if more then one listeners exists

[ https://issues.apache.org/jira/browse/SPARK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868469#comment-15868469 ] Shixiong Zhu commented on SPARK-19594: -- I suggest that overriding "def postToAll(eve

[jira] [Commented] (SPARK-17689) _temporary files breaks the Spark SQL streaming job.

[ https://issues.apache.org/jira/browse/SPARK-17689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868466#comment-15868466 ] Shixiong Zhu commented on SPARK-17689: -- Just curious: who created "_temporary"? > _

[jira] [Updated] (SPARK-19603) Fix StreamingQuery explain command

[ https://issues.apache.org/jira/browse/SPARK-19603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19603: - Summary: Fix StreamingQuery explain command (was: Fix the stream explain command) > Fix Streami

[jira] [Created] (SPARK-19603) Fix the stream explain command

Shixiong Zhu created SPARK-19603: Summary: Fix the stream explain command Key: SPARK-19603 URL: https://issues.apache.org/jira/browse/SPARK-19603 Project: Spark Issue Type: Bug Comp

<    5   6   7   8   9   10   11   12   13   14   >