Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13595
Thanks!
This patch introduced an compilation error because `DataFrameReader.text`'s
return type had been changed back to `DataFrame` very recently, and I should
have noticed this and updated
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13606
@srowen , the [[streaming programming guide] -
accumulators-and-broadcast-variables](https://github.com/apache/spark/blob/1e2c9311871968426e019164b129652fd6d0037f/docs/streaming-programming-guide.md
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13595
@zsxwing @tdas, sure, this can wait. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13597
@marmbrus @cloud-fan @zsxwing , would you mind taking a look? Thanks! :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13595
@tdas @zsxwing , would you mind taking a look? Thanks! :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13597#discussion_r66596861
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -572,8 +573,13 @@ final class DataFrameWriter[T] private[sql](ds
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/13597
[SPARK-15871][SQL] Add `assertNotPartitioned` check in `DataFrameWriter`
## What changes were proposed in this pull request?
Sometimes it doesn't make sense to specify partitioning
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13595#discussion_r66589007
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataFrameReaderWriterSuite.scala
---
@@ -371,66 +371,80 @@ class
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13595#discussion_r66588898
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -433,8 +433,7 @@ final class DataFrameWriter[T] private[sql](ds:
Dataset
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/13595
[MINOR][SQL] Standardize 'continuous queries' to 'streaming
Datasets/DataFrames'
## What changes were proposed in this pull request?
This patch does some replacing (since `streaming
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13575#discussion_r66467191
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextFileFormat.scala
---
@@ -120,24 +109,31 @@ class TextFileFormat
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13575#discussion_r66467095
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala
---
@@ -488,7 +488,12 @@ private[sql] class
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13575#discussion_r66466672
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala
---
@@ -146,16 +173,53 @@ class JsonFileFormat
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13575#discussion_r66466502
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVRelation.scala
---
@@ -143,39 +146,99 @@ object CSVRelation extends
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13575#discussion_r66466310
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVRelation.scala
---
@@ -143,39 +146,99 @@ object CSVRelation extends
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13575#discussion_r66465201
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -246,7 +247,12 @@ case class DataSource
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13575
@marmbrus @tdas @zsxwing , would you mind taking a look? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/13575
[SPARK-15472][SQL] Add support for writing in `csv`, `json`, `text` formats
in Structured Streaming
## What changes were proposed in this pull request?
This patch adds support for writing
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/13518
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13518
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13518
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user lw-lin reopened a pull request:
https://github.com/apache/spark/pull/13518
[WIP][SPARK-15472][SQL] Add support for writing in `csv`, `json`, `text`
formats in Structured Streaming
## What changes were proposed in this pull request?
This patch adds support
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/13518
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13518
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user lw-lin reopened a pull request:
https://github.com/apache/spark/pull/13518
[WIP][SPARK-15472][SQL] Add support for writing in `csv`, `json`, `text`
formats in Structured Streaming
## What changes were proposed in this pull request?
This patch adds support
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/13518
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13518
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user lw-lin reopened a pull request:
https://github.com/apache/spark/pull/13518
[SPARK-15472][SQL] Add support for writing in `csv`, `json`, `text` formats
in Structured Streaming
## What changes were proposed in this pull request?
This patch adds support
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/13518
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/13258
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/13518
[SPARK-15472][SQL] Add support for writing in `csv`, `json`, `text` formats
in Structured Streaming
## What changes were proposed in this pull request?
This patch adds support for writing
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13515#discussion_r65812444
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -366,7 +366,7 @@ object SparkSubmit
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13507
@liancheng @tdas @zsxwing would you mind taking a look? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/13507
[SPARK-15765][SQL][Streaming] Make continuous Parquet writing consistent
with non-consistent Parquet writing
## What changes were proposed in this pull request?
Currently there are some
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13258
@zsxwing , yeah let me update this in the next one day or so - I was
waiting for https://github.com/apache/spark/pull/13431. Thanks for the reminder!
---
If your project is set up for it, you can
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/12981
@srowen, there are no other examples that need an update :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12981
@rxin sure, I'll resolve the conflicts very soon; and once the Java APIs
are updated in the next couple of days, I'll update the Java examples
accordingly.
Thank you for bringing
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/13258#issuecomment-222035774
@zsxwing sure let's add an abstract layer. I'll rebase and do this in the
next two days or so. Thanks for the review! :-)
---
If your project is set up for it, you can
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/13258#issuecomment-220917708
@marmbrus @tdas @zsxwing would you mind taking a look? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/13258
[SPARK-15472][SQL][Streaming] Add partitioned `csv`, `json`, `text` format
support for FileStreamSink
## What changes were proposed in this pull request?
This patch adds partitioned `csv
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/13251
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/13251#issuecomment-220833527
This is still WIP; will reopen soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/13251#issuecomment-220826352
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/13251
[SPARK-15472][SQL][Streaming] Add support for partitioned `text` format in
FileStreamSink
## What changes were proposed in this pull request?
This patch adds support for partitioned `text
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12725#issuecomment-219336043
@marmbrus @zsxwing maybe this is ready to go? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12725#issuecomment-218016900
@zsxwing would you take another look? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12981#issuecomment-217689734
Once the AccumulatorV2 Java API is finalized, I would update the Java &
Python part. Thanks
---
If your project is set up for it, you can reply to this email and
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/12981
[WIP][SPARK-15208][Core][Docs] Update spark examples with AccumulatorV2
## What changes were proposed in this pull request?
The patch updates the codes & docs in the example module as
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12725#issuecomment-217608390
I've addressed comments and expanded tests; @zsxwing would you mind taking
another look? Thanks!
---
If your project is set up for it, you can reply to this email
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12725#discussion_r62410545
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/IncrementalExecution.scala
---
@@ -27,12 +27,12 @@ import
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12725#discussion_r62410548
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala
---
@@ -122,7 +122,7 @@ class StreamExecution
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12797#issuecomment-216705209
@marmbrus @zsxwing thank you for the patient review! :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12797#discussion_r61977204
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TriggerExecutor.scala
---
@@ -65,8 +65,13 @@ case class ProcessingTimeExecutor
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12797#discussion_r61976675
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TriggerExecutor.scala
---
@@ -65,8 +65,13 @@ case class ProcessingTimeExecutor
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12797#discussion_r61976475
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/StreamTest.scala ---
@@ -142,7 +142,10 @@ trait StreamTest extends QueryTest with Timeouts
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12797#discussion_r61837747
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/ProcessingTimeExecutorSuite.scala
---
@@ -21,19 +21,41 @@ import
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12797#discussion_r61837727
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TriggerExecutor.scala
---
@@ -65,8 +65,22 @@ case class ProcessingTimeExecutor
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12797#discussion_r61837469
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TriggerExecutor.scala
---
@@ -65,8 +65,22 @@ case class ProcessingTimeExecutor
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12650#issuecomment-216427977
@zsxwing I've made updates per your comments; would you take a another
look? Thanks!
---
If your project is set up for it, you can reply to this email and have your
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12650#discussion_r61835377
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala ---
@@ -46,7 +45,11 @@ private[sql] object SQLExecution {
val
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12650#discussion_r61835312
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala
---
@@ -99,7 +99,15 @@ class StreamExecution
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12797#issuecomment-215928373
@marmbrus @tdas @zsxwing would you mind taking a look? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12725#issuecomment-215927916
For things to be easy to review, I've added the manual timed executor for
testing general cases in [a separate
PR](https://github.com/apache/spark/pull/12797). When
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12797#discussion_r61663843
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -136,6 +136,22 @@ class StreamSuite extends StreamTest
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12797#discussion_r61663787
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/ProcessingTimeExecutorSuite.scala
---
@@ -21,19 +21,41 @@ import
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/12797
[SPARK-15022][SPARK-15023] Add support for testing against the
`ProcessingTime(intervalMS > 0)` trigger and `ManualClock`
## What changes were proposed in this pull request?
Curren
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12797#issuecomment-215924685
still editing, will explain why ProcessingTimeExecutor, where for a batch
it should run batchRunner only once but might run multiple times under certain
conditions
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12725#issuecomment-215405018
Sure, I'll add a manual timed executor and some dedicated tests as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12681#issuecomment-214983254
@davies thanks for the review & merging :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12725#issuecomment-214979767
@marmbrus @tdas @zsxwing would you mind taking a look? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/12725
[SPARK-14942][SQL][Streaming] First construct a batch then run the batch
for continuous queries
## Problem
Currently in `StreamExecution`, we first run the batch, then construct the
next
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12638#issuecomment-214943354
just rebase to master to resolve some conflicts
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12638#discussion_r61193722
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala
---
@@ -88,7 +88,7 @@ class FileStreamSource
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12681#issuecomment-214939654
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12681#issuecomment-214753491
Some flaky tests unrelated to this PR
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12638#issuecomment-214714643
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12681#issuecomment-214713457
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12521#discussion_r61041131
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/DataFrameReaderWriterSuite.scala
---
@@ -368,4 +368,79 @@ class DataFrameReaderWriterSuite
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12681#issuecomment-214617805
Some flaky tests not caused by this PR.
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12638#issuecomment-214609215
some build issues unrelated to this PR.
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12638#issuecomment-214606875
@marmbrus thanks for the patient reminder!
Since I've reverted the renaming, and I've checked there's no other
completely unused class under
GitHub user lw-lin reopened a pull request:
https://github.com/apache/spark/pull/12638
[SPARK-14874][SQL][Streaming] Remove the obsolete Batch representation
## What changes were proposed in this pull request?
The `Batch` class, which had been used to indicate progress
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12681#issuecomment-214605511
@davies (who made the first change) might want to take a look?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12681#issuecomment-214601397
Actually this wouldn't cause any problem and wouldn't fail any test suits
**_for now_**, because the read of `acquiredButNotUsed` is guaranteed to see
most recent value
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/12681
[SPARK-14911][Core] Fix a potential data race in TaskMemoryManager
## What changes were proposed in this pull request?
[[SPARK-13210][SQL] catch OOM when allocate memory and expand
array
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12638#issuecomment-214586964
Sure, so I'm closing this PR since the removal itself is not worthy for
committers to process.
@marmbrus thanks for the review!
---
If your project is set up
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/12638
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12521#issuecomment-214585357
Updates: thanks to [[SPARK-14473][SQL] Define analysis rules to catch
operations not supported in
streaming](https://github.com/apache/spark/commit
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12650#issuecomment-213919607
@andrewor14 @zsxwing would you mind taking a look? Thanks! :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/12650
[SPARK-14884][SQL][Streaming][WebUI] Fix call site for continuous queries
## What changes were proposed in this pull request?
Since we've been processing continuous queries in separate
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12638#issuecomment-213870571
@marmbrus @tdas would you mind taking a look? Thanks! :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
GitHub user lw-lin reopened a pull request:
https://github.com/apache/spark/pull/12638
[SPARK-14874][SQL][Streaming] Remove the obsolete Batch representation
## What changes were proposed in this pull request?
The `Batch` class, which had been used to indicate progress
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/12638
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/12638
[SPARK-14874][SQL][Streaming] Cleanup the useless Batch class
## What changes were proposed in this pull request?
The `Batch` class, which had been used to indicate progress in a stream
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12450#issuecomment-212405238
@srowen thank you for the review & merging :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12521#issuecomment-212303861
@rxin @marmbrus would you mind taking a look when you have time? Thanks! :-)
And I'm not sure we should disallow calling methods like `parquet()`,
`text
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/12521
[SPARK-14747][SQL] Add assertStreaming/assertNoneStreaming checks in
DataFrameWriter
## Problem
If an end user happens to write code mixed with continuous-query-oriented
methods and non
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12489#issuecomment-211724261
@zsxwing would you mind taking a look? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/12489
[SPARK-14701][Streaming] First stop the event loop, then stop the
checkpoint writer in JobGenerator
## What changes were proposed in this pull request?
The stopping order of the `event
Github user lw-lin commented on the pull request:
https://github.com/apache/spark/pull/12450#issuecomment-211679362
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
401 - 500 of 597 matches
Mail list logo
