Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15903
thanks @srowen . I've gone though all markdown files in `docs` module,
searched by regular expression `[^\n]\n\- `, found 55 occurrences and only this
one out of them requires the treatment. thanks
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15903
[Doc][Minor] Kafka doc: breakup into lines
## Before
![before](https://cloud.githubusercontent.com/assets/15843379/20340231/99b039fe-ac1b-11e6-9ba9-b44582427459.png)
## After
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15852
@uncleGen @tcondie thanks for working on this.
My major concern is this approach might disallow changing the
`compactInterval` once there were at least two compact files. Should we
disallow
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15828
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15828
rebased.
- the first & second commits were a simple refactor of the `
CompactibleFileStreamLogSuite.scala`
- **the third commit was the actual change of this patch**
---
If
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15870
Hint for reviewers:
- the first commit simply moved some code into its own file; other than
that nothing was touched;
- the second commit did some refactor
so review only the second
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15870
[SPARK-18425][Structured Streaming][Tests] Test `CompactibleFileStreamLog`
directly
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15828
[WIP][SS] CompactibleFileStreamLog should not rely on "compactInterval" to
detect a compaction batch
## What changes were proposed in this pull request?
Right now CompactibleFil
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15480
@hvanhovell it'd be great if you can take a look at this, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15786#discussion_r86902077
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/MemorySinkSuite.scala ---
@@ -187,6 +187,22 @@ class MemorySinkSuite extends StreamTest
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15786#discussion_r86901995
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -212,4 +212,8 @@ class MemorySink(val schema: StructType
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15786
@zsxwing could you also take a look
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15786
@brkyvz would you take another look
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15480
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15786
[SPARK-18261][SS] Add statistics to MemorySink for joining
## What changes were proposed in this pull request?
Right now, there is no way to join the output of a memory sink with any
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15480
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15480#discussion_r86494742
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
---
@@ -537,7 +537,6 @@ class CodegenContext
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15681
I'm closing this for now; can re-open if other people should demand this.
thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/15681
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15715
@srowen thanks -- could we also merge this into branch-2.1?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15715
Sounds good!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15715
Good question!
Actually I had tried keeping the indent of the div tags but un-indenting
only the code before I open this pr -- and it looked like this:
![snip20161102_8](https
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15715
![snip20161101_7](https://cloud.githubusercontent.com/assets/15843379/19896158/469d72b0-a08e-11e6-972f-5706ab4aa4c4.png)
Please see the screenshot above -- this is `flume doc` -- I got
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15715
@srowen thanks.
I'm afraid the streaming `flume integration doc`, `kinesis integration
doc`, as well as `kafka08 integration doc` also need code highlights, but this
`{% hightlight
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15715
@koeninger @srowen it'd be great if you could take a look at this too :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15715
Fix the leading spaces is needed, because without the fixing the code
snippet would contain leading spaces as well(see the pic below), which is quite
inconsistent with the other programming guides
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15715
[SPARK-18198][Doc][Streaming] Highlight code snippets
## What changes were proposed in this pull request?
We should use `{% highlight lang %}``{% endhighlight %}` to highlight code
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15681
Deprecating the existing one would mean we still need to introduce some
`createRDDInternal`, and let the deprecated one call it; then we can just
remove the deprecated one some time in the future
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15681
Thank you @koeninger !
Please let me cc @srowen who's been around to also take a look~
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15712
@rxin @ericl please take a look thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15712
[SPARK-18103][SQL] Rename `MetadataLogFileCatalog` to `MetadataLogFileIndex`
## What changes were proposed in this pull request?
This is a follow-up to https://github.com/apache/spark/pull
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15681
Reverted all changes and added another createRDD overload ! :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13575
No problem here. Spark-17924 is super great (I've been watching it all the
time) and @rxin thank you for bringing that up! :-D
---
If your project is set up for it, you can reply to this email
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13575
No problem here. Spark-17924 is super great (I've been watching it all the
time) and thank you for bringing that up!
---
If your project is set up for it, you can reply to this email and have your
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15681
@koeninger well received and thanks! JIRA opened and unrelated changes
removed.
Let's see what Jenkins would say.
---
If your project is set up for it, you can reply to this email and have
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15681
@koeninger could you also take a look at this, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15679
@srowen @koeninger thanks for the review!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15679#discussion_r85642639
--- Diff: docs/streaming-kafka-0-10-integration.md ---
@@ -141,9 +214,11 @@ For data stores that support transactions, saving
offsets in the same transactio
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15679#discussion_r85642607
--- Diff: docs/streaming-kafka-0-10-integration.md ---
@@ -165,6 +240,36 @@ For data stores that support transactions, saving
offsets in the same transactio
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15679#discussion_r85641731
--- Diff: docs/streaming-kafka-0-10-integration.md ---
@@ -120,15 +184,24 @@ Kafka has an offset commit API that stores offsets in
a special Kafka topic
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15679#discussion_r85641730
--- Diff: docs/streaming-kafka-0-10-integration.md ---
@@ -120,15 +184,24 @@ Kafka has an offset commit API that stores offsets in
a special Kafka topic
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15679
@koeninger thanks. These were extracted from my local compiled example
projects -- no compiled kafka010 examples yet in the Spark repo, right?
---
If your project is set up for it, you can reply
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15681#discussion_r85640495
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaUtils.scala
---
@@ -56,32 +57,18 @@ object KafkaUtils extends Logging
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15681#discussion_r85640467
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaUtils.scala
---
@@ -56,32 +57,18 @@ object KafkaUtils extends Logging
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15681#discussion_r85640450
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaUtils.scala
---
@@ -56,32 +57,18 @@ object KafkaUtils extends Logging
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15681#discussion_r85640442
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala
---
@@ -44,11 +44,6 @@ import
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15681
[Minor][Streaming][Kafka] Kafka010 .createRDD() scala API should expect
scala Map
## What changes were proposed in this pull request?
Thoughout `external/kafka-010`, Java APIs
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15679#discussion_r85640168
--- Diff: docs/streaming-kafka-0-10-integration.md ---
@@ -165,6 +240,36 @@ For data stores that support transactions, saving
offsets in the same transactio
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15679#discussion_r85640157
--- Diff: docs/streaming-kafka-0-10-integration.md ---
@@ -103,6 +153,20 @@ Note that you cannot use `PreferBrokers`, because
without the stream
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15679#discussion_r85640159
--- Diff: docs/streaming-kafka-0-10-integration.md ---
@@ -165,6 +240,36 @@ For data stores that support transactions, saving
offsets in the same transactio
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15679#discussion_r85640154
--- Diff: docs/streaming-kafka-0-10-integration.md ---
@@ -85,6 +121,20 @@ If you have a use case that is better suited to batch
processing, you can create
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15679#discussion_r85640143
--- Diff: docs/streaming-kafka-0-10-integration.md ---
@@ -44,6 +44,42 @@ For Scala/Java applications using SBT/Maven project
definitions, link your strea
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15679
Oh there's a on-going 2.0.2 RC cut. It'd be great if we can make this into
2.0.2. So also cc @srowen to take a look, thanks!
---
If your project is set up for it, you can reply to this email
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15679
@koeninger @zsxwing it'd be great if you can take a look, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15679
[SPARK-16312][Follow-up][STREAMING][KAFKA][DOC] Add java code snippet for
Kafka 0.10 integration doc
## What changes were proposed in this pull request?
added java code snippet for Kafka
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15588
Spark Streaming would do a very simple dummy job ensure that all slaves
have registered before scheduling the `Receiver`s; please see
https://github.com/apache/spark/blob/v2.0.0/streaming/src/main
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13705
closing this in favor of SPARK-17924
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/13705
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/13507
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/13507
I'm closing this in favor of SPARK-17924, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15480#discussion_r83990016
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala
---
@@ -118,7 +118,45 @@ object
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15519
This looks good to me, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15497
Please go ahead and take over -- let's fix this before 2.0.2, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/15497
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15497
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15497#discussion_r83584538
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryListenerSuite.scala
---
@@ -81,7 +81,7 @@ class StreamingQueryListenerSuite
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15497#discussion_r83584420
--- Diff: core/src/main/scala/org/apache/spark/util/ManualClock.scala ---
@@ -27,6 +27,7 @@ package org.apache.spark.util
private[spark] class
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15480#discussion_r83574850
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala
---
@@ -118,7 +118,45 @@ object
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15480#discussion_r83574830
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala
---
@@ -118,7 +118,45 @@ object
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15480#discussion_r83574113
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala
---
@@ -118,7 +118,45 @@ object
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15497#discussion_r83570900
--- Diff: core/src/main/scala/org/apache/spark/util/ManualClock.scala ---
@@ -27,6 +27,7 @@ package org.apache.spark.util
private[spark] class
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15497#discussion_r83570798
--- Diff: core/src/main/scala/org/apache/spark/util/ManualClock.scala ---
@@ -27,6 +27,7 @@ package org.apache.spark.util
private[spark] class
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15497
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15492
Could you take a look at my PR https://github.com/apache/spark/pull/15497 -
sorry I didn't noticed you've already started testing this.
---
If your project is set up for it, you can reply
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15497
@tdas @zsxwing could you take a look
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15497
[Test][SPARK-16002][Follow-up] Fix flaky test in StreamingQueryListenerSuite
## What changes were proposed in this pull request?
editing
## How was this patch tested
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15480
@davies @kiszk it'd be great if you can take a look
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15480
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15480
Flaky test I think not related to this PR. Thanks, @HyukjinKwon !
Jenkins retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/15482
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15482
Now that td has issued a commit
https://github.com/apache/spark/commit/05800b4b4e7873ebc445dfcd020b76d7539686e1
exactly like this, I'm closing this. Sure `flaky-test` is more proper than
`hot-fix
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15482
[Hot fix] ignore("single listener, check trigger statuses") for now
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lw-lin/spa
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15480
[SPARK-16845][SQL] `GeneratedClass$SpecificOrdering` grows beyond 64 KB
## What changes were proposed in this pull request?
Prior to this patch, we'll generate `compare
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/15461
Closing for now; will re-open soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/15461
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/15461
[SPARK-16845][SQL] `GeneratedClass$SpecificOrdering` grows beyond 64 KB
## What changes were proposed in this pull request?
Prior to this patch, we'll generate `compare
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/15195#discussion_r80654592
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
---
@@ -290,8 +284,8 @@ final class DataStreamWriter[T] private[sql
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14118#discussion_r79088930
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVInferSchema.scala
---
@@ -232,66 +232,62 @@ private[csv] object
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14118#discussion_r79088096
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -329,7 +329,8 @@ def csv(self, path, schema=None, sep=None,
encoding=None, quote=None, escape=Non
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14118#discussion_r79088141
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVInferSchema.scala
---
@@ -232,66 +232,62 @@ private[csv] object
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/14118
@HyukjinKwon thanks for the information!
@srowen yea I still think this is good to go.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/14118
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14914#discussion_r77131280
--- Diff:
mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala ---
@@ -999,7 +999,7 @@ object Matrices {
val data = new
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14914#discussion_r77130880
--- Diff:
mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala ---
@@ -999,7 +999,7 @@ object Matrices {
val data = new
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14914#discussion_r77128029
--- Diff:
mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala ---
@@ -999,7 +999,7 @@ object Matrices {
val data = new
GitHub user lw-lin opened a pull request:
https://github.com/apache/spark/pull/14914
[SPARK-17359][SQL][MLLib] Use ArrayBuffer.+=(A) instead of
ArrayBuffer.append(A) in performance critical paths
## What changes were proposed in this pull request?
We should generally use
Github user lw-lin closed the pull request at:
https://github.com/apache/spark/pull/14298
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/14298
@clockfly
Your PR's improvements overall look reasonable to me, so I'm closing this
one. Good job!
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/14118
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/14118
> What if I am writing explicitly an empty string out? Does it become just
1,,2?
Yes. It becomes `1,,2` in 2.0, and the same `1,,2` with this patch -- no
behavior changes.
&g
201 - 300 of 597 matches
Mail list logo