Github user HyukjinKwon commented on the issue:
https://github.com/apache/zeppelin/pull/3253
I'm not aware of release plan in Zeppelin since I'm just one of
contributors. For the current status, the Spark should be downgraded as far as
I can tell.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/zeppelin/pull/3253
Spark 2.4 support was added at
https://github.com/apache/zeppelin/pull/3206, which exactly addresses the issue
you faced. This will be available in new release of Zeppelin.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23226
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23268
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23270
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23262
Let's remove. No point of keeping unused method. The code will remain in
the commit anyway. Also, there's no quite good point of keeping few lines
method that's called only at one place
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23202
Similar discussion is going on at
https://github.com/apache/spark/pull/23201#discussion_r240156871. Let me keep
tracking them. Sorry for late response, @MaxGekk
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23201#discussion_r240156871
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala
---
@@ -121,7 +122,26 @@ private[sql] class
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23201#discussion_r240153595
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala
---
@@ -121,7 +122,26 @@ private[sql] class
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23262
LGTM otheriwse
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23262#discussion_r240151214
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala ---
@@ -17,51 +17,39 @@
package
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23262#discussion_r240141142
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala ---
@@ -17,51 +17,39 @@
package
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22683
Looks okay to me too. UI change requires screenshots of UI tho strictly.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23268
Let's fix PR description as well. You can leave the comments above resolved.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23249#discussion_r240123260
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala
---
@@ -262,6 +261,15 @@ case class
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23270
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23132
@MaxGekk, mind fixing PR description accordingly?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23268#discussion_r240110126
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala
---
@@ -53,19 +53,12 @@ private[hive] object HiveShim
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23268#discussion_r240107064
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala
---
@@ -53,19 +53,12 @@ private[hive] object HiveShim
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23268#discussion_r240105070
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala
---
@@ -53,19 +53,12 @@ private[hive] object HiveShim
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/18784
Looks inactive. @srowen and @felixcheung, do you know anyone who might be
interested in this?
---
-
To unsubscribe, e-mail
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23268#discussion_r240101927
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala
---
@@ -53,19 +53,12 @@ private[hive] object HiveShim
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23268
Yup, the test looks okay in that way. Let's file a JIRA and only leave the
test case.
---
-
To unsubscribe, e-mail: reviews
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23268
Let's close this one.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23268#discussion_r240097931
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala
---
@@ -53,19 +53,12 @@ private[hive] object HiveShim
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23268#discussion_r240097105
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala
---
@@ -53,19 +53,12 @@ private[hive] object HiveShim
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23266#discussion_r240074711
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/SupportsBatchRead.java
---
@@ -20,14 +20,27 @@
import
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23266#discussion_r240073831
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/SupportsBatchRead.java
---
@@ -20,14 +20,27 @@
import
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/16812
I still don't think need this since the workaround is easy. If other
committers find it worth, I won't object.
If there are no interests fro this PR afterwards, I would just close
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23263
> Visualizing a workflow is nice, but Spark's Pipelines are typically
pretty straightforward and linear. I could imagine producing a nicer
visualization than what you get from reading the Sp
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23256
Ooops,
> tbh, more meaningful dataset as example would be better...
did you expect to fix more instances here Felix? Sorry, I misr
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23256
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23263#discussion_r240004006
--- Diff: mllib/src/main/scala/org/apache/spark/ml/Estimator.scala ---
@@ -65,7 +65,19 @@ abstract class Estimator[M <: Model[M]] extends
PipelineSt
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23263#discussion_r240003885
--- Diff: mllib/src/test/scala/org/apache/spark/ml/MLEventsSuite.scala ---
@@ -0,0 +1,199 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23263#discussion_r240003869
--- Diff: mllib/src/main/scala/org/apache/spark/ml/Pipeline.scala ---
@@ -132,7 +132,8 @@ class Pipeline @Since("1.4.0") (
* @ret
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23263
The tests pass in my local. I'll fix them shortly.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23263
cc @srowen, @cloud-fan (since it mimics SQL's event listener), @jkbradley,
@mengxr and @yanboliang. Mind if I ask to take a look please? WDYT about
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23263#discussion_r23747
--- Diff: mllib/src/main/scala/org/apache/spark/ml/Predictor.scala ---
@@ -210,7 +214,7 @@ abstract class PredictionModel[FeaturesType, M
Github user HyukjinKwon closed the pull request at:
https://github.com/apache/spark/pull/23261
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/23263
[SPARK-23674][ML] Adds Spark ML Events
## What changes were proposed in this pull request?
This PR proposes to add ML events so that other developers can track and
add some actions
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23261
I'm going to rebase and reopen a PR to retrigger AppVeyor tests. The
failure looks unrelated
---
-
To unsubscribe, e-mail
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23261#discussion_r239998861
--- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala ---
@@ -163,7 +163,7 @@ abstract class MLWriter extends BaseReadWrite
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23261#discussion_r239998855
--- Diff: mllib/src/main/scala/org/apache/spark/ml/Predictor.scala ---
@@ -210,7 +214,7 @@ abstract class PredictionModel[FeaturesType, M
GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/23261
[SPARK-23674][ML] Adds Spark ML Events
## What changes were proposed in this pull request?
This PR proposes to add ML events so that other developers can track add
some actions
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23072#discussion_r239701916
--- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd ---
@@ -968,6 +970,17 @@ predicted <- predict(model, df)
head(predic
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23072#discussion_r239701364
--- Diff: R/pkg/tests/fulltests/test_mllib_clustering.R ---
@@ -319,4 +319,18 @@ test_that("spark.posterior and spark.perplexity", {
ex
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23072#discussion_r239701069
--- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd ---
@@ -968,6 +970,17 @@ predicted <- predict(model, df)
head(predic
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23072#discussion_r239700846
--- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd ---
@@ -968,6 +970,17 @@ predicted <- predict(model, df)
head(predic
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23226#discussion_r239662927
--- Diff:
core/src/test/java/org/apache/spark/unsafe/map/AbstractBytesToBytesMapSuite.java
---
@@ -622,6 +622,17 @@ public void
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23248
Thanks, @cloud-fan. I will take a look within tomorrow - don't block by me.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23248#discussion_r239441136
--- Diff: python/pyspark/sql/tests/test_udf.py ---
@@ -23,7 +23,7 @@
from pyspark import SparkContext
from pyspark.sql import
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23245
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23246
Ditto. Let's close these and focus on fixing actual bugs.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23247
@wangjiaochun, I think you better stop fixing trivial stuff in each PR.
Those stuff can be fixed when the codes around here is fixed, or let other
people fix it later
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23215
+1 from me to. It makes sense to me and having configuration sounds good.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23243
Backport from which JIRA @weixiuli? Usually the fix should go to master
first and it's backported to other branches when it's needed. If it should be
fixed in master branch as well, let's file
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23237
Looks some classes, for instance, `KafkaUtils` exposes this (I guess
mistakenly?). Let's don't bother this and close this PR
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23237
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
Thanks guys .. I will try to take a look for it as well (although it's
going to take relatively a long while). Let me get this in for now anyway
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23225
If a test passes without/with this changes, it wouldn't verify the
regression.
---
-
To unsubscribe, e-mail: reviews
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
Thanks for a close look, @BryanCutler. I just increased the timeout because
I was just trying to keep the test as is and fix it. The actual test doesn't
look taking so much time but it looks
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
test this please
--
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23120
a late LGTM as well
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23235#discussion_r239049825
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -35,6 +35,8 @@ displayTitle: Spark SQL Upgrading Guide
- Since Spark 3.0, CSV
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
cc @BryanCutler and @viirya
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
(cc @squito as well since it's from #23111)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23236
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/23236
[SPARK-26275][PYTHON][ML] Increases timeout for
StreamingLogisticRegressionWithSGDTests.test_training_and_prediction test
## What changes were proposed in this pull request?
Looks
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23227
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user HyukjinKwon reopened a pull request:
https://github.com/apache/zeppelin/pull/3253
[ZEPPELIN-3551] Upgrade Scala to 2.11.12
### What is this PR for?
This is just to update scala to 2.11.12 which to be consistent with spark
(SPARK-24418).
This PR takes over
Github user HyukjinKwon closed the pull request at:
https://github.com/apache/zeppelin/pull/3253
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23229
I don't think it's worth to change naming the variable in a single PR.
Let's do that when we fix some codes around here, or let other people try to
fix later
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23224
Can we file a JIRA? I think it's not minor.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23230
Oh, the original one was 3.0. Although this doc change can go to branch-2.4
alone as well, let me revert it in branch-2.4 for management simplicity
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23230
Merged to master and branch-2.4.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23227
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23226#discussion_r238977650
--- Diff:
core/src/test/java/org/apache/spark/unsafe/map/AbstractBytesToBytesMapSuite.java
---
@@ -622,6 +622,17 @@ public void
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23225
Also, it needs a JIRA. it's not minor one.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23225
How come existing tests cover if the empty file is created or not?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user HyukjinKwon commented on the issue:
https://github.com/apache/zeppelin/pull/3034
Hey Jeff, I can take over this one too if you're busy since I took a look
for similar code paths.
---
GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/zeppelin/pull/3253
[ZEPPELIN-3551] Upgrade Scala to 2.11.12
### What is this PR for?
This is just to update scala to 2.11.12 which to be consistent with spark
(SPARK-24418).
This PR takes over
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23203
Thank you @cloud-fan, @viirya, @srowen, and @BryanCutler.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23203
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23216#discussion_r238892679
--- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala
---
@@ -56,7 +56,7 @@ private[spark] class ResultTask[T, U
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23203#discussion_r238887812
--- Diff: python/run-tests.py ---
@@ -93,17 +93,18 @@ def run_individual_python_test(target_dir, test_name,
pyspark_python):
"py
Github user HyukjinKwon commented on the issue:
https://github.com/apache/zeppelin/pull/3033
Hey @zjffdu, busy? I can take this over - looks there's only minor conflict.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23203
Yea, will update it as well after this one gets merged.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23080
It's fixed in upcoming Spark. Spark 2.4 does not support it.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23196#discussion_r238496050
--- Diff:
sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala
---
@@ -49,8 +49,8 @@ class
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23196#discussion_r238495344
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -33,6 +33,8 @@ displayTitle: Spark SQL Upgrading Guide
- Spark applications which
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22590
Hm, let's try to find a way to expose other parse options. I think we
shouldn't allow every options available on Univocity
1 - 100 of 12680 matches
Mail list logo
