Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21109#discussion_r193743605
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1205,6 +1205,19 @@ object SQLConf {
.booleanConf
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21109#discussion_r193736438
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/joins/InnerJoinSuite.scala
---
@@ -70,27 +70,41 @@ class InnerJoinSuite extends
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21109#discussion_r193737191
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/InMemoryUnsafeRowQueue.scala
---
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21109#discussion_r193735681
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala
---
@@ -131,13 +135,100 @@ object ExtractEquiJoinKeys extends
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21109#discussion_r193733146
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala
---
@@ -131,13 +135,100 @@ object ExtractEquiJoinKeys extends
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21109#discussion_r193734550
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala
---
@@ -131,13 +135,100 @@ object ExtractEquiJoinKeys extends
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21109#discussion_r193735061
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala
---
@@ -131,13 +135,100 @@ object ExtractEquiJoinKeys extends
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21109#discussion_r193736960
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/joins/InnerJoinSuite.scala
---
@@ -117,101 +131,170 @@ class InnerJoinSuite extends
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21502#discussion_r193742604
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/joins/BroadcastJoinSuite.scala
---
@@ -152,6 +152,26 @@ class BroadcastJoinSuite extends
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/21477#discussion_r193740695
--- Diff: python/pyspark/sql/streaming.py ---
@@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None,
continuous=None):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13599
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13599
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91519/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13599
**[Test build #91519 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91519/testReport)**
for PR 13599 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21467
**[Test build #91521 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91521/testReport)**
for PR 21467 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21467
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21467
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91521/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21469
**[Test build #91523 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91523/testReport)**
for PR 21469 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18900
**[Test build #91522 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91522/testReport)**
for PR 18900 at commit
Github user debugger87 commented on a diff in the pull request:
https://github.com/apache/spark/pull/18900#discussion_r193730957
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
---
@@ -1019,6 +1021,8 @@ private[hive] object HiveClientImpl {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21467
**[Test build #91521 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91521/testReport)**
for PR 21467 at commit
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/21502#discussion_r193724774
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/joins/BroadcastJoinSuite.scala
---
@@ -152,6 +152,26 @@ class BroadcastJoinSuite
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21499
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21499
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3831/
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193696451
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,23 @@ object DateTimeUtils {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21499
**[Test build #91520 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91520/testReport)**
for PR 21499 at commit
Github user ssonker commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193694372
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,23 @@ object DateTimeUtils {
Github user DazhuangSu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19691#discussion_r193691275
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -510,40 +511,86 @@ case class
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21505
We would appreciate it if you put the performance before and after this PR?
It would be good to use `Benchmark` class.
---
-
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193688565
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,23 @@ object DateTimeUtils {
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193687670
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -114,20 +114,19 @@ object DateTimeUtils {
}
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193687346
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -114,20 +114,19 @@ object DateTimeUtils {
}
Github user ssonker commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193686772
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,24 @@ object DateTimeUtils {
Github user cxzl25 commented on a diff in the pull request:
https://github.com/apache/spark/pull/18900#discussion_r193685282
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
---
@@ -1019,6 +1021,8 @@ private[hive] object HiveClientImpl {
Github user ssonker commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193679978
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,24 @@ object DateTimeUtils {
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/20636
cc @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21061
Let me think about the implementation to keep the order.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21477#discussion_r193678459
--- Diff: python/pyspark/sql/streaming.py ---
@@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None,
continuous=None):
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193678440
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,24 @@ object DateTimeUtils {
Github user ssonker commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193676953
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,24 @@ object DateTimeUtils {
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193676413
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,24 @@ object DateTimeUtils {
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21499#discussion_r193675689
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/DecimalType.scala ---
@@ -161,13 +161,17 @@ object DecimalType extends AbstractDataType {
Github user ssonker commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193675674
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,24 @@ object DateTimeUtils {
Github user ssonker commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193675439
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,24 @@ object DateTimeUtils {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/13599#discussion_r193674158
--- Diff: python/pyspark/context.py ---
@@ -1035,6 +1044,41 @@ def getConf(self):
conf.setAll(self._conf.getAll())
return conf
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/13599#discussion_r193673500
--- Diff: docs/submitting-applications.md ---
@@ -218,6 +218,115 @@ These commands can be used with `pyspark`,
`spark-shell`, and `spark-submit` to
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/13599#discussion_r193674619
--- Diff: python/pyspark/context.py ---
@@ -1035,6 +1044,41 @@ def getConf(self):
conf.setAll(self._conf.getAll())
return conf
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/13599#discussion_r193672797
--- Diff: docs/submitting-applications.md ---
@@ -218,6 +218,115 @@ These commands can be used with `pyspark`,
`spark-shell`, and `spark-submit` to
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21505#discussion_r193674578
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -111,6 +113,24 @@ object DateTimeUtils {
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21505
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21505
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user ssonker opened a pull request:
https://github.com/apache/spark/pull/21505
[SPARK-24457][SQL] Improving performance of stringToTimestamp by cachâ¦
â¦ing Calendar instances for input timezones instead of creating new
everytime
## What changes were proposed in
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13599
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3830/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13599
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13599
**[Test build #91519 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91519/testReport)**
for PR 13599 at commit
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/13599#discussion_r193664416
--- Diff: docs/submitting-applications.md ---
@@ -218,6 +218,115 @@ These commands can be used with `pyspark`,
`spark-shell`, and `spark-submit` to
For
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/13599#discussion_r193659778
--- Diff: docs/submitting-applications.md ---
@@ -218,6 +218,115 @@ These commands can be used with `pyspark`,
`spark-shell`, and `spark-submit` to
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193649314
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -359,17 +368,42 @@ private[spark] class TaskSchedulerImpl(
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193648185
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1310,6 +1311,44 @@ class DAGScheduler(
}
}
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193647168
--- Diff:
core/src/main/scala/org/apache/spark/barrier/BarrierCoordinator.scala ---
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193658009
--- Diff:
core/src/main/scala/org/apache/spark/barrier/BarrierCoordinator.scala ---
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193640783
--- Diff:
core/src/main/scala/org/apache/spark/barrier/BarrierTaskContext.scala ---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193644506
--- Diff:
core/src/main/scala/org/apache/spark/barrier/BarrierCoordinator.scala ---
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/13599
@JoshRosen, I roughly heard that you took a look about this before. Do you
have a concern to address maybe?
---
-
To
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/13599
@holdenk and @zjffdu, I believe manual tests are a-okay if it's difficult
to write a test. We can manually test and expose this as an experimental
feature too.
BTW, I believe we can
Github user zjffdu commented on the issue:
https://github.com/apache/spark/pull/13599
Thanks for the interest on this PR and the info about `Pipfiles`. I think
we could support that after this PR get merged so that we can provide users
more options for virtualenv based on their
Github user kokes commented on the issue:
https://github.com/apache/spark/pull/13599
Hi, thanks for all the work on this! I see requirements.txt mentioned here
and there and, browsing this and other JIRAs, it seems to be the proposed way
to specify dependencies in PySpark. As you
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21482
**[Test build #91518 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91518/testReport)**
for PR 21482 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21482
**[Test build #91517 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91517/testReport)**
for PR 21482 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21482
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21482
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21482
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91518/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21482
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91517/
Test FAILed.
---
Github user kokes commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r193639554
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -154,6 +176,24 @@ private[spark] object Config
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/20929
yea, thanks for the comments! I'll try to fix based on the comments.
---
-
To unsubscribe, e-mail:
Github user galv commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193290266
--- Diff: python/pyspark/worker.py ---
@@ -232,6 +236,13 @@ def main(infile, outfile):
shuffle.DiskBytesSpilled = 0
Github user galv commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193269255
--- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala ---
@@ -627,6 +627,52 @@ class SparkContextSuite extends SparkFunSuite with
Github user galv commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193289530
--- Diff: python/pyspark/worker.py ---
@@ -232,6 +236,13 @@ def main(infile, outfile):
shuffle.DiskBytesSpilled = 0
Github user galv commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193291076
--- Diff: python/pyspark/worker.py ---
@@ -232,6 +236,13 @@ def main(infile, outfile):
shuffle.DiskBytesSpilled = 0
Github user galv commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193555968
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -123,6 +124,21 @@ private[spark] class TaskSetManager(
// TODO: We
Github user galv commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r193269297
--- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala ---
@@ -627,6 +627,52 @@ class SparkContextSuite extends SparkFunSuite with
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21500
Retaining versions of state is also relevant to do snapshotting the last
version in files: HDFSBackedStateStoreProvider doesn't snapshot if the version
doesn't exist in loadedMaps. So we may
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21501#discussion_r193635361
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/StopWordsRemover.scala ---
@@ -84,7 +86,36 @@ class StopWordsRemover @Since("1.5.0")
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21477#discussion_r193634436
--- Diff: python/pyspark/sql/streaming.py ---
@@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None,
continuous=None):
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21477#discussion_r193633540
--- Diff: python/pyspark/sql/streaming.py ---
@@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None,
continuous=None):
301 - 384 of 384 matches
Mail list logo
