GitHub user davies opened a pull request:
https://github.com/apache/spark/pull/11659
[SPARK-13830] prefer block manager than direct result for large result
## What changes were proposed in this pull request?
The current RPC can't handle large blocks very well, it's very slow
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/11088#issuecomment-195522100
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this featur
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11646#issuecomment-195522580
**[Test build #52939 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52939/consoleFull)**
for PR 11646 at commit
[`65596df`](https://gi
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11659#issuecomment-195522563
**[Test build #52938 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52938/consoleFull)**
for PR 11659 at commit
[`e982656`](https://gi
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11088#issuecomment-195522607
**[Test build #52940 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52940/consoleFull)**
for PR 11088 at commit
[`2460716`](https://gi
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11613#issuecomment-195524921
**[Test build #52941 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52941/consoleFull)**
for PR 11613 at commit
[`ebb25f9`](https://gi
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/11646#discussion_r55881083
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -454,19 +454,60 @@ trait FileFormat {
requiredColumns: Arr
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11646#issuecomment-195530198
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11646#issuecomment-195530204
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11646#issuecomment-195530072
**[Test build #52939 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52939/consoleFull)**
for PR 11646 at commit
[`65596df`](https://g
Github user jaceklaskowski commented on a diff in the pull request:
https://github.com/apache/spark/pull/11615#discussion_r55883199
--- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala ---
@@ -270,9 +270,25 @@ private[spark] object JettyUtils extends Logging {
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/11613#issuecomment-195534728
@andrewor14 @nongli, this should now be ready for review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as we
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11613#issuecomment-195536492
**[Test build #52942 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52942/consoleFull)**
for PR 11613 at commit
[`dbca8cf`](https://gi
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/11591#issuecomment-195539011
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11591#issuecomment-195541177
**[Test build #52943 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52943/consoleFull)**
for PR 11591 at commit
[`79e1bf5`](https://gi
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11452#issuecomment-195542456
**[Test build #52933 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52933/consoleFull)**
for PR 11452 at commit
[`fda639b`](https://g
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11452#issuecomment-195543244
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11452#issuecomment-195543238
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11498#issuecomment-195543538
**[Test build #52944 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52944/consoleFull)**
for PR 11498 at commit
[`8b8f4cb`](https://gi
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/11649#issuecomment-195544498
I guess my question is why we are doing this for filters at all. It makes
sense for joins, but perhaps we should just eliminate that part of the rule.
---
If your pr
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11498#issuecomment-195544738
**[Test build #52945 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52945/consoleFull)**
for PR 11498 at commit
[`c04d3d7`](https://gi
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11646#issuecomment-195547936
**[Test build #52946 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52946/consoleFull)**
for PR 11646 at commit
[`35be8d5`](https://gi
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11498#issuecomment-195549353
**[Test build #52947 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52947/consoleFull)**
for PR 11498 at commit
[`9fdde9f`](https://gi
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/11644#issuecomment-195549744
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
ena
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11615#discussion_r55888190
--- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala ---
@@ -270,9 +270,25 @@ private[spark] object JettyUtils extends Logging {
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/11615#discussion_r55888314
--- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala ---
@@ -270,9 +270,25 @@ private[spark] object JettyUtils extends Logging {
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/11573#discussion_r55888485
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -64,10 +83,86 @@ private[sql] class SparkQl(conf: ParserConf =
SimplePar
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11644#issuecomment-195551694
**[Test build #52948 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52948/consoleFull)**
for PR 11644 at commit
[`8177e69`](https://gi
Github user BenFradet commented on the pull request:
https://github.com/apache/spark/pull/10355#issuecomment-195557577
@MLnick you were right, thanks for spotting that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If y
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/11573#discussion_r55889875
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -64,10 +83,86 @@ private[sql] class SparkQl(conf: ParserConf =
SimplePar
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11659#discussion_r55889878
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -97,9 +97,9 @@ private[spark] class Executor(
// Set the classloader f
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11615#issuecomment-195560080
**[Test build #52949 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52949/consoleFull)**
for PR 11615 at commit
[`e772aed`](https://gi
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11612#discussion_r55891228
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -265,25 +265,52 @@ private[yarn] class YarnAllocator(
// For
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11544#issuecomment-195563762
**[Test build #52934 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52934/consoleFull)**
for PR 11544 at commit
[`228e078`](https://g
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11544#issuecomment-195564066
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/11510#issuecomment-195564195
I assume no one has any problems with this, so I'll merge when tests pass
again. retest this please
---
If your project is set up for it, you can reply to this email an
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11544#issuecomment-195564067
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/11649#issuecomment-195564292
@marmbrus
Are you saying we should not add null filters for `Filter`?
https://github.com/apache/spark/pull/11372 The original PR shows it can benefit
some
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/11570#issuecomment-195564408
Ping anyone?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11088#issuecomment-195564366
**[Test build #52940 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52940/consoleFull)**
for PR 11088 at commit
[`2460716`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11659#issuecomment-195564685
**[Test build #52950 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52950/consoleFull)**
for PR 11659 at commit
[`142b43e`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11088#issuecomment-195564715
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11510#issuecomment-195564730
**[Test build #52951 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52951/consoleFull)**
for PR 11510 at commit
[`d7f1ffa`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11088#issuecomment-195564713
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/11573#discussion_r55892525
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -29,7 +29,26 @@ private[sql] class SparkQl(conf: ParserConf =
SimplePars
Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/11208#issuecomment-195565806
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/11208#issuecomment-195565781
cc @yhuai @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have t
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/11573#discussion_r55892958
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/AlterTableCommandParser.scala
---
@@ -0,0 +1,428 @@
+/*
+ * Licensed to the A
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/11573#discussion_r55893285
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -29,7 +29,26 @@ private[sql] class SparkQl(conf: ParserConf =
SimplePars
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11208#issuecomment-195567000
**[Test build #52952 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52952/consoleFull)**
for PR 11208 at commit
[`e060dea`](https://gi
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/11651#issuecomment-195567314
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled a
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/11651#issuecomment-195567401
Merging this into master, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does no
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11208#issuecomment-195569065
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11208#issuecomment-195569069
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11208#issuecomment-195569043
**[Test build #52952 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52952/consoleFull)**
for PR 11208 at commit
[`e060dea`](https://g
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/11573#issuecomment-195570543
I have left a few comments. It is a good starting point. Thank you for
working on this!
---
If your project is set up for it, you can reply to this email and have your
r
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11613#issuecomment-195570778
**[Test build #52935 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52935/consoleFull)**
for PR 11613 at commit
[`858aea2`](https://g
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11613#issuecomment-195571056
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11613#issuecomment-195571052
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55895725
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -520,3 +447,42 @@ private[spark] class MemoryStore(
logMem
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55896680
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -793,7 +798,12 @@ private[spark] class BlockManager(
// We w
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55896316
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -520,3 +447,42 @@ private[spark] class MemoryStore(
logMem
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55897125
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11659#issuecomment-195574451
**[Test build #52938 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52938/consoleFull)**
for PR 11659 at commit
[`e982656`](https://g
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55896526
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -717,8 +717,13 @@ private[spark] class BlockManager(
level: St
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11659#issuecomment-195575304
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55897629
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/11660
[SPARK-X] Guard against race condition when re-caching disk blocks in
memory
When reading data from the DiskStore and attempting to cache it back into
the memory store, we should guard again
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55897743
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -520,3 +447,42 @@ private[spark] class MemoryStore(
logMemo
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55897722
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -520,3 +447,42 @@ private[spark] class MemoryStore(
logMemo
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11659#issuecomment-195575309
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11660#issuecomment-195576832
**[Test build #52953 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52953/consoleFull)**
for PR 11660 at commit
[`5342712`](https://gi
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55897952
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user tnachen commented on the pull request:
https://github.com/apache/spark/pull/11272#issuecomment-195576917
@andrewor14 I'll try to create a test to verify this, when is the 2.0
closing date?
---
If your project is set up for it, you can reply to this email and have your
rep
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55898084
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55898188
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55898378
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/11649#issuecomment-195579496
In this commit, three parts are done:
- removed the `IsNotNull` constraints for compound expressions
- added `IsNotNull` constraints support for all the `Binar
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11613#issuecomment-195581011
**[Test build #52941 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52941/consoleFull)**
for PR 11613 at commit
[`ebb25f9`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11649#issuecomment-195580608
**[Test build #52954 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52954/consoleFull)**
for PR 11649 at commit
[`baa2cda`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11613#issuecomment-195582626
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55898638
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11613#issuecomment-195582619
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55898608
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55898706
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/11660#issuecomment-195583417
/cc @andrewor14. In addition to fixing that race-condition, this patch also
addresses that duplicated code that you pointed out in a review of my previous
PR.
---
I
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11607#discussion_r55898782
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ---
@@ -346,14 +376,18 @@ private[client] class Shim_v0_13 extends Shim_v0_1
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/11607#issuecomment-195583935
This makes sense to me if it helps with the original issue; did you get a
chance to try that out?
---
If your project is set up for it, you can reply to this email and
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55898874
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55898848
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55898978
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
GitHub user dongjoon-hyun opened a pull request:
https://github.com/apache/spark/pull/11661
[SPARK-13834] Update sbt for 2.x.
## What changes were proposed in this pull request?
For 2.0.0, we had better bump `sbt`, too. SBT 0.13.11 fixes wrong warnings
and improve increment
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55899254
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user sameeragarwal commented on the pull request:
https://github.com/apache/spark/pull/11649#issuecomment-195585134
@marmbrus by adding additional `isNotNull` checks, we are hoping to
generate code that can quickly short circuit the filter condition checks in the
generated code
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11661#issuecomment-195586138
**[Test build #52955 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52955/consoleFull)**
for PR 11661 at commit
[`e3e62ba`](https://gi
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55899657
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/11649#issuecomment-195588783
I see. We should probably comment on that in `NullFiltering` if that is
the intention. Anytime there are implicit contracts between components like
this it makes thi
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/11613#discussion_r55900156
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -100,48 +92,136 @@ private[spark] class MemoryStore(
*/
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/11649#issuecomment-195589358
Also, does codegen really need us to add redundant expressions into the
query plan? Can't it just look at the constraints and pick those that are
useful for it?
---
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/11651
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
201 - 300 of 606 matches
Mail list logo
