GitHub user tdas opened a pull request:
https://github.com/apache/spark/pull/844
Updated scripts for auditing releases
- Added script to automatically generate change list CHANGES.txt
- Added test for verifying linking against maven distributions of
`spark-sql` and `spark-hive`
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/844#issuecomment-43720269
@pwendell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/844#issuecomment-43720431
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/844#issuecomment-43720416
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/844#issuecomment-43723792
LGTM - thanks TD this is great! Having SQL and Hive modules in there is
awesome.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/844#issuecomment-43726777
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15114/
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/844#issuecomment-43726776
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/842#issuecomment-43726780
Thanks. I've merged this into master branch-1.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/843
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/844#issuecomment-43726813
Jenkins, test this again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/841#issuecomment-43726915
He's on vacation this week so it might take a while for him to get back :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/839#issuecomment-43726958
Thanks. I've merged this into master branch-1.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/842
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/825#issuecomment-43727197
We can easily add right outer join support to the hash join though. In
general, the nested loop join performs very unfavorably compared with a hash
join implementation.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43727297
Jenkins, add to whitelist.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43727320
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43727343
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43727532
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43727534
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15115/
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43729433
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43729450
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43731406
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43731416
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43733545
`ensureFreeSpace` has 2 jobs. 1) iterate entries and select blocks to be
dropped. 2) if to-be-dropped blocks can free enough space, mark them as
dropping and return
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43734386
Build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43734389
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15117/
---
If your project is set up for it, you can
GitHub user nikhils05 opened a pull request:
https://github.com/apache/spark/pull/845
[SPARK-1820][tools] Make GenerateMimaIgnore @DeveloperApi annotation aware.
Solution for : Add all the classes with DeveloperApi annotation in Mima
excludes.
You can merge this pull request
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/845#issuecomment-43734997
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user mridulm commented on a diff in the pull request:
https://github.com/apache/spark/pull/791#discussion_r12888619
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -243,10 +250,13 @@ private class MemoryStore(blockManager: BlockManager,
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43736593
- With the latest commit, the issue with dropping flag is gone - which is
great.
- There is a change of behavior w.r.t earlier code.
Whether the earlier
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43737610
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43737611
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15116/
---
If your project is set up for it, you can
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43738827
@mridulm I checked the code of BlockManager#doPut.
val putBlockInfo = {
val tinfo = new BlockInfo(level, tellMaster)
// Do atomically !
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43739167
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user ueshin commented on the pull request:
https://github.com/apache/spark/pull/825#issuecomment-43741634
@rxin Ah, you mean that we should add right/full outer join support in
addition to #734?
I agree with the unfavorable performance of the nested loop join, so we
should
Github user ueshin commented on the pull request:
https://github.com/apache/spark/pull/825#issuecomment-43742049
@rxin BTW, speaking of performance, could you please review the code #836?
I think this is a kind of blocker issue of join strategy.
---
If your project is set up for
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43742247
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43742248
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15118/
---
If your project is set up for it, you can
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43744004
This seems really promising!! However, can you explain whether the
following sequence of events is possible or not in `ensureFreeSpace`?
Both thread 1 and thread 2
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43744225
@cloud-fan there are multiple calls to memoryStore to directly put a block
- not just from external addition.
So looking at only doPut might not help ?
---
If your
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43744304
@tdas there is a dropping flag which prevents this.
Or did I misunderstand your query ?
---
If your project is set up for it, you can reply to this email and have
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43744657
@tdas yes - thread 1 should set A's dropping to true; so thread 2 should
not select it
---
If your project is set up for it, you can reply to this email and have your
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43744797
Is that so? Since selection and marking are occurring into different
`entries.synchronized` blocks, selection and marking are not atomic together.
So two threads can select
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/844#issuecomment-43744964
Jenkins, retest this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43749375
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43749356
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43751967
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43751952
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43752165
@tdas you missed an important thing. `trToPut` call `ensureFreeSpace`
within the putLock, so one thread have to wait another thread done both
selecting and marking.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43753931
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15119/
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43753928
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43756438
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/772#issuecomment-43756441
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15120/
---
If your project is set up for it, you can
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43756945
@tdas as @cloud-fan stated, the code uses the implementation detail that
the private method is always called within context of a tryToPut lock - and not
called by anyone
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43775013
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43775032
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43778259
@tdas @mridulm what about we moving the `putLock.synchronized` into
`ensureFreeSpace ` and let `tryToPut` call `ensureFreeSpace ` directly? I think
it will be more
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43783643
@cloud-fan makes more sense.
Also, please rename it to something more appropriate (since it is not
longer trying to put within that block !)
@tdas, can you
Github user kanzhang commented on the pull request:
https://github.com/apache/spark/pull/841#issuecomment-43786603
@rxin thanks for the heads up. I appreciate help from anyone to help burn
down my open PRs, the oldest being over a month old.
---
If your project is set up for it, you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43788086
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15121/
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/837#issuecomment-43788084
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/828#issuecomment-43790944
@mateiz, @mengxr
I am using [the code](https://github.com/witgo/spark/compare/cachePoint) to
test ALS.
A brief description of the test:
| Item |
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-43795376
@cloud-fan @mridulm Aaah, I get it now. I knew I was missing something! I
agree with @mridulm that this is a tricky lock structure and needs to be
cleaner. Putting the
GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/846
[SPARK-1896] Respect spark.master before MASTER in REPL
The hierarchy for the shell is as follows:
```
MASTER --master spark.master (spark-defaults.conf)
```
This is inconsistent
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/846#issuecomment-43798409
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/846#issuecomment-43798432
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/847
[Typo] Stoped - Stopped
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark yarn-typo
Alternatively you can review and apply
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/697#issuecomment-43798939
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/697#issuecomment-43798959
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/847#issuecomment-43798932
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/847#issuecomment-43798958
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user ahirreddy commented on the pull request:
https://github.com/apache/spark/pull/697#issuecomment-43799163
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/697#issuecomment-43799234
Thanks. I will merge once Travis returns.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
GitHub user mengxr opened a pull request:
https://github.com/apache/spark/pull/848
[SPARK-1870] Make spark-submit --jars work in yarn-cluster mode.
Sent secondary jars to distributed cache of all containers and add the
cached jars to classpath before executors start.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/847#issuecomment-43799605
Thanks. I've merged this into master branch-1.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/848#issuecomment-43800111
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/841#discussion_r12916889
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDD.scala ---
@@ -274,6 +274,10 @@ class SchemaRDD(
seed: Long) =
new
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/719#issuecomment-43803035
@ankurdave is this good now?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/719#issuecomment-43803060
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/846#issuecomment-43803106
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/846#issuecomment-43803107
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15122/
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/719#issuecomment-43803413
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/719#issuecomment-43803425
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/847#issuecomment-43803493
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/847#issuecomment-43803495
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15123/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/697#issuecomment-43803492
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/719#issuecomment-43804545
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15126/
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/848#issuecomment-43804541
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15125/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/848#issuecomment-43804540
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/719#issuecomment-43804544
Build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/697
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user kanzhang commented on the pull request:
https://github.com/apache/spark/pull/697#issuecomment-43810385
@rxin @ahirreddy , thanks for the quick response!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user kanzhang commented on a diff in the pull request:
https://github.com/apache/spark/pull/841#discussion_r12921105
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDD.scala ---
@@ -274,6 +274,10 @@ class SchemaRDD(
seed: Long) =
new
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/848#discussion_r12921552
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala ---
@@ -479,37 +485,24 @@ object ClientBase {
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/848#discussion_r12921709
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala ---
@@ -479,37 +485,24 @@ object ClientBase {
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/848#issuecomment-43812877
Thanks. It looks great for me, and better than my patch.
cachedSecondaryJarLinks.foreach(addPwdClasspathEntry) is not needed since
we have
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/848#issuecomment-43814337
The symbolic links may not be under the PWD. That is why it didn't work
before.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/848#issuecomment-43814642
It works under driver before, so the major issue is those files are not in
executor's distributed cache. But I like the idea to add them explicitly so
we'll not miss
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/848#issuecomment-43815204
Yes, we can also control the ordering in this way.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/848#discussion_r12923791
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala ---
@@ -479,37 +485,24 @@ object ClientBase {
1 - 100 of 175 matches
Mail list logo
