Github user NirmalReddy commented on the pull request:
https://github.com/apache/incubator-spark/pull/475#issuecomment-35234000
@pwendell Can you please verify this patch. Thanks !!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user NirmalReddy commented on the pull request:
https://github.com/apache/incubator-spark/pull/403#issuecomment-35233963
@pwendell Can you please verify this patch. Thanks !!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/585#issuecomment-35233368
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12739/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/585#issuecomment-35233367
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/585#issuecomment-35232312
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/585#issuecomment-35232311
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/402#issuecomment-35230376
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/402#issuecomment-35230379
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12738/
---
If you
Github user ScrapCodes commented on the pull request:
https://github.com/apache/incubator-spark/pull/585#issuecomment-35230219
Hey Patrick,
Since they don not have wildcards for ignored classes, we would have to
list down all reported errors explicitly
[link](https://github.c
Github user punya commented on the pull request:
https://github.com/apache/incubator-spark/pull/605#issuecomment-35230127
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-pos
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/402#issuecomment-35229620
One or more automated tests failed
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12737/
---
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/402#issuecomment-35229619
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/402#issuecomment-35229464
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/402#issuecomment-35229465
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user nicklan commented on the pull request:
https://github.com/apache/incubator-spark/pull/603#issuecomment-35229452
@mateiz I agree it would be good to have the tachyon scripts inside the
jar, but afaik there isn't currently a JAR available with them anywhere, so I
just copied
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/402#issuecomment-35229332
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/402#issuecomment-35229331
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user nicklan commented on the pull request:
https://github.com/apache/incubator-spark/pull/603#discussion_r9782432
Not sure on this one, but spark shouldn't end up having this on the
classpath anyway right, since it's buried in the sbin/tachyon folder
---
If your project is se
Github user punya commented on the pull request:
https://github.com/apache/incubator-spark/pull/605#issuecomment-35228237
I'll give it a shot.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/605#issuecomment-35228095
Moving these to src/main/java is a good idea, but I wonder if most of these
files would be better refactored into Scala. This should be trivial for all
except J
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#issuecomment-35227969
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12736/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#issuecomment-35227968
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#issuecomment-35227821
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#issuecomment-35227822
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12735/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/605#issuecomment-35227425
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To
GitHub user punya opened a pull request:
https://github.com/apache/incubator-spark/pull/605
Move all Java code to src/main/java
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-spark move-java-sources
Alternative
Github user jyotiska commented on the pull request:
https://github.com/apache/incubator-spark/pull/581#issuecomment-35227176
@JoshRosen any updates on this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#issuecomment-35227135
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#issuecomment-35227134
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#issuecomment-35226957
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#discussion_r9781736
Went ahead and made these private[spark]. It seems unlikely that anyone
else would use these methods, and making them private means that the type
parameter reorde
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#issuecomment-35226956
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#issuecomment-35226986
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/600#issuecomment-35226744
No need for your renaming PR @punya -- I've already taken care of it in
#604 :) Thanks!
---
If your project is set up for it, you can reply to this email and h
Github user punya commented on the pull request:
https://github.com/apache/incubator-spark/pull/600#issuecomment-35226354
Thanks @aarondav! I wondered about the dishonest `ClassTag`s too but
convinced myself that in this case it was totally harmless because
`subtractByKey` ignores the
Github user punya closed the pull request at:
https://github.com/apache/incubator-spark/pull/600
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/597#issuecomment-35226013
What is the use-case you have in mind here? Just some sort of final status
of all executors right before terminating a job/shell?
If you're just interes
Github user CodingCat commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#issuecomment-35225862
@aarondav thank you for the comments, another round of fix
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitH
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/600#issuecomment-35225376
This PR doesn't need to block on this discussion, since it doesn't actually
rely on the fake ClassTag, so I've merged it into master.
I've created
[JIR
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9781258
update or remove comment at the top of this file that talks about the
options -- removal is fine since we have the list in code now
---
If your project is set up
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/572#issuecomment-35225143
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12734/
---
If you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/572#issuecomment-35225140
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9781246
pattern should start with ^ and end with $ -- just tried with something
like "4gz" and it passed
---
If your project is set up for it, you can reply to this emai
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9781248
change OPTIONS to SPARK_SHELL_OPTS!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, plea
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9781252
nit: maybe "the maximum number of cores to be used by the spark shell"
---
If your project is set up for it, you can reply to this email and have your
reply appea
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9781247
update for -em
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your resp
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/572#issuecomment-35224269
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/572#issuecomment-35224270
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user holdenk commented on the pull request:
https://github.com/apache/incubator-spark/pull/572#discussion_r9781015
Removed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/603#issuecomment-35223558
@aarondav we don't, Tachyon actually compiles them at runtime it seems, but
you can compile them when you publish the Tachyon JAR to avoid that.
---
If your proj
Github user holdenk commented on the pull request:
https://github.com/apache/incubator-spark/pull/572#discussion_r9780813
SharedSparkContext isn't available inside of mllutils tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user bijaybisht closed the pull request at:
https://github.com/apache/incubator-spark/pull/568
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this fe
Github user CodingCat commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9780400
fixed the above two
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post you
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/536#discussion_r9780371
I don't think a semicolon is gramatically correct here. In fact, I think
"The `updateStateByKey` operation allows you to maintain some state data
and continuou
Github user holdenk commented on the pull request:
https://github.com/apache/incubator-spark/pull/572#discussion_r9780345
Changed in both places.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post y
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/536#discussion_r9780346
Not sure if this is the section you were talking about:
http://kafka.apache.org/documentation.html#kafkahadoopconsumerapi
---
If your project is set up for it, y
Github user holdenk commented on the pull request:
https://github.com/apache/incubator-spark/pull/572#discussion_r9780341
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If yo
Github user holdenk commented on the pull request:
https://github.com/apache/incubator-spark/pull/572#discussion_r9780337
oops, fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response
Github user holdenk commented on the pull request:
https://github.com/apache/incubator-spark/pull/572#discussion_r9780330
Done :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If
Github user CodingCat commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9780299
I thought that "Created spark context" is something signalling a
significant step in starting spark-shell, we'd better write it to the log file
to facilitate deb
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#issuecomment-35221936
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#issuecomment-35221937
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12733/
---
If you
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/568#issuecomment-35221890
Merged in master and branch-0.9. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do s
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/573#issuecomment-35221813
I'm not 100% caught up on the state of this issue. Is #570 a "complete fix"
for this issue, or is this PR still the best fix we have in the pipeline? Is it
in a
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9780214
Most of the rest of the code in this file uses echo, so I would avoid
changing this unless you have a good reason.
---
If your project is set up for it, you can
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9780203
A slightly confusing point is that the driver is in the same memory space
as the shell, and we're really just controlling the memory of the shell process
itself.
Github user CodingCat commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9780202
still ugly...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your resp
Github user CodingCat commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9780199
enmaybe --execmem?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/599#discussion_r9780175
It is perhaps unfortunate that SPARK_WORKER_MEMORY actually does exist, and
controls the total amount of memory that a worker can lease across all
executors on a
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/603#issuecomment-35221556
Sorry if this is a stupid question, but why do we need Tachyon JSPs? Are we
going to host Tachyon pages from our own UI?
---
If your project is set up for it,
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/603#discussion_r9780128
I don't know much about log4j, but could this accidentally override Spark's
own logging properties if it is the first log4j.properties file found on the
classpath
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#discussion_r9780111
These changes may actually be problematic, as they are part of a publicly
accessible API. Even removing the specialization of V is not really
backwards-compatible
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#discussion_r9780099
ClassTags do not store generic information, so here we are still just
finding Tuple2.
---
If your project is set up for it, you can reply to this email and have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#issuecomment-35221290
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#issuecomment-35221292
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/604#discussion_r9780093
Note: I removed the ClassTag for V, as it was not necessary. Also, I
reordered the type parameters to put K in front.
---
If your project is set up for it, you c
GitHub user aarondav opened a pull request:
https://github.com/apache/incubator-spark/pull/604
SPARK-1098: Minor cleanup of ClassTag usage in Java API
Our usage of fake ClassTags in this manner is probably not healthy, but I'm
not sure if there's a better solution available, so I ju
Github user martinjaggi commented on the pull request:
https://github.com/apache/incubator-spark/pull/575#issuecomment-35220431
@dlwh actually i think it's the same story in structured prediction (SGD or
BCFW), immediate updates on the vector are usually faster for the local
machine.
Github user dlwh commented on the pull request:
https://github.com/apache/incubator-spark/pull/575#issuecomment-35220185
@martinjaggi I've often found that minibatching makes things converge much
more quickly, since you get a nice variance reduction in the estimate of
the gradie
Github user martinjaggi commented on the pull request:
https://github.com/apache/incubator-spark/pull/575#issuecomment-35219684
@dlwh Thanks! This is of course a nice idea. Perhaps surprisingly (and good
for us) such tricks seem not even necessary in the current state of the art
algor
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/575#discussion_r9779496
These factory methods can probably just be called `dense`, `sparse`, etc.
---
If your project is set up for it, you can reply to this email and have your
reply appe
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35218897
BTW a usage example for my test project is
https://github.com/mateiz/java8-test/blob/master/src/main/java/test/Main.java.
This is what I'd like our code to look l
Github user dlwh commented on the pull request:
https://github.com/apache/incubator-spark/pull/575#issuecomment-35218872
@martinjaggi For how it's usually implemented, that's right. But you can
quite likely get better performance doing minibatches with dense vector/CSC
multiply
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/539#issuecomment-35218879
BTW, I've looked into this myself, and created a short project at
https://github.com/mateiz/java8-test to show how an RDD-like API might work in
Java. To make a l
Github user martinjaggi commented on the pull request:
https://github.com/apache/incubator-spark/pull/575#issuecomment-35218573
@fommil No matrix operations are performed at all so far, only vector
addition (of type dense += sparse). See the code in this PR by @mengxr . Vector
operati
Github user fommil commented on the pull request:
https://github.com/apache/incubator-spark/pull/575#issuecomment-35218098
@martinjaggi I'm happy to advise on what the best sparse format would be
for any particular problem that you're wanting to solve in spark. just let me
know the ma
Github user martinjaggi commented on the pull request:
https://github.com/apache/incubator-spark/pull/575#issuecomment-35217718
Hope you don't get me wrong, I was not at all proposing to fix a single
scheme, neither for serialization, or for the choice of sparse library. I was
just su
Github user fommil commented on the pull request:
https://github.com/apache/incubator-spark/pull/575#issuecomment-35216981
@martinjaggi I believe you would be making a massive mistake by agreeing on
a single serialisation scheme for sparse vectors, unless that format is
independent of
Github user CodingCat closed the pull request at:
https://github.com/apache/incubator-spark/pull/602
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature
Github user martinjaggi commented on the pull request:
https://github.com/apache/incubator-spark/pull/575#issuecomment-35212055
Really looking forward to having sparse vectors in MLlib soon, this is
super important! And thanks for your efforts so far!
Just a quick comment abou
Github user kellrott commented on the pull request:
https://github.com/apache/incubator-spark/pull/180#issuecomment-35211241
Are there any other remaining issues that are preventing this pull request
from being reviewed/merged?
If your project is set up for it, you can reply to this
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/602#issuecomment-35211178
Neato, Jenkins actually listened to me. Merged into master, thanks!
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/602#issuecomment-35209157
Merged build finished.
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-pos
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/602#issuecomment-35209158
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12732/
If your pr
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/600#issuecomment-35208921
To answer my own question, the reason we do this is probably because we
have no choice when integrating with Java. Additionally, use of ClassTag as in
our API r
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/602#issuecomment-35208277
Merged build triggered.
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-p
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/602#issuecomment-35208278
Merged build started.
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/602#issuecomment-35208183
Jenkins doesn't like me at the moment, @rxin is trying to get that fixed.
In the meantime, can someone with permissions say "Jenkins, test this please." ?
If y
Github user CodingCat commented on the pull request:
https://github.com/apache/incubator-spark/pull/602#issuecomment-35207540
@aarondav just fixed
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your
Github user aarondav commented on the pull request:
https://github.com/apache/incubator-spark/pull/602#issuecomment-35207406
Looks good to me, just fix the little typo and I'll merge it in. (I would
do it myself but I'm not certain how to amend the commit while keeping you as
the orig
1 - 100 of 140 matches
Mail list logo
