I'll kick of the voting with a +1.
On Sat, Jan 18, 2014 at 11:05 PM, Patrick Wendell wrote:
> Please vote on releasing the following candidate as Apache Spark
> (incubating) version 0.9.0.
>
> A draft of the release notes along with the changes file is attached
> to this e-mail.
>
> The tag to be
This vote is cancelled in favor of rc2 which I'll post shortly.
On Sat, Jan 18, 2014 at 12:14 PM, Patrick Wendell wrote:
> Mridul, thanks a *lot* for pointing this out. This is indeed an issue
> and something which warrants cutting a new RC.
>
> - Patrick
>
> On Sat, Jan 18, 2014 at 11:14 AM, Mri
Hey All,
I ran into an issue when trying to run SparkPi as described in the Spark on
YARN doc.
14/01/18 10:52:09 ERROR spark.SparkContext: Error adding jar
(java.io.FileNotFoundException:
spark-examples-assembly-0.9.0-incubating-SNAPSHOT.jar (No such file or
directory)), was the --addJars option
Hah! Stupid English language -- by "fixed" I mean established/stabilized,
not repaired.
On Sat, Jan 18, 2014 at 12:42 PM, Mark Hamstra wrote:
> Yeah, I can get on board with that -- gives us another chance to
> re-think/re-work config files to address the limitations Matei mentioned
> before th
Yeah, I can get on board with that -- gives us another chance to
re-think/re-work config files to address the limitations Matei mentioned
before the interface is fixed for 1.0.
On Sat, Jan 18, 2014 at 12:27 PM, Patrick Wendell wrote:
> Hey Mark - ya if we did add this I think it would be in the
Hey Mark - ya if we did add this I think it would be in the next major release.
On Sat, Jan 18, 2014 at 12:17 PM, Mark Hamstra wrote:
> That later release should be at least 0.10.0, then, since use of config
> files won't be backward compatible with 0.9.0.
>
>
> On Sat, Jan 18, 2014 at 12:11 PM,
That later release should be at least 0.10.0, then, since use of config
files won't be backward compatible with 0.9.0.
On Sat, Jan 18, 2014 at 12:11 PM, Matei Zaharia wrote:
> We can add config files in a later release. They were never officially
> released, and were only in master for about a m
Yeah, this is exactly my reasoning as well.
Matei
On Jan 18, 2014, at 12:14 PM, Mridul Muralidharan wrote:
> IMO we should shoot for more stable interfaces and not break them just
> to workaround bugs - unless the benefit of breaking compatibility is
> offset by the added functionality.
> Since
Mridul, thanks a *lot* for pointing this out. This is indeed an issue
and something which warrants cutting a new RC.
- Patrick
On Sat, Jan 18, 2014 at 11:14 AM, Mridul Muralidharan wrote:
> I would vote -1 for this release until we resolve config property
> issue [1] : if there is a known resolu
IMO we should shoot for more stable interfaces and not break them just
to workaround bugs - unless the benefit of breaking compatibility is
offset by the added functionality.
Since I was not around for a while, I am not sure how much config file
feature was requested ...
Regards,
Mridul
On Sun, J
We can add config files in a later release. They were never officially
released, and were only in master for about a month.
One other thing to note is that the config file feature is kind of limited
anyway. Users will want to have a separate config file with each app, which
they have to ship wi
Really? Disabling config files seems to me to be a bigger/more onerous
change for users than spark.speculation=true|false =>
spark.speculation.enabled=true|false and spark.locality.wait =>
spark.locality.wait.default.
On Sat, Jan 18, 2014 at 11:36 AM, Matei Zaharia wrote:
> This is definitely a
I also just went over the config options to see how pervasive this is. In
addition to speculation, there is one more "conflict" of this kind:
spark.locality.wait
spark.locality.wait.node
spark.locality.wait.process
spark.locality.wait.rack
spark.speculation
spark.speculation.interval
spark.specu
This is definitely an important issue to fix. Instead of renaming properties,
one solution would be to replace Typesafe Config with just reading Java system
properties, and disable config files for this release. I kind of like that over
renaming.
Matei
On Jan 18, 2014, at 11:30 AM, Mridul Mura
Hi,
Speculation was an example, there are others in spark which are
affected by this ...
Some of them have been around for a while, so will break existing code/scripts.
Regards,
Mridul
On Sun, Jan 19, 2014 at 12:51 AM, Nan Zhu wrote:
> change spark.speculation to spark.speculation.switch?
>
>
change spark.speculation to spark.speculation.switch?
maybe we can restrict that all properties in Spark should be "three levels"
On Sat, Jan 18, 2014 at 2:10 PM, Mridul Muralidharan wrote:
> Hi,
>
> Unless I am mistaken, the change to using typesafe ConfigFactory has
> broken some of the sys
I would vote -1 for this release until we resolve config property
issue [1] : if there is a known resolution for this (which I could not
find unfortunately, apologies if it exists !), then will change my
vote.
Thanks,
Mridul
[1]
http://apache-spark-developers-list.1001551.n3.nabble.com/Config-p
Hi,
Unless I am mistaken, the change to using typesafe ConfigFactory has
broken some of the system properties we use in spark.
For example: if we have both
-Dspark.speculation=true -Dspark.speculation.multiplier=0.95
set, then the spark.speculation property is dropped.
The rules of parseProper
18 matches
Mail list logo
