I actually find my version of 3 more readable than the one with the `_`,
which looks too much like a partially applied function. It's a minor
issue, though.
On Sat, Apr 16, 2016 at 11:56 PM, Hyukjin Kwon wrote:
> Hi Mark,
>
> I know but that could harm readability. AFAIK, for this reason, that
Hi,
Is this a bad idea to create `SparkContext` with a `local-cluster` mode by
yourself like '
https://github.com/apache/spark/blob/master/core/src/test/scala/org/apache/spark/ShuffleSuite.scala#L55
'?
// maropu
On Sun, Apr 17, 2016 at 9:47 AM, Evan Chan wrote:
> Hey folks,
>
> I'd like to use
What I want to find out is how to run tests like Spark's with
local-cluster, just like that suite, but in your own projects. Has
anyone done this?
On Sun, Apr 17, 2016 at 5:37 AM, Takeshi Yamamuro wrote:
> Hi,
> Is this a bad idea to create `SparkContext` with a `local-cluster` mode by
> yourse
On Sat, Apr 16, 2016 at 11:12 PM, Reynold Xin wrote:
> First, really thank you for leading the discussion.
>
> I am concerned that it'd hurt Spark more than it helps. As many others
> have pointed out, this unnecessarily creates a new tier of connectors or
> 3rd party libraries appearing to be en
Take a look at spark testing base.
https://github.com/holdenk/spark-testing-base/blob/master/README.md
On Apr 17, 2016 10:28 AM, "Evan Chan" wrote:
> What I want to find out is how to run tests like Spark's with
> local-cluster, just like that suite, but in your own projects. Has
> anyone done
Hi Devs,
In case of stop the world GC events on the driver JVM, since all the
application threads will be stopped, there won't be any new task scheduled /
launched on the executors. In cases where the full collection is happening,
the applications threads may be stopped for a long time, and if the
I realized that recent Jenkins among different pull requests always fails
in the following two tests
"SPARK-8020: set sql conf in spark conf"
"SPARK-9757 Persist Parquet relation with decimal column"
Here are examples.
https://github.com/apache/spark/pull/11956 (consoleFull:
https://amplab.cs.be
Your understanding is correct. If the driver is stuck in GC, then during
that period it cannot schedule any tasks.
On Sun, Apr 17, 2016 at 10:27 AM, Rahul Tanwani
wrote:
> Hi Devs,
>
> In case of stop the world GC events on the driver JVM, since all the
> application threads will be stopped, th
Does that not mean, GC settings with concurrent collectors should be
preferred over parallel collectors atleast on the driver side? If so, why
not have concurrent collectors specified by default when the driver JVM is
launched without any overriding on this part?
--
View this message in context:
i find version 3 without the _ also more readable
On Sun, Apr 17, 2016 at 3:02 AM, Mark Hamstra
wrote:
> I actually find my version of 3 more readable than the one with the `_`,
> which looks too much like a partially applied function. It's a minor
> issue, though.
>
> On Sat, Apr 16, 2016 at 1
+1
Yea, I am facing this problem as well,
https://github.com/apache/spark/pull/12452
I thought they are spurious because the tests are passed in my local.
2016-04-18 3:26 GMT+09:00 Kazuaki Ishizaki :
> I realized that recent Jenkins among different pull requests always fails
> in the followin
Hi guys,
Any update on this?
Best
On Tue, Apr 12, 2016 at 12:46 PM, Niranda Perera
wrote:
> Hi all,
>
> I have encountered a small issue in the standalone recovery mode.
>
> Let's say there was an application A running in the cluster. Due to some
> issue, the entire cluster, together with the
Also hitting this: https://github.com/apache/spark/pull/12455.
On Sun, Apr 17, 2016 at 9:22 PM, Hyukjin Kwon wrote:
> +1
>
> Yea, I am facing this problem as well,
> https://github.com/apache/spark/pull/12452
>
> I thought they are spurious because the tests are passed in my local.
>
>
>
> 201
I haven't looked closely at this, but I think your proposal makes sense.
On Sun, Apr 17, 2016 at 6:40 PM, Niranda Perera
wrote:
> Hi guys,
>
> Any update on this?
>
> Best
>
> On Tue, Apr 12, 2016 at 12:46 PM, Niranda Perera > wrote:
>
>> Hi all,
>>
>> I have encountered a small issue in the s
Jon, Thanks. I think I've figured it out, actually. It's really
simple, one needs to simply set spark.executor.extraClassPath to the
current value of the java class path (java.class.path system
property). Also, to not use HiveContext, which gives errors about
initializing a Derby database mu
15 matches
Mail list logo