We have maven-enforcer-plugin defined in the pom. I don't know why it
didn't work for you. Could you try rebuild with maven2 and confirm
that there is no error message? If that is the case, please create a
JIRA for it. Thanks! -Xiangrui
On Wed, Jul 9, 2014 at 3:53 AM, Bharath Ravi Kumar
It seems to me a setup issue. I just tested news20.binary (1355191
features) on a 2-node EC2 cluster and it worked well. I added one line
to conf/spark-env.sh:
export SPARK_JAVA_OPTS= -Dspark.akka.frameSize=20
and launched spark-shell with --driver-memory 20g. Could you re-try
with an EC2
Hi Bharath,
1) Did you sync the spark jar and conf to the worker nodes after build?
2) Since the dataset is not large, could you try local mode first
using `spark-summit --driver-memory 12g --master local[*]`?
3) Try to use less number of partitions, say 5.
If the problem is still there, please
Hi Xiangrui,
The issue with aggergating/counting over large feature vectors (as part of
LogisticRegressionWithSGD) continues to exist, but now in another form:
while the execution doesn't freeze (due to SPARK-1112), it now fails at the
second or third gradient descent iteration consistently with
Hi Bharath,
This is related to SPARK-1112, which we already found the root cause.
I will let you know when this is fixed.
Best,
Xiangrui
On Tue, Jun 17, 2014 at 7:37 PM, Bharath Ravi Kumar reachb...@gmail.com wrote:
Couple more points:
1)The inexplicable stalling of execution with large
Thanks. I'll await the fix to re-run my test.
On Thu, Jun 19, 2014 at 8:28 AM, Xiangrui Meng men...@gmail.com wrote:
Hi Bharath,
This is related to SPARK-1112, which we already found the root cause.
I will let you know when this is fixed.
Best,
Xiangrui
On Tue, Jun 17, 2014 at 7:37 PM,
Hi,
(Apologies for the long mail, but it's necessary to provide sufficient
details considering the number of issues faced.)
I'm running into issues testing LogisticRegressionWithSGD a two node
cluster (each node with 24 cores and 16G available to slaves out of 24G on
the system). Here's a
Hi Bharath,
Thanks for posting the details! Which Spark version are you using?
Best,
Xiangrui
On Tue, Jun 17, 2014 at 6:48 AM, Bharath Ravi Kumar reachb...@gmail.com wrote:
Hi,
(Apologies for the long mail, but it's necessary to provide sufficient
details considering the number of issues
Hi Xiangrui ,
I'm using 1.0.0.
Thanks,
Bharath
On 18-Jun-2014 1:43 am, Xiangrui Meng men...@gmail.com wrote:
Hi Bharath,
Thanks for posting the details! Which Spark version are you using?
Best,
Xiangrui
On Tue, Jun 17, 2014 at 6:48 AM, Bharath Ravi Kumar reachb...@gmail.com
wrote:
Couple more points:
1)The inexplicable stalling of execution with large feature sets appears
similar to that reported with the news-20 dataset:
http://mail-archives.apache.org/mod_mbox/spark-user/201406.mbox/%3c53a03542.1010...@gmail.com%3E
2) The NPE trying to call mapToPair convert an RDDLong,
10 matches
Mail list logo
