[jira] [Commented] (SPARK-4325) Improve spark-ec2 cluster launch times

2014-12-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14257939#comment-14257939 ] Nicholas Chammas commented on SPARK-4325: - OK, I created a

[jira] [Created] (SPARK-4948) Use pssh instead of bash-isms and remove unnecessary operations

2014-12-23 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-4948: --- Summary: Use pssh instead of bash-isms and remove unnecessary operations Key: SPARK-4948 URL: https://issues.apache.org/jira/browse/SPARK-4948 Project: Spark

[jira] [Commented] (SPARK-2541) Standalone mode can't access secure HDFS anymore

2014-12-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14256357#comment-14256357 ] Nicholas Chammas commented on SPARK-2541: - I was just wondering if we needed to

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-12-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14256683#comment-14256683 ] Nicholas Chammas commented on SPARK-3821: - Per the discussion earlier, I've

[jira] [Reopened] (SPARK-4325) Improve spark-ec2 cluster launch times

2014-12-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas reopened SPARK-4325: - Hey [~joshrosen], though [#3195|https://github.com/apache/spark/pull/3195] relates to this

[jira] [Commented] (SPARK-4241) spark_ec2.py support China AWS region: cn-north-1

2014-12-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14256694#comment-14256694 ] Nicholas Chammas commented on SPARK-4241: - [~joshrosen] - I noticed you linked

[jira] [Commented] (SPARK-3432) Fix logging of unit test execution time

2014-12-21 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14255106#comment-14255106 ] Nicholas Chammas commented on SPARK-3432: - [~rxin] Is this still an issue? As far

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-12-20 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254605#comment-14254605 ] Nicholas Chammas commented on SPARK-3561: - {quote} One of the main reasons for

[jira] [Commented] (SPARK-2541) Standalone mode can't access secure HDFS anymore

2014-12-20 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254625#comment-14254625 ] Nicholas Chammas commented on SPARK-2541: - Hey [~tgraves], is there someone we

[jira] [Created] (SPARK-4868) Twitter DStream.map() throws Task not serializable

2014-12-16 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-4868: --- Summary: Twitter DStream.map() throws Task not serializable Key: SPARK-4868 URL: https://issues.apache.org/jira/browse/SPARK-4868 Project: Spark Issue

[jira] [Commented] (SPARK-4868) Twitter DStream.map() throws Task not serializable

2014-12-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249387#comment-14249387 ] Nicholas Chammas commented on SPARK-4868: - cc [~tdas], [~adav] Twitter

[jira] [Commented] (SPARK-4868) Twitter DStream.map() throws Task not serializable

2014-12-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249582#comment-14249582 ] Nicholas Chammas commented on SPARK-4868: - Replacing {{noop()}} with this {code}

[jira] [Commented] (SPARK-4826) Possible flaky tests in WriteAheadLogBackedBlockRDDSuite: java.lang.IllegalStateException: File exists and there is no append support!

2014-12-14 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246091#comment-14246091 ] Nicholas Chammas commented on SPARK-4826: - This raises an interesting test

[jira] [Commented] (SPARK-4826) Possible flaky tests in WriteAheadLogBackedBlockRDDSuite: java.lang.IllegalStateException: File exists and there is no append support!

2014-12-14 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246122#comment-14246122 ] Nicholas Chammas commented on SPARK-4826: - I just cooked up a quick way of

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14240519#comment-14240519 ] Nicholas Chammas commented on SPARK-3431: - OK, thanks for the updates, Sean and

[jira] [Comment Edited] (SPARK-3431) Parallelize execution of tests

2014-12-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14240519#comment-14240519 ] Nicholas Chammas edited comment on SPARK-3431 at 12/10/14 2:01 AM:

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14240530#comment-14240530 ] Nicholas Chammas commented on SPARK-3431: - For the record, the suite that I'm

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238330#comment-14238330 ] Nicholas Chammas commented on SPARK-3431: - I am currently (and have been) actively

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238596#comment-14238596 ] Nicholas Chammas commented on SPARK-3431: - Thanks for assigning the issue to me,

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238666#comment-14238666 ] Nicholas Chammas commented on SPARK-3431: - OK, here's a patch for {{pom.xml}} that

[jira] [Comment Edited] (SPARK-3431) Parallelize execution of tests

2014-12-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238666#comment-14238666 ] Nicholas Chammas edited comment on SPARK-3431 at 12/8/14 11:36 PM:

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238703#comment-14238703 ] Nicholas Chammas commented on SPARK-3431: - Here are some of the errors: {code}

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238769#comment-14238769 ] Nicholas Chammas commented on SPARK-3431: - [~nkeywal] - I took a quick look at

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14236540#comment-14236540 ] Nicholas Chammas commented on SPARK-3431: - Here's an example failure I don't

[jira] [Comment Edited] (SPARK-3431) Parallelize execution of tests

2014-12-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14234702#comment-14234702 ] Nicholas Chammas edited comment on SPARK-3431 at 12/6/14 3:11 AM:

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-04 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14234702#comment-14234702 ] Nicholas Chammas commented on SPARK-3431: - I think I'm on to something, but I need

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-04 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14234783#comment-14234783 ] Nicholas Chammas commented on SPARK-3431: - As an aside, I expect there to be some

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-12-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233568#comment-14233568 ] Nicholas Chammas commented on SPARK-3431: - [~joshrosen] I tried [that patch you

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-11-20 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1421#comment-1421 ] Nicholas Chammas commented on SPARK-3431: - [~joshrosen] - Per [this comment |

[jira] [Commented] (SPARK-4325) Improve spark-ec2 cluster launch times

2014-11-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14214239#comment-14214239 ] Nicholas Chammas commented on SPARK-4325: - {quote} Replace instances of download;

[jira] [Commented] (SPARK-4326) unidoc is broken on master

2014-11-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207177#comment-14207177 ] Nicholas Chammas commented on SPARK-4326: - Side question: Should we be (or are we

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-11-10 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14205358#comment-14205358 ] Nicholas Chammas commented on SPARK-3821: - Here's the [benchmark of the launch

[jira] [Created] (SPARK-4325) Improve spark-ec2 cluster launch times

2014-11-10 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-4325: --- Summary: Improve spark-ec2 cluster launch times Key: SPARK-4325 URL: https://issues.apache.org/jira/browse/SPARK-4325 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-11-10 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14205358#comment-14205358 ] Nicholas Chammas edited comment on SPARK-3821 at 11/10/14 9:34 PM:

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-11-10 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14205401#comment-14205401 ] Nicholas Chammas commented on SPARK-3821: - Thanks for taking a look [~danospv].

[jira] [Updated] (SPARK-4325) Improve spark-ec2 cluster launch times

2014-11-10 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-4325: Description: There are several optimizations we know we can make to [{{setup.sh}} |

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-11-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14203786#comment-14203786 ] Nicholas Chammas commented on SPARK-3821: - Thanks for the feedback [~shivaram].

[jira] [Commented] (SPARK-4216) Eliminate duplicate Jenkins GitHub posts from AMPLab

2014-11-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14202553#comment-14202553 ] Nicholas Chammas commented on SPARK-4216: - {quote} 1) we carry on w/the duplicate

[jira] [Updated] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-11-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3821: Attachment: packer-proposal.html Develop an automated way of creating Spark images (AMI,

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-11-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14203280#comment-14203280 ] Nicholas Chammas commented on SPARK-3821: - After much dilly-dallying, I am happy

[jira] [Updated] (SPARK-4243) Spark SQL SELECT COUNT DISTINCT optimization

2014-11-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-4243: Description: Spark SQL runs slow when using this code: {code} val sqlContext = new

[jira] [Commented] (SPARK-4216) Eliminate duplicate Jenkins GitHub posts from AMPLab

2014-11-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14199533#comment-14199533 ] Nicholas Chammas commented on SPARK-4216: - Side note: I remember there was a time

[jira] [Commented] (SPARK-4241) spark_ec2.py support China AWS region: cn-north-1

2014-11-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14199710#comment-14199710 ] Nicholas Chammas commented on SPARK-4241: - Could you give it a try and let me know

[jira] [Commented] (SPARK-4241) spark_ec2.py support China AWS region: cn-north-1

2014-11-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14199743#comment-14199743 ] Nicholas Chammas commented on SPARK-4241: - It's not a standard AMI. Also, we can't

[jira] [Commented] (SPARK-4241) spark_ec2.py support China AWS region: cn-north-1

2014-11-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14199932#comment-14199932 ] Nicholas Chammas commented on SPARK-4241: - Thanks for looking into that, Haitao.

[jira] [Commented] (SPARK-1070) Add check for JIRA ticket in the Github pull request title/summary with CI

2014-11-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14194952#comment-14194952 ] Nicholas Chammas commented on SPARK-1070: - [~hsaputra] - The [Spark PR Board |

[jira] [Commented] (SPARK-4216) Eliminate Jenkins GitHub posts from AMPLab

2014-11-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195482#comment-14195482 ] Nicholas Chammas commented on SPARK-4216: - cc [~shaneknapp] [~joshrosen]

[jira] [Updated] (SPARK-4216) Eliminate duplicate Jenkins GitHub posts from AMPLab

2014-11-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-4216: Summary: Eliminate duplicate Jenkins GitHub posts from AMPLab (was: Eliminate Jenkins

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-10-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192273#comment-14192273 ] Nicholas Chammas commented on SPARK-3821: - Hey folks, I was hoping to post a

[jira] [Created] (SPARK-4137) Relative paths don't get handled correctly by spark-ec2

2014-10-29 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-4137: --- Summary: Relative paths don't get handled correctly by spark-ec2 Key: SPARK-4137 URL: https://issues.apache.org/jira/browse/SPARK-4137 Project: Spark

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188938#comment-14188938 ] Nicholas Chammas commented on SPARK-3398: - No problem. I've opened [SPARK-4137] to

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187519#comment-14187519 ] Nicholas Chammas commented on SPARK-3398: - [~michael.griffiths] -

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187553#comment-14187553 ] Nicholas Chammas commented on SPARK-3398: - Hmm, I'm curious: # Why did you have to

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187593#comment-14187593 ] Nicholas Chammas commented on SPARK-3398: - OK, so you're invoking {{spark-ec2}}

[jira] [Commented] (SPARK-4123) Show new dependencies added in pull requests

2014-10-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187598#comment-14187598 ] Nicholas Chammas commented on SPARK-4123: - Sure thing, [~pwendell], though I will

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187861#comment-14187861 ] Nicholas Chammas commented on SPARK-3398: - So I spun up an Ubuntu server on EC2

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187898#comment-14187898 ] Nicholas Chammas commented on SPARK-3398: - I think I've found the issue. It

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187900#comment-14187900 ] Nicholas Chammas commented on SPARK-3398: - If that fixes it for you, then I think

[jira] [Comment Edited] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187900#comment-14187900 ] Nicholas Chammas edited comment on SPARK-3398 at 10/29/14 2:48 AM:

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-10-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183245#comment-14183245 ] Nicholas Chammas commented on SPARK-3821: - Hey [~danospv], I'm currently in the

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-10-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183390#comment-14183390 ] Nicholas Chammas commented on SPARK-3821: - Going for something like EMR's CLI is

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-10-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169331#comment-14169331 ] Nicholas Chammas commented on SPARK-922: [~joshrosen] - Do you mean [this

[jira] [Comment Edited] (SPARK-922) Update Spark AMI to Python 2.7

2014-10-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169331#comment-14169331 ] Nicholas Chammas edited comment on SPARK-922 at 10/13/14 2:19 PM:

[jira] [Created] (SPARK-3928) Support wildcard matches on Parquet files

2014-10-13 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3928: --- Summary: Support wildcard matches on Parquet files Key: SPARK-3928 URL: https://issues.apache.org/jira/browse/SPARK-3928 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3928) Support wildcard matches on Parquet files

2014-10-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169580#comment-14169580 ] Nicholas Chammas commented on SPARK-3928: - cc [~marmbrus] Support wildcard

[jira] [Updated] (SPARK-3849) Automate remaining Scala style rules

2014-10-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3849: Description: Style problems continue to take up a large amount of review time, mostly

[jira] [Updated] (SPARK-3849) Automate remaining Spark Code Style Guide rules

2014-10-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3849: Summary: Automate remaining Spark Code Style Guide rules (was: Automate remaining Scala

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-10-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168459#comment-14168459 ] Nicholas Chammas commented on SPARK-922: [~joshrosen] Are you open to having this

[jira] [Updated] (SPARK-3850) Scala style: disallow trailing spaces

2014-10-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3850: Summary: Scala style: disallow trailing spaces (was: Scala style: Disallow trailing

[jira] [Updated] (SPARK-3850) Scala style: disallow trailing spaces

2014-10-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3850: Description: [Ted Yu on the dev

[jira] [Commented] (SPARK-3376) Memory-based shuffle strategy to reduce overhead of disk I/O

2014-10-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165359#comment-14165359 ] Nicholas Chammas commented on SPARK-3376: - [~matei], [~rxin], [~pwendell]: This is

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163618#comment-14163618 ] Nicholas Chammas commented on SPARK-3561: - {quote} Obviously this does not work in

[jira] [Created] (SPARK-3849) Automate remaining Scala style rules

2014-10-08 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3849: --- Summary: Automate remaining Scala style rules Key: SPARK-3849 URL: https://issues.apache.org/jira/browse/SPARK-3849 Project: Spark Issue Type:

[jira] [Created] (SPARK-3850) Scala style: Disallow trailing spaces

2014-10-08 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3850: --- Summary: Scala style: Disallow trailing spaces Key: SPARK-3850 URL: https://issues.apache.org/jira/browse/SPARK-3850 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-10-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14162663#comment-14162663 ] Nicholas Chammas commented on SPARK-3821: - [~shivaram] / [~pwendell]: # In a Spark

[jira] [Created] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-10-06 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3821: --- Summary: Develop an automated way of creating Spark images (AMI, Docker, and others) Key: SPARK-3821 URL: https://issues.apache.org/jira/browse/SPARK-3821

[jira] [Commented] (SPARK-3314) Script creation of AMIs

2014-10-06 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160901#comment-14160901 ] Nicholas Chammas commented on SPARK-3314: - Sounds good. I've created [SPARK-3821]

[jira] [Updated] (SPARK-3479) Have Jenkins show which category of tests failed in his GitHub messages

2014-10-06 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3479: Summary: Have Jenkins show which category of tests failed in his GitHub messages (was:

[jira] [Commented] (SPARK-3314) Script creation of AMIs

2014-10-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14158777#comment-14158777 ] Nicholas Chammas commented on SPARK-3314: - Hey [~holdenk], I think this is a great

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2014-10-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156904#comment-14156904 ] Nicholas Chammas commented on SPARK-2870: - [~marmbrus] - A related feature that I

[jira] [Commented] (SPARK-3105) Calling cache() after RDDs are pipelined has no effect in PySpark

2014-10-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156936#comment-14156936 ] Nicholas Chammas commented on SPARK-3105: - I think it's definitely important for

[jira] [Updated] (SPARK-2463) Creating then stopping StreamingContext multiple times from shell generates duplicate Streaming tabs in UI

2014-09-30 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2463: Summary: Creating then stopping StreamingContext multiple times from shell generates

[jira] [Commented] (SPARK-2008) Enhance spark-ec2 to be able to add and remove slaves to an existing cluster

2014-09-30 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153723#comment-14153723 ] Nicholas Chammas commented on SPARK-2008: - [~shivaram] [~pwendell] - Is this a

[jira] [Commented] (SPARK-2247) Data frame (or Pandas) like API for structured data

2014-09-30 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14154264#comment-14154264 ] Nicholas Chammas commented on SPARK-2247: - Is [Adato's work on Distributed

[jira] [Commented] (SPARK-3522) Make spark-ec2 verbosity configurable

2014-09-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151787#comment-14151787 ] Nicholas Chammas commented on SPARK-3522: - Always logging to a file sounds like a

[jira] [Commented] (SPARK-3522) Make spark-ec2 verbosity configurable

2014-09-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151158#comment-14151158 ] Nicholas Chammas commented on SPARK-3522: - [~pwendell] [~shivaram] - Does this

[jira] [Updated] (SPARK-3522) Make spark-ec2 verbosity configurable

2014-09-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3522: Description: When launching a cluster, {{spark-ec2}} spits out a lot of stuff that feels

[jira] [Updated] (SPARK-3479) Have Jenkins show which tests failed in his GitHub messages

2014-09-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3479: Description: A nice thing to do for starters would be to report which category of tests

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-09-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143656#comment-14143656 ] Nicholas Chammas commented on SPARK-3431: - [~joshrosen] I can take a crack at this

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-09-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143707#comment-14143707 ] Nicholas Chammas commented on SPARK-3431: - {quote} Do you know how maven / sbt

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2014-09-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143720#comment-14143720 ] Nicholas Chammas commented on SPARK-2870: - [~marmbrus] - API-wise, how are you

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-09-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143744#comment-14143744 ] Nicholas Chammas commented on SPARK-3431: - I see. I'll try to look into it then. I

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135597#comment-14135597 ] Nicholas Chammas commented on SPARK-3533: - [~kzhang] - I noticed you authored

[jira] [Commented] (SPARK-2022) Spark 1.0.0 is failing if mesos.coarse set to true

2014-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135663#comment-14135663 ] Nicholas Chammas commented on SPARK-2022: - Pinging [~pwendell] about closing this

[jira] [Commented] (SPARK-1455) Determine which test suites to run based on code changes

2014-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136418#comment-14136418 ] Nicholas Chammas commented on SPARK-1455: - Which approach do y'all prefer? # Have

[jira] [Commented] (SPARK-2463) Creating multiple StreamingContexts from shell generates duplicate Streaming tabs in UI

2014-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136432#comment-14136432 ] Nicholas Chammas commented on SPARK-2463: - [~joshrosen] - Though the most recent

[jira] [Commented] (SPARK-3528) Reading data from file:/// should be called NODE_LOCAL not PROCESS_LOCAL

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133927#comment-14133927 ] Nicholas Chammas commented on SPARK-3528: - [~aash] - How about for data read from

[jira] [Commented] (SPARK-3526) Docs section on data locality

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133935#comment-14133935 ] Nicholas Chammas commented on SPARK-3526: - FYI: Looks like the valid localities

[jira] [Updated] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3533: Affects Version/s: 1.1.0 Add saveAsTextFileByKey() method to RDDs

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134800#comment-14134800 ] Nicholas Chammas commented on SPARK-922: [~joshrosen] By the way, as part of this

<    5   6   7   8   9   10   11   >