[jira] [Commented] (SPARK-13740) add null check for _verify_type in types.py

[ https://issues.apache.org/jira/browse/SPARK-13740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274738#comment-15274738 ] Nicholas Chammas commented on SPARK-13740: -- I noticed the PR only modifies PySpark. Are similar

[jira] [Commented] (SPARK-11319) PySpark silently accepts null values in non-nullable DataFrame fields.

[ https://issues.apache.org/jira/browse/SPARK-11319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274728#comment-15274728 ] Nicholas Chammas commented on SPARK-11319: -- [~marmbrus] / [~yhuai] - Does SPARK-13740 resolve

[jira] [Commented] (SPARK-14932) Allow DataFrame.replace() to replace values with None

[ https://issues.apache.org/jira/browse/SPARK-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259266#comment-15259266 ] Nicholas Chammas commented on SPARK-14932: -- [~marmbrus] - Not sure if you're a good person to

[jira] [Created] (SPARK-14932) Allow DataFrame.replace() to replace values with None

Nicholas Chammas created SPARK-14932: Summary: Allow DataFrame.replace() to replace values with None Key: SPARK-14932 URL: https://issues.apache.org/jira/browse/SPARK-14932 Project: Spark

[jira] [Created] (SPARK-14742) Redirect spark-ec2 doc to new location

Nicholas Chammas created SPARK-14742: Summary: Redirect spark-ec2 doc to new location Key: SPARK-14742 URL: https://issues.apache.org/jira/browse/SPARK-14742 Project: Spark Issue Type:

[jira] [Commented] (SPARK-8327) Ganglia failed to start while starting standalone on EC 2 spark with spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15249144#comment-15249144 ] Nicholas Chammas commented on SPARK-8327: - [~vvladymyrov] - Is this still an issue? If so, I

[jira] [Commented] (SPARK-6527) sc.binaryFiles can not access files on s3

[ https://issues.apache.org/jira/browse/SPARK-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15249141#comment-15249141 ] Nicholas Chammas commented on SPARK-6527: - Did the s3a suggestion work? If not, did anybody file

[jira] [Comment Edited] (SPARK-6527) sc.binaryFiles can not access files on s3

[ https://issues.apache.org/jira/browse/SPARK-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15249141#comment-15249141 ] Nicholas Chammas edited comment on SPARK-6527 at 4/20/16 2:27 AM: -- Did

[jira] [Closed] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas closed SPARK-3821. --- Resolution: Won't Fix I'm resolving this as "Won't Fix" due to lack of interest, both on my

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214577#comment-15214577 ] Nicholas Chammas commented on SPARK-3533: - I've added 2 workaround to this issue to the

[jira] [Updated] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3533: Description: Users often have a single RDD of key-value pairs that they want to save to

[jira] [Commented] (SPARK-7481) Add Hadoop 2.6+ profile to pull in object store FS accessors

[ https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197451#comment-15197451 ] Nicholas Chammas commented on SPARK-7481: - (Sorry Steve; can't comment on your proposal since I

[jira] [Commented] (SPARK-7505) Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc.

[ https://issues.apache.org/jira/browse/SPARK-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15181776#comment-15181776 ] Nicholas Chammas commented on SPARK-7505: - I believe items 1, 3, and 4 still apply. They're minor

[jira] [Commented] (SPARK-13596) Move misc top-level build files into appropriate subdirs

[ https://issues.apache.org/jira/browse/SPARK-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15180072#comment-15180072 ] Nicholas Chammas commented on SPARK-13596: -- Looks like {{tox.ini}} is only used by {{pep8}}, so

[jira] [Commented] (SPARK-7481) Add Hadoop 2.6+ profile to pull in object store FS accessors

[ https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176559#comment-15176559 ] Nicholas Chammas commented on SPARK-7481: - I'm not comfortable working with Maven so I can't

[jira] [Commented] (SPARK-7481) Add Hadoop 2.6+ profile to pull in object store FS accessors

[ https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176551#comment-15176551 ] Nicholas Chammas commented on SPARK-7481: - {quote} One issue here that hadoop 2.6's hadoop-aws

[jira] [Commented] (SPARK-7481) Add Hadoop 2.6+ profile to pull in object store FS accessors

[ https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174438#comment-15174438 ] Nicholas Chammas commented on SPARK-7481: - Many people seem to be downgrading to use Spark built

[jira] [Commented] (SPARK-5189) Reorganize EC2 scripts so that nodes can be provisioned independent of Spark master

[ https://issues.apache.org/jira/browse/SPARK-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119220#comment-15119220 ] Nicholas Chammas commented on SPARK-5189: - FWIW, I found this issue to be practically unsolvable

[jira] [Commented] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098887#comment-15098887 ] Nicholas Chammas commented on SPARK-12824: -- Ah, good catch. This appears to be a known behavior

[jira] [Commented] (SPARK-12824) Failure to maintain consistent RDD references in pyspark

[ https://issues.apache.org/jira/browse/SPARK-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098336#comment-15098336 ] Nicholas Chammas commented on SPARK-12824: -- I can reproduce this issue. Here's a more concise

[jira] [Comment Edited] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14203280#comment-14203280 ] Nicholas Chammas edited comment on SPARK-3821 at 12/18/15 9:08 PM: ---

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053977#comment-15053977 ] Nicholas Chammas commented on SPARK-2870: - > Do you think its OK to close this issue? I haven't

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053131#comment-15053131 ] Nicholas Chammas commented on SPARK-2870: - Go for it. I don't think anyone else is. > Thorough

[jira] [Created] (SPARK-12107) Update spark-ec2 versions

Nicholas Chammas created SPARK-12107: Summary: Update spark-ec2 versions Key: SPARK-12107 URL: https://issues.apache.org/jira/browse/SPARK-12107 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-9999) Dataset API on top of Catalyst/DataFrame

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022735#comment-15022735 ] Nicholas Chammas edited comment on SPARK- at 11/23/15 8:06 PM: ---

[jira] [Commented] (SPARK-9999) Dataset API on top of Catalyst/DataFrame

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022735#comment-15022735 ] Nicholas Chammas commented on SPARK-: - [~sandyr] - Hmm, so are you saying that, generally

[jira] [Commented] (SPARK-9999) Dataset API on top of Catalyst/DataFrame

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022957#comment-15022957 ] Nicholas Chammas commented on SPARK-: - If you are referring to my comment, note that I am

[jira] [Commented] (SPARK-11903) Deprecate make-distribution.sh --skip-java-test

[ https://issues.apache.org/jira/browse/SPARK-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020729#comment-15020729 ] Nicholas Chammas commented on SPARK-11903: -- Also, we could just leave the option in there and

[jira] [Created] (SPARK-11903) Deprecate make-distribution.sh --skip-java-test

Nicholas Chammas created SPARK-11903: Summary: Deprecate make-distribution.sh --skip-java-test Key: SPARK-11903 URL: https://issues.apache.org/jira/browse/SPARK-11903 Project: Spark

[jira] [Commented] (SPARK-11903) Deprecate make-distribution.sh --skip-java-test

[ https://issues.apache.org/jira/browse/SPARK-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020725#comment-15020725 ] Nicholas Chammas commented on SPARK-11903: -- cc [~pwendell] and [~srowen] - Y'all probably know

[jira] [Commented] (SPARK-11903) Deprecate make-distribution.sh --skip-java-test

[ https://issues.apache.org/jira/browse/SPARK-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020728#comment-15020728 ] Nicholas Chammas commented on SPARK-11903: -- Oh, could you elaborate a bit? From what I

[jira] [Updated] (SPARK-11903) Deprecate make-distribution.sh --skip-java-test

[ https://issues.apache.org/jira/browse/SPARK-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-11903: - Description: The {{\-\-skip-java-test}} option to {{make-distribution.sh}} [does not

[jira] [Commented] (SPARK-9999) Dataset API on top of Catalyst/DataFrame

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15019214#comment-15019214 ] Nicholas Chammas commented on SPARK-: - Arriving a little late to this discussion. Quick

[jira] [Commented] (SPARK-11744) bin/pyspark --version doesn't return version and exit

[ https://issues.apache.org/jira/browse/SPARK-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005572#comment-15005572 ] Nicholas Chammas commented on SPARK-11744: -- Not sure who would be the best person to comment on

[jira] [Updated] (SPARK-11744) bin/pyspark --version doesn't return version and exit

[ https://issues.apache.org/jira/browse/SPARK-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-11744: - Description: {{bin/pyspark \-\-help}} offers a {{\-\-version}} option: {code} $

[jira] [Created] (SPARK-11744) bin/pyspark --version doesn't return version and exit

Nicholas Chammas created SPARK-11744: Summary: bin/pyspark --version doesn't return version and exit Key: SPARK-11744 URL: https://issues.apache.org/jira/browse/SPARK-11744 Project: Spark

[jira] [Updated] (SPARK-11744) bin/pyspark --version doesn't return version and exit

[ https://issues.apache.org/jira/browse/SPARK-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-11744: - Description: {{bin/pyspark \-\-help}} offers a {{\-\-version}} option: {code} $

[jira] [Commented] (SPARK-3342) m3 instances don't get local SSDs

[ https://issues.apache.org/jira/browse/SPARK-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974660#comment-14974660 ] Nicholas Chammas commented on SPARK-3342: - FWIW, that statement on M3 instances is [no longer

[jira] [Commented] (SPARK-10002) SSH problem during Setup of Spark(1.3.0) cluster on EC2

[ https://issues.apache.org/jira/browse/SPARK-10002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969814#comment-14969814 ] Nicholas Chammas commented on SPARK-10002: -- [~deepalib] - Is {{--private-ips}} the solution, as

[jira] [Commented] (SPARK-2622) Add Jenkins build numbers to SparkQA messages

[ https://issues.apache.org/jira/browse/SPARK-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803169#comment-14803169 ] Nicholas Chammas commented on SPARK-2622: - [~mxm] - I noticed you have been posting this kind of

[jira] [Commented] (SPARK-2622) Add Jenkins build numbers to SparkQA messages

[ https://issues.apache.org/jira/browse/SPARK-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804559#comment-14804559 ] Nicholas Chammas commented on SPARK-2622: - No worries. Thanks for quickly finding and resolving

[jira] [Commented] (SPARK-4216) Eliminate duplicate Jenkins GitHub posts from AMPLab

[ https://issues.apache.org/jira/browse/SPARK-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791016#comment-14791016 ] Nicholas Chammas commented on SPARK-4216: - Thanks Josh! > Eliminate duplicate Jenkins GitHub

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735323#comment-14735323 ] Nicholas Chammas commented on SPARK-3369: - Sean said: {quote} I don't think there's a "why" –

[jira] [Commented] (SPARK-10191) spark-ec2 cannot stop running cluster

[ https://issues.apache.org/jira/browse/SPARK-10191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710093#comment-14710093 ] Nicholas Chammas commented on SPARK-10191: -- Can you fill in the description here

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705446#comment-14705446 ] Nicholas Chammas commented on SPARK-3533: - {quote} Nicholas Chammas Have you been

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705182#comment-14705182 ] Nicholas Chammas commented on SPARK-3533: - No need to open a separate ticket if

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699613#comment-14699613 ] Nicholas Chammas commented on SPARK-3533: - [~silasdavis] - If you already have a

[jira] [Commented] (SPARK-8960) Style cleanup of spark_ec2.py

[ https://issues.apache.org/jira/browse/SPARK-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14622542#comment-14622542 ] Nicholas Chammas commented on SPARK-8960: - Style cleanup is OK, but should be

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605822#comment-14605822 ] Nicholas Chammas commented on SPARK-8670: - Not sure. Does Scala offer the same

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606178#comment-14606178 ] Nicholas Chammas commented on SPARK-8670: - FYI: `df.stats.age` works neither on

[jira] [Comment Edited] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606328#comment-14606328 ] Nicholas Chammas edited comment on SPARK-8670 at 6/29/15 9:01 PM:

[jira] [Resolved] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-8670. - Resolution: Invalid Nested columns can't be referenced (but they can be selected)

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606328#comment-14606328 ] Nicholas Chammas commented on SPARK-8670: - After a discussion with [~davies], it

[jira] [Created] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

Nicholas Chammas created SPARK-8670: --- Summary: Nested columns can't be referenced (but they can be selected) Key: SPARK-8670 URL: https://issues.apache.org/jira/browse/SPARK-8670 Project: Spark

[jira] [Updated] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-8670: Description: This is strange and looks like a regression from 1.3. {code} import json

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603572#comment-14603572 ] Nicholas Chammas commented on SPARK-8670: - cc [~rxin], [~davies] Nested columns

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603658#comment-14603658 ] Nicholas Chammas commented on SPARK-8670: - I thought, per the discussion on

[jira] [Resolved] (SPARK-6220) Allow extended EC2 options to be passed through spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-6220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-6220. - Resolution: Won't Fix Resolving this issue as won't fix since it is of low importance and

[jira] [Updated] (SPARK-8576) Add spark-ec2 options to assign launched instances into IAM roles and to set instance-initiated shutdown behavior

[ https://issues.apache.org/jira/browse/SPARK-8576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-8576: Summary: Add spark-ec2 options to assign launched instances into IAM roles and to set

[jira] [Created] (SPARK-8576) Add spark-ec2 options to assigned launched instances into IAM roles and to set instance-initiated shutdown behavior

Nicholas Chammas created SPARK-8576: --- Summary: Add spark-ec2 options to assigned launched instances into IAM roles and to set instance-initiated shutdown behavior Key: SPARK-8576 URL:

[jira] [Commented] (SPARK-8417) spark-class has illegal statement

[ https://issues.apache.org/jira/browse/SPARK-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591988#comment-14591988 ] Nicholas Chammas commented on SPARK-8417: - I'm not sure what I'm looking at. Can

[jira] [Commented] (SPARK-8429) Add ability to set additional tags

[ https://issues.apache.org/jira/browse/SPARK-8429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14592010#comment-14592010 ] Nicholas Chammas commented on SPARK-8429: - What is your use case for this feature?

[jira] [Commented] (SPARK-6220) Allow extended EC2 options to be passed through spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-6220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14586334#comment-14586334 ] Nicholas Chammas commented on SPARK-6220: - please forgive my greenness No need.

[jira] [Created] (SPARK-8316) Upgrade Maven to 3.3.3

Nicholas Chammas created SPARK-8316: --- Summary: Upgrade Maven to 3.3.3 Key: SPARK-8316 URL: https://issues.apache.org/jira/browse/SPARK-8316 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-5398) Support the eu-central-1 region for spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-5398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573138#comment-14573138 ] Nicholas Chammas commented on SPARK-5398: - I don't have the credentials to do

[jira] [Commented] (SPARK-5398) Support the eu-central-1 region for spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-5398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573140#comment-14573140 ] Nicholas Chammas commented on SPARK-5398: - I don't have the credentials to do

[jira] [Issue Comment Deleted] (SPARK-5398) Support the eu-central-1 region for spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-5398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-5398: Comment: was deleted (was: I don't have the credentials to do that, unfortunately. Maybe

[jira] [Commented] (SPARK-7900) Reduce number of tagging calls in spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14571460#comment-14571460 ] Nicholas Chammas commented on SPARK-7900: - I'm marking this as a duplicate of

[jira] [Resolved] (SPARK-7900) Reduce number of tagging calls in spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-7900. - Resolution: Duplicate Reduce number of tagging calls in spark-ec2

[jira] [Commented] (SPARK-4983) Add sleep() before tagging EC2 instances to allow instance metadata to propagate

[ https://issues.apache.org/jira/browse/SPARK-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14571467#comment-14571467 ] Nicholas Chammas commented on SPARK-4983: - Per the discussion on [SPARK-7900], I

[jira] [Updated] (SPARK-5189) Reorganize EC2 scripts so that nodes can be provisioned independent of Spark master

[ https://issues.apache.org/jira/browse/SPARK-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-5189: Description: As of 1.2.0, we launch Spark clusters on EC2 by setting up the master first,

[jira] [Updated] (SPARK-5189) Reorganize EC2 scripts so that nodes can be provisioned independent of Spark master

[ https://issues.apache.org/jira/browse/SPARK-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-5189: Description: As of 1.2.0, we launch Spark clusters on EC2 by setting up the master first,

[jira] [Commented] (SPARK-7900) Reduce number of tagging calls in spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563208#comment-14563208 ] Nicholas Chammas commented on SPARK-7900: - The name tags are optional, but we can

[jira] [Commented] (SPARK-7900) Reduce number of tagging calls in spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563053#comment-14563053 ] Nicholas Chammas commented on SPARK-7900: - An alternative approach would be to

[jira] [Created] (SPARK-7900) Reduce number of tagging calls in spark-ec2

Nicholas Chammas created SPARK-7900: --- Summary: Reduce number of tagging calls in spark-ec2 Key: SPARK-7900 URL: https://issues.apache.org/jira/browse/SPARK-7900 Project: Spark Issue Type:

[jira] [Commented] (SPARK-7505) Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc.

[ https://issues.apache.org/jira/browse/SPARK-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14556860#comment-14556860 ] Nicholas Chammas commented on SPARK-7505: - cc [~davies] - I think the most

[jira] [Commented] (SPARK-7507) pyspark.sql.types.StructType and Row should implement __iter__()

[ https://issues.apache.org/jira/browse/SPARK-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555482#comment-14555482 ] Nicholas Chammas commented on SPARK-7507: - Since {{Row}} seems most analogous to a

[jira] [Commented] (SPARK-7507) pyspark.sql.types.StructType and Row should implement __iter__()

[ https://issues.apache.org/jira/browse/SPARK-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14554550#comment-14554550 ] Nicholas Chammas commented on SPARK-7507: - Related: A Stack Overflow question

[jira] [Commented] (SPARK-7640) Private VPC with default Spark AMI breaks yum

[ https://issues.apache.org/jira/browse/SPARK-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551795#comment-14551795 ] Nicholas Chammas commented on SPARK-7640: - [~brdwrd] - According to [this doc on

[jira] [Commented] (SPARK-7640) Private VPC with default Spark AMI breaks yum

[ https://issues.apache.org/jira/browse/SPARK-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544169#comment-14544169 ] Nicholas Chammas commented on SPARK-7640: - {quote} Switch everything to support

[jira] [Commented] (SPARK-7640) Private VPC with default Spark AMI breaks yum

[ https://issues.apache.org/jira/browse/SPARK-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544288#comment-14544288 ] Nicholas Chammas commented on SPARK-7640: - If there is no way around this (like,

[jira] [Comment Edited] (SPARK-7606) Document all PySpark SQL/DataFrame public methods with @since tag

[ https://issues.apache.org/jira/browse/SPARK-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542701#comment-14542701 ] Nicholas Chammas edited comment on SPARK-7606 at 5/13/15 8:57 PM:

[jira] [Commented] (SPARK-7606) Document all PySpark SQL/DataFrame public methods with @since tag

[ https://issues.apache.org/jira/browse/SPARK-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542701#comment-14542701 ] Nicholas Chammas commented on SPARK-7606: - Just looked into this. If we are using

[jira] [Created] (SPARK-7606) Document all PySpark SQL/DataFrame public methods with @since tag

Nicholas Chammas created SPARK-7606: --- Summary: Document all PySpark SQL/DataFrame public methods with @since tag Key: SPARK-7606 URL: https://issues.apache.org/jira/browse/SPARK-7606 Project: Spark

[jira] [Commented] (SPARK-7606) Document all PySpark SQL/DataFrame public methods with @since tag

[ https://issues.apache.org/jira/browse/SPARK-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542176#comment-14542176 ] Nicholas Chammas commented on SPARK-7606: - (I just cloned SPARK-7588.) Dunno what

[jira] [Commented] (SPARK-7507) pyspark.sql.types.StructType and Row should implement __iter__()

[ https://issues.apache.org/jira/browse/SPARK-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542175#comment-14542175 ] Nicholas Chammas commented on SPARK-7507: - cc [~davies] On a related note to our

[jira] [Updated] (SPARK-7505) Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc.

[ https://issues.apache.org/jira/browse/SPARK-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-7505: Description: The PySpark docs for DataFrame need the following fixes and improvements: #

[jira] [Commented] (SPARK-7548) Add explode expression

[ https://issues.apache.org/jira/browse/SPARK-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538937#comment-14538937 ] Nicholas Chammas commented on SPARK-7548: - To provide a motivating example for the

[jira] [Updated] (SPARK-7509) Add drop column to Python DataFrame API

[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-7509: Target Version/s: 1.4.0 I'm targeting this for 1.4.0, though that's optimistic given that

[jira] [Commented] (SPARK-7549) Support aggregating over nested fields

[ https://issues.apache.org/jira/browse/SPARK-7549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538965#comment-14538965 ] Nicholas Chammas commented on SPARK-7549: - To provide a motivating example for the

[jira] [Commented] (SPARK-7509) Add drop column to Python DataFrame API

[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538978#comment-14538978 ] Nicholas Chammas commented on SPARK-7509: - Oh, well nevermind then. :) Add drop

[jira] [Commented] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538832#comment-14538832 ] Nicholas Chammas commented on SPARK-7133: - [SPARK-7544} Implement struct, array,

[jira] [Comment Edited] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538832#comment-14538832 ] Nicholas Chammas edited comment on SPARK-7133 at 5/11/15 11:02 PM:

[jira] [Created] (SPARK-7544) pyspark.sql.types.Row should implement __getitem__

Nicholas Chammas created SPARK-7544: --- Summary: pyspark.sql.types.Row should implement __getitem__ Key: SPARK-7544 URL: https://issues.apache.org/jira/browse/SPARK-7544 Project: Spark Issue

[jira] [Commented] (SPARK-7544) pyspark.sql.types.Row should implement __getitem__

[ https://issues.apache.org/jira/browse/SPARK-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538830#comment-14538830 ] Nicholas Chammas commented on SPARK-7544: - cc [~rxin], [~davies]

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539255#comment-14539255 ] Nicholas Chammas commented on SPARK-2870: - Another use case for this feature is

[jira] [Commented] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538272#comment-14538272 ] Nicholas Chammas commented on SPARK-7133: - [~rxin] - Should we also implement

[jira] [Commented] (SPARK-7507) pyspark.sql.types.StructType and Row should implement __iter__()

[ https://issues.apache.org/jira/browse/SPARK-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538283#comment-14538283 ] Nicholas Chammas commented on SPARK-7507: - On a related note, perhaps we should

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537324#comment-14537324 ] Nicholas Chammas commented on SPARK-2870: - cc [~rxin] - Are we still targeting

[jira] [Commented] (SPARK-7280) Add a method for dropping a column in Java/Scala

[ https://issues.apache.org/jira/browse/SPARK-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536885#comment-14536885 ] Nicholas Chammas commented on SPARK-7280: - OK, good to know. Add a method for

<    1   2   3   4   5   6   7   8   9   10   >