[jira] [Created] (SPARK-12736) Standalone Master cannot be started due to NoClassDefFoundError: org/spark-project/guava/collect/Maps

2016-01-09 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-12736: --- Summary: Standalone Master cannot be started due to NoClassDefFoundError: org/spark-project/guava/collect/Maps Key: SPARK-12736 URL:

[jira] [Updated] (SPARK-12726) ParquetConversions doesn't always propagate metastore table identifier to ParquetRelation

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12726: -- Component/s: SQL > ParquetConversions doesn't always propagate metastore table identifier to >

[jira] [Updated] (SPARK-12729) phantom references to replace the finalize call in python broadcast

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12729: -- Component/s: (was: p) PySpark [~davies] if you would please just tag your issues

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-09 Thread Nikita Tarasenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090588#comment-15090588 ] Nikita Tarasenko commented on SPARK-12177: -- Hi, Mark! Of course, it possible to collaborate =)

[jira] [Resolved] (SPARK-5978) Spark, Examples have Hadoop1/2 compat issues with Hadoop 2.0.x (e.g. CDH4)

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5978. -- Resolution: Not A Problem No longer a problem at this stage given all these older Hadoop versions are

[jira] [Resolved] (SPARK-4565) Add docs about advanced spark application development

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4565. -- Resolution: Won't Fix No follow up > Add docs about advanced spark application development >

[jira] [Resolved] (SPARK-3368) Spark cannot be used with Avro and Parquet

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3368. -- Resolution: Cannot Reproduce I think this is long since obsolete > Spark cannot be used with Avro and

[jira] [Resolved] (SPARK-5940) Graph Loader: refactor + add more formats

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5940. -- Resolution: Won't Fix > Graph Loader: refactor + add more formats >

[jira] [Created] (SPARK-12737) Decrease the redundant activeIds sent to remote mirrors in "aggregateMessagesWithActiveSet"

2016-01-09 Thread qbwu (JIRA)
qbwu created SPARK-12737: Summary: Decrease the redundant activeIds sent to remote mirrors in "aggregateMessagesWithActiveSet" Key: SPARK-12737 URL: https://issues.apache.org/jira/browse/SPARK-12737 Project:

[jira] [Updated] (SPARK-12732) Fix LinearRegression.train for the case when label is constant and fitIntercept=false

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12732: -- Component/s: MLlib Sounds good [~iyounus], feel free to submit a PR. > Fix LinearRegression.train for

[jira] [Resolved] (SPARK-5260) Expose JsonRDD.allKeysWithValueTypes() in a utility class

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5260. -- Resolution: Won't Fix > Expose JsonRDD.allKeysWithValueTypes() in a utility class >

[jira] [Resolved] (SPARK-4566) Multiple --py-files command line options to spark-submit replace instead of adding to previous options

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4566. -- Resolution: Won't Fix No follow up > Multiple --py-files command line options to spark-submit replace

[jira] [Resolved] (SPARK-6215) Shorten apply and update funcs in GenerateProjection

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6215. -- Resolution: Won't Fix > Shorten apply and update funcs in GenerateProjection >

[jira] [Resolved] (SPARK-6221) SparkSQL should support auto merging output files

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6221. -- Resolution: Won't Fix > SparkSQL should support auto merging output files >

[jira] [Resolved] (SPARK-6277) Allow Hadoop configurations and env variables to be referenced in spark-defaults.conf

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6277. -- Resolution: Won't Fix > Allow Hadoop configurations and env variables to be referenced in >

[jira] [Resolved] (SPARK-4496) smallint (16 bit value) is being send as a 32 bit value in the thrift interface.

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4496. -- Resolution: Invalid > smallint (16 bit value) is being send as a 32 bit value in the thrift >

[jira] [Resolved] (SPARK-3920) Add option to support aggregation using treeAggregate in decision tree

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3920. -- Resolution: Won't Fix > Add option to support aggregation using treeAggregate in decision tree >

[jira] [Updated] (SPARK-5865) Add doc warnings for methods that return local data structures

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5865: - Labels: starter (was: ) > Add doc warnings for methods that return local data structures >

[jira] [Resolved] (SPARK-3113) Using local spark-submit with an EC2 cluster fails to execute job

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3113. -- Resolution: Cannot Reproduce I'm guessing this is CannotReproduce given the lack of activity, and that

[jira] [Resolved] (SPARK-4241) spark_ec2.py support China AWS region: cn-north-1

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4241. -- Resolution: Won't Fix I'm guessing this is WontFix given the lack of activity, and that EC2 support is

[jira] [Resolved] (SPARK-1555) enable ec2/spark_ec2.py to stop/delete cluster non-interactively

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1555. -- Resolution: Won't Fix I'm guessing this is WontFix given the lack of activity, and that EC2 support is

[jira] [Commented] (SPARK-5865) Add doc warnings for methods that return local data structures

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090599#comment-15090599 ] Sean Owen commented on SPARK-5865: -- OK by me; seems like an old but valid starter JIRA > Add doc

[jira] [Assigned] (SPARK-5273) Improve documentation examples for LinearRegression

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5273: --- Assignee: (was: Apache Spark) > Improve documentation examples for LinearRegression >

[jira] [Assigned] (SPARK-5273) Improve documentation examples for LinearRegression

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5273: --- Assignee: Apache Spark > Improve documentation examples for LinearRegression >

[jira] [Commented] (SPARK-5273) Improve documentation examples for LinearRegression

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090612#comment-15090612 ] Apache Spark commented on SPARK-5273: - User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-12736) Standalone Master cannot be started due to NoClassDefFoundError: org/spark-project/guava/collect/Maps

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090570#comment-15090570 ] Apache Spark commented on SPARK-12736: -- User 'jaceklaskowski' has created a pull request for this

[jira] [Resolved] (SPARK-12712) test-dependencies.sh fails with difference in manifests

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12712. --- Resolution: Not A Problem If it's not a problem with the Spark build per se, I'm not sure this is

[jira] [Resolved] (SPARK-5793) Add explode to Column

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5793. -- Resolution: Won't Fix > Add explode to Column > - > > Key:

[jira] [Resolved] (SPARK-5766) Slow RowMatrix multiplication

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5766. -- Resolution: Not A Problem Or to some degree this is already done via netlib > Slow RowMatrix

[jira] [Resolved] (SPARK-6162) Handle missing values in GBM

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6162. -- Resolution: Won't Fix > Handle missing values in GBM > > >

[jira] [Resolved] (SPARK-1593) Add status command to Spark Daemons(master/worker)

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1593. -- Resolution: Won't Fix > Add status command to Spark Daemons(master/worker) >

[jira] [Resolved] (SPARK-1881) Executor caching

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1881. -- Resolution: Not A Problem > Executor caching > > > Key: SPARK-1881 >

[jira] [Resolved] (SPARK-5916) $SPARK_HOME/bin/beeline conflicts with $HIVE_HOME/bin/beeline

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5916. -- Resolution: Won't Fix > $SPARK_HOME/bin/beeline conflicts with $HIVE_HOME/bin/beeline >

[jira] [Commented] (SPARK-12737) Decrease the redundant activeIds sent to remote mirrors in "aggregateMessagesWithActiveSet"

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090627#comment-15090627 ] Apache Spark commented on SPARK-12737: -- User 'qbwu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12737) Decrease the redundant activeIds sent to remote mirrors in "aggregateMessagesWithActiveSet"

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12737: Assignee: Apache Spark > Decrease the redundant activeIds sent to remote mirrors in >

[jira] [Assigned] (SPARK-12737) Decrease the redundant activeIds sent to remote mirrors in "aggregateMessagesWithActiveSet"

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12737: Assignee: (was: Apache Spark) > Decrease the redundant activeIds sent to remote

[jira] [Updated] (SPARK-12729) phantom references to replace the finalize call in python broadcast

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12729: -- Component/s: p > phantom references to replace the finalize call in python broadcast >

[jira] [Resolved] (SPARK-4863) Suspicious exception handlers

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4863. -- Resolution: Not A Problem No follow up > Suspicious exception handlers > -

[jira] [Resolved] (SPARK-5005) Failed to start spark-shell when using yarn-client mode with the Spark1.2.0

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5005. -- Resolution: Cannot Reproduce > Failed to start spark-shell when using yarn-client mode with the

[jira] [Resolved] (SPARK-4510) Add k-medoids Partitioning Around Medoids (PAM) algorithm

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4510. -- Resolution: Won't Fix > Add k-medoids Partitioning Around Medoids (PAM) algorithm >

[jira] [Resolved] (SPARK-5280) Import RDF graphs into GraphX

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5280. -- Resolution: Won't Fix > Import RDF graphs into GraphX > - > >

[jira] [Resolved] (SPARK-5455) Add MultipleTransformer abstract class

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5455. -- Resolution: Won't Fix > Add MultipleTransformer abstract class > --

[jira] [Resolved] (SPARK-2385) Missing guide for running JDBC server on YARN

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2385. -- Resolution: Won't Fix > Missing guide for running JDBC server on YARN >

[jira] [Assigned] (SPARK-12736) Standalone Master cannot be started due to NoClassDefFoundError: org/spark-project/guava/collect/Maps

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12736: Assignee: Apache Spark > Standalone Master cannot be started due to NoClassDefFoundError:

[jira] [Assigned] (SPARK-12736) Standalone Master cannot be started due to NoClassDefFoundError: org/spark-project/guava/collect/Maps

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12736: Assignee: (was: Apache Spark) > Standalone Master cannot be started due to

[jira] [Resolved] (SPARK-5828) Dynamic partition pattern support

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5828. -- Resolution: Won't Fix > Dynamic partition pattern support > - > >

[jira] [Resolved] (SPARK-5627) Enhance spark-ec2 to return machine-readable output

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5627. -- Resolution: Won't Fix I'm guessing this is WontFix given the lack of activity, and that EC2 support is

[jira] [Resolved] (SPARK-5745) Allow to use custom TaskMetrics implementation

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5745. -- Resolution: Won't Fix > Allow to use custom TaskMetrics implementation >

[jira] [Resolved] (SPARK-5792) hive udfs like "get_json_object and json_tuple" doesnot work in spark 1.2.0

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5792. -- Resolution: Won't Fix I think this is obsolete given it's traced to Jackson version conflict > hive

[jira] [Resolved] (SPARK-3522) Make spark-ec2 verbosity configurable

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3522. -- Resolution: Won't Fix I'm guessing this is WontFix given the lack of activity, and that EC2 support is

[jira] [Resolved] (SPARK-3901) Add SocketSink capability for Spark metrics

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3901. -- Resolution: Won't Fix > Add SocketSink capability for Spark metrics >

[jira] [Resolved] (SPARK-2930) clarify docs on using webhdfs with spark.yarn.access.namenodes

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2930. -- Resolution: Won't Fix I haven't seen any follow up or demand for this > clarify docs on using webhdfs

[jira] [Resolved] (SPARK-5140) Two RDDs which are scheduled concurrently should be able to wait on parent in all cases

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5140. -- Resolution: Won't Fix > Two RDDs which are scheduled concurrently should be able to wait on parent in

[jira] [Resolved] (SPARK-5710) Combines two adjacent `Cast` expressions into one

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5710. -- Resolution: Won't Fix > Combines two adjacent `Cast` expressions into one >

[jira] [Assigned] (SPARK-12706) support grouping/grouping_id function together group set

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12706: Assignee: Apache Spark (was: Davies Liu) > support grouping/grouping_id function

[jira] [Commented] (SPARK-12706) support grouping/grouping_id function together group set

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090804#comment-15090804 ] Apache Spark commented on SPARK-12706: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12706) support grouping/grouping_id function together group set

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12706: Assignee: Davies Liu (was: Apache Spark) > support grouping/grouping_id function

[jira] [Created] (SPARK-12740) grouping()/grouping_id() should work with having and order by

2016-01-09 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12740: -- Summary: grouping()/grouping_id() should work with having and order by Key: SPARK-12740 URL: https://issues.apache.org/jira/browse/SPARK-12740 Project: Spark

[jira] [Updated] (SPARK-12738) GROUPING__ID is wrong

2016-01-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12738: --- Description: For group set, GROUPING__ID should be 1 if the column is aggregated, or 0. The current

[jira] [Updated] (SPARK-12739) Details of batch in Streaming tab uses two Duration columns

2016-01-09 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-12739: Attachment: SPARK-12739.png > Details of batch in Streaming tab uses two Duration columns

[jira] [Created] (SPARK-12738) GROUPING__ID is wrong

2016-01-09 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12738: -- Summary: GROUPING__ID is wrong Key: SPARK-12738 URL: https://issues.apache.org/jira/browse/SPARK-12738 Project: Spark Issue Type: Bug Components: SQL

[jira] [Created] (SPARK-12739) Details of batch in Streaming tab uses two Duration columns

2016-01-09 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-12739: --- Summary: Details of batch in Streaming tab uses two Duration columns Key: SPARK-12739 URL: https://issues.apache.org/jira/browse/SPARK-12739 Project: Spark

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-09 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090717#comment-15090717 ] Mark Grover commented on SPARK-12177: - #1 Sounds great, thanks! #2 Yeah, that's the only way I can

[jira] [Updated] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-01-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12705: Description: The following query can't be resolved: {code} scala> sqlContext.sql("select sum(a)

[jira] [Commented] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-01-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090860#comment-15090860 ] Xiao Li commented on SPARK-12705: - Will submit a PR soon. Thanks! > Sorting column can't be resolved if

[jira] [Assigned] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12705: Assignee: Apache Spark > Sorting column can't be resolved if it's not in projection >

[jira] [Commented] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090877#comment-15090877 ] Apache Spark commented on SPARK-12705: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12705: Assignee: (was: Apache Spark) > Sorting column can't be resolved if it's not in

[jira] [Resolved] (SPARK-12735) Move spark-ec2 scripts to AMPLab

2016-01-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12735. - Resolution: Fixed Fix Version/s: 2.0.0 > Move spark-ec2 scripts to AMPLab >

[jira] [Updated] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Liang updated SPARK-12691: Description: Multiple unionAll on Dataframe seems to cause repeated calculations. Here is the

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:22 AM: -- Hi Bo Meng,

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:25 AM: -- Hi Bo Meng,

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:23 AM: -- Hi Bo Meng,

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:24 AM: -- Hi Bo Meng,

[jira] [Commented] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang commented on SPARK-12691: - Hi Bo Meng, I understand you point, but why size of dataframe

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:28 AM: -- Hi Bo Meng,

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:27 AM: -- Hi Bo Meng,

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:29 AM: -- Hi Bo Meng,

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:30 AM: -- Hi Bo Meng,

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:34 AM: -- Hi Bo Meng,

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:37 AM: -- Hi Bo Meng,

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:37 AM: -- Hi Bo Meng,

[jira] [Issue Comment Deleted] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Liang updated SPARK-12691: Comment: was deleted (was: Hi Bo Meng, I understand your point, but I don't think this has

[jira] [Commented] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090906#comment-15090906 ] Allen Liang commented on SPARK-12691: - Hi Bo Meng, I understand your point, but I don't think this

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:46 AM: -- Hi Bo Meng,

[jira] [Comment Edited] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090896#comment-15090896 ] Allen Liang edited comment on SPARK-12691 at 1/10/16 5:47 AM: -- Hi Bo Meng,

[jira] [Issue Comment Deleted] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Liang updated SPARK-12691: Comment: was deleted (was: Hi Bo Meng, I understand your point, but I don't think this has

[jira] [Commented] (SPARK-12691) Multiple unionAll on Dataframe seems to cause repeated calculations in a "Fibonacci" manner

2016-01-09 Thread Allen Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090908#comment-15090908 ] Allen Liang commented on SPARK-12691: - Hi Bo Meng, I understand your point, but I don't think this

[jira] [Commented] (SPARK-12612) Add missing Hadoop profiles to dev/run-tests-*.py scripts

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090913#comment-15090913 ] Apache Spark commented on SPARK-12612: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Commented] (SPARK-10359) Enumerate Spark's dependencies in a file and diff against it for new pull requests

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090917#comment-15090917 ] Apache Spark commented on SPARK-10359: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Commented] (SPARK-10359) Enumerate Spark's dependencies in a file and diff against it for new pull requests

2016-01-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090922#comment-15090922 ] Apache Spark commented on SPARK-10359: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Commented] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-01-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090493#comment-15090493 ] Xiao Li commented on SPARK-12705: - Will try to fix it this weekend. Thanks! > Sorting column can't be

[jira] [Commented] (SPARK-12731) PySpark docstring cleanup

2016-01-09 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090507#comment-15090507 ] holdenk commented on SPARK-12731: - cc [~josephkb] based on our chat on my other PR > PySpark docstring

[jira] [Commented] (SPARK-12426) Docker JDBC integration tests are failing again

2016-01-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090534#comment-15090534 ] Sean Owen commented on SPARK-12426: --- I'm not sure how to give edit access -- don't think I'm an admin

[jira] [Commented] (SPARK-12711) ML StopWordsRemover does not protect itself from column name duplication

2016-01-09 Thread Wojciech Jurczyk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090535#comment-15090535 ] Wojciech Jurczyk commented on SPARK-12711: -- [~josephkb]Is there any particular reason why

[jira] [Assigned] (SPARK-12543) Support subquery in select/where/having

2016-01-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12543: -- Assignee: Davies Liu > Support subquery in select/where/having >