[jira] [Updated] (SPARK-6575) Add configuration to disable schema merging while converting metastore Parquet tables

2015-03-30 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6575: Priority: Blocker (was: Major) > Add configuration to disable schema merging while converting metastore >

[jira] [Commented] (SPARK-5564) Support sparse LDA solutions

2015-03-30 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388128#comment-14388128 ] Debasish Das commented on SPARK-5564: - [~sparks] we are trying to access the EC2 datas

[jira] [Assigned] (SPARK-4550) In sort-based shuffle, store map outputs in serialized form

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4550: --- Assignee: Apache Spark (was: Sandy Ryza) > In sort-based shuffle, store map outputs in seria

[jira] [Assigned] (SPARK-4550) In sort-based shuffle, store map outputs in serialized form

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4550: --- Assignee: Sandy Ryza (was: Apache Spark) > In sort-based shuffle, store map outputs in seria

[jira] [Commented] (SPARK-6627) Clean up of shuffle code and interfaces

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388107#comment-14388107 ] Apache Spark commented on SPARK-6627: - User 'pwendell' has created a pull request for

[jira] [Assigned] (SPARK-6627) Clean up of shuffle code and interfaces

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6627: --- Assignee: Apache Spark (was: Patrick Wendell) > Clean up of shuffle code and interfaces > --

[jira] [Assigned] (SPARK-6627) Clean up of shuffle code and interfaces

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6627: --- Assignee: Patrick Wendell (was: Apache Spark) > Clean up of shuffle code and interfaces > --

[jira] [Created] (SPARK-6627) Clean up of shuffle code and interfaces

2015-03-30 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-6627: -- Summary: Clean up of shuffle code and interfaces Key: SPARK-6627 URL: https://issues.apache.org/jira/browse/SPARK-6627 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-4514) SparkContext localProperties does not inherit property updates across thread reuse

2015-03-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388095#comment-14388095 ] Josh Rosen commented on SPARK-4514: --- I don't know that there's a good way to fix this fo

[jira] [Updated] (SPARK-6625) Add common string filters to data sources

2015-03-30 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6625: Target Version/s: 1.3.1 > Add common string filters to data sources > --

[jira] [Created] (SPARK-6626) TwitterUtils.createStream documentation error

2015-03-30 Thread Jayson Sunshine (JIRA)
Jayson Sunshine created SPARK-6626: -- Summary: TwitterUtils.createStream documentation error Key: SPARK-6626 URL: https://issues.apache.org/jira/browse/SPARK-6626 Project: Spark Issue Type: D

[jira] [Updated] (SPARK-6625) Add common string filters to data sources

2015-03-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6625: --- Assignee: Reynold Xin > Add common string filters to data sources > --

[jira] [Updated] (SPARK-6625) Add common string filters to data sources

2015-03-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6625: --- Description: Filters such as startsWith, endsWith, contains will be very useful for data sources that

[jira] [Assigned] (SPARK-6625) Add common string filters to data sources

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6625: --- Assignee: (was: Apache Spark) > Add common string filters to data sources > -

[jira] [Commented] (SPARK-6625) Add common string filters to data sources

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388039#comment-14388039 ] Apache Spark commented on SPARK-6625: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-6625) Add common string filters to data sources

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6625: --- Assignee: Apache Spark > Add common string filters to data sources >

[jira] [Updated] (SPARK-6625) Add common string filters to data sources

2015-03-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6625: --- Description: Filters such as StartsWith, EndsWith, Contains will be very useful for search-like data

[jira] [Assigned] (SPARK-6623) Alias DataFrame.na.fill/drop in Python

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6623: --- Assignee: (was: Apache Spark) > Alias DataFrame.na.fill/drop in Python >

[jira] [Commented] (SPARK-6623) Alias DataFrame.na.fill/drop in Python

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388026#comment-14388026 ] Apache Spark commented on SPARK-6623: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-6623) Alias DataFrame.na.fill/drop in Python

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6623: --- Assignee: Apache Spark > Alias DataFrame.na.fill/drop in Python > ---

[jira] [Commented] (SPARK-6258) Python MLlib API missing items: Clustering

2015-03-30 Thread Hrishikesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388003#comment-14388003 ] Hrishikesh commented on SPARK-6258: --- [~josephkb] Thank you for your response and valuabl

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388002#comment-14388002 ] Apache Spark commented on SPARK-5124: - User 'zsxwing' has created a pull request for t

[jira] [Commented] (SPARK-6612) Python KMeans parity

2015-03-30 Thread Hrishikesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388000#comment-14388000 ] Hrishikesh commented on SPARK-6612: --- Please assign this ticket to me. > Python KMeans p

[jira] [Assigned] (SPARK-3454) Expose JSON representation of data shown in WebUI

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3454: --- Assignee: Imran Rashid (was: Apache Spark) > Expose JSON representation of data shown in Web

[jira] [Assigned] (SPARK-3454) Expose JSON representation of data shown in WebUI

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3454: --- Assignee: Apache Spark (was: Imran Rashid) > Expose JSON representation of data shown in Web

[jira] [Created] (SPARK-6625) Add common string filters to data sources

2015-03-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-6625: -- Summary: Add common string filters to data sources Key: SPARK-6625 URL: https://issues.apache.org/jira/browse/SPARK-6625 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-6624) Convert filters into CNF for data sources

2015-03-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-6624: -- Summary: Convert filters into CNF for data sources Key: SPARK-6624 URL: https://issues.apache.org/jira/browse/SPARK-6624 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-6623) Alias DataFrame.na.fill/drop in Python

2015-03-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-6623: -- Summary: Alias DataFrame.na.fill/drop in Python Key: SPARK-6623 URL: https://issues.apache.org/jira/browse/SPARK-6623 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-6622) Spark SQL cannot communicate with Hive meta store

2015-03-30 Thread Deepak Kumar V (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Kumar V updated SPARK-6622: -- Description: I have multiple tables (among them is dw_bid) that are created through Apache Hive

[jira] [Updated] (SPARK-6603) SQLContext.registerFunction -> SQLContext.udf.register

2015-03-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6603: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-6116 > SQLContext.registerFunction -> S

[jira] [Updated] (SPARK-6622) Spark SQL cannot communicate with Hive meta store

2015-03-30 Thread Deepak Kumar V (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Kumar V updated SPARK-6622: -- Attachment: exception.txt Full stack trace > Spark SQL cannot communicate with Hive meta store

[jira] [Created] (SPARK-6622) Spark SQL cannot communicate with Hive meta store

2015-03-30 Thread Deepak Kumar V (JIRA)
Deepak Kumar V created SPARK-6622: - Summary: Spark SQL cannot communicate with Hive meta store Key: SPARK-6622 URL: https://issues.apache.org/jira/browse/SPARK-6622 Project: Spark Issue Type:

[jira] [Commented] (SPARK-6562) DataFrame.na.replace value support

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387933#comment-14387933 ] Apache Spark commented on SPARK-6562: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-6562) DataFrame.na.replace value support

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6562: --- Assignee: (was: Apache Spark) > DataFrame.na.replace value support >

[jira] [Assigned] (SPARK-6562) DataFrame.na.replace value support

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6562: --- Assignee: Apache Spark > DataFrame.na.replace value support > ---

[jira] [Updated] (SPARK-6562) DataFrame.na.replace value support

2015-03-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6562: --- Summary: DataFrame.na.replace value support (was: DataFrame.replace value support) > DataFrame.na.re

[jira] [Assigned] (SPARK-6618) HiveMetastoreCatalog.lookupRelation should use fine-grained lock

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6618: --- Assignee: Apache Spark (was: Yin Huai) > HiveMetastoreCatalog.lookupRelation should use fine

[jira] [Assigned] (SPARK-6618) HiveMetastoreCatalog.lookupRelation should use fine-grained lock

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6618: --- Assignee: Yin Huai (was: Apache Spark) > HiveMetastoreCatalog.lookupRelation should use fine

[jira] [Commented] (SPARK-6618) HiveMetastoreCatalog.lookupRelation should use fine-grained lock

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387924#comment-14387924 ] Apache Spark commented on SPARK-6618: - User 'yhuai' has created a pull request for thi

[jira] [Assigned] (SPARK-6555) Override equals and hashCode in MetastoreRelation

2015-03-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-6555: - Assignee: Cheng Lian > Override equals and hashCode in MetastoreRelation > --

[jira] [Commented] (SPARK-6573) expect pandas null values as numpy.nan (not only as None)

2015-03-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387921#comment-14387921 ] Reynold Xin commented on SPARK-6573: Are numpy.nan turned into Double.NaN in the JVM?

[jira] [Resolved] (SPARK-6119) DataFrame.dropna support

2015-03-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6119. Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Assignee: Reynold Xin

[jira] [Resolved] (SPARK-6563) DataFrame.fillna

2015-03-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6563. Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Assignee: Reynold Xin

[jira] [Assigned] (SPARK-6621) Calling EventLoop.stop in EventLoop.onReceive and EventLoop.onError should call onStop

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6621: --- Assignee: Apache Spark > Calling EventLoop.stop in EventLoop.onReceive and EventLoop.onError

[jira] [Commented] (SPARK-6621) Calling EventLoop.stop in EventLoop.onReceive and EventLoop.onError should call onStop

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387911#comment-14387911 ] Apache Spark commented on SPARK-6621: - User 'zsxwing' has created a pull request for t

[jira] [Assigned] (SPARK-6621) Calling EventLoop.stop in EventLoop.onReceive and EventLoop.onError should call onStop

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6621: --- Assignee: (was: Apache Spark) > Calling EventLoop.stop in EventLoop.onReceive and EventLo

[jira] [Commented] (SPARK-5456) Decimal Type comparison issue

2015-03-30 Thread Kuldeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387904#comment-14387904 ] Kuldeep commented on SPARK-5456: [~karthikg01] 1) Switch to hive context, I am not trying

[jira] [Created] (SPARK-6621) Calling EventLoop.stop in EventLoop.onReceive and EventLoop.onError should call onStop

2015-03-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-6621: --- Summary: Calling EventLoop.stop in EventLoop.onReceive and EventLoop.onError should call onStop Key: SPARK-6621 URL: https://issues.apache.org/jira/browse/SPARK-6621 Pr

[jira] [Commented] (SPARK-6620) Speed up toDF() and rdd() functions by constructing converters in ScalaReflection

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387898#comment-14387898 ] Apache Spark commented on SPARK-6620: - User 'vlyubin' has created a pull request for t

[jira] [Assigned] (SPARK-6620) Speed up toDF() and rdd() functions by constructing converters in ScalaReflection

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6620: --- Assignee: (was: Apache Spark) > Speed up toDF() and rdd() functions by constructing conve

[jira] [Assigned] (SPARK-6620) Speed up toDF() and rdd() functions by constructing converters in ScalaReflection

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6620: --- Assignee: Apache Spark > Speed up toDF() and rdd() functions by constructing converters in >

[jira] [Created] (SPARK-6620) Speed up toDF() and rdd() functions by constructing converters in ScalaReflection

2015-03-30 Thread Volodymyr Lyubinets (JIRA)
Volodymyr Lyubinets created SPARK-6620: -- Summary: Speed up toDF() and rdd() functions by constructing converters in ScalaReflection Key: SPARK-6620 URL: https://issues.apache.org/jira/browse/SPARK-6620

[jira] [Closed] (SPARK-6606) Accumulator deserialized twice because the NarrowCoGroupSplitDep contains rdd object.

2015-03-30 Thread SuYan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SuYan closed SPARK-6606. Resolution: Duplicate Duplicate with SPARK-5360, see https://github.com/apache/spark/pull/4145 > Accumulator deseri

[jira] [Commented] (SPARK-5371) Failure to analyze query with UNION ALL and double aggregation

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387847#comment-14387847 ] Apache Spark commented on SPARK-5371: - User 'marmbrus' has created a pull request for

[jira] [Assigned] (SPARK-5371) Failure to analyze query with UNION ALL and double aggregation

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5371: --- Assignee: Michael Armbrust (was: Apache Spark) > Failure to analyze query with UNION ALL and

[jira] [Assigned] (SPARK-5371) Failure to analyze query with UNION ALL and double aggregation

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5371: --- Assignee: Apache Spark (was: Michael Armbrust) > Failure to analyze query with UNION ALL and

[jira] [Updated] (SPARK-5371) Failure to analyze query with UNION ALL and double aggregation

2015-03-30 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5371: Summary: Failure to analyze query with UNION ALL and double aggregation (was: SparkSQL Fail

[jira] [Assigned] (SPARK-5371) SparkSQL Fails to parse Query with UNION ALL in subquery

2015-03-30 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-5371: --- Assignee: Michael Armbrust > SparkSQL Fails to parse Query with UNION ALL in subquery

[jira] [Updated] (SPARK-5371) SparkSQL Fails to parse Query with UNION ALL in subquery

2015-03-30 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5371: Priority: Critical (was: Major) Target Version/s: 1.3.1 Affects Version/s:

[jira] [Updated] (SPARK-5371) SparkSQL Fails to analyze Query with UNION ALL in subquery

2015-03-30 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5371: Summary: SparkSQL Fails to analyze Query with UNION ALL in subquery (was: SparkSQL Fails to

[jira] [Resolved] (SPARK-6605) Same transformation in DStream leads to different result

2015-03-30 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus resolved SPARK-6605. - Resolution: Won't Fix {{reduceByKeyAndWindow }} has two implementations and leads to two different

[jira] [Comment Edited] (SPARK-6605) Same transformation in DStream leads to different result

2015-03-30 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387811#comment-14387811 ] SaintBacchus edited comment on SPARK-6605 at 3/31/15 1:54 AM: --

[jira] [Commented] (SPARK-6619) Improve Jar caching on executors

2015-03-30 Thread Mingyu Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387783#comment-14387783 ] Mingyu Kim commented on SPARK-6619: --- [~li-zhihui], [~joshrosen] since you worked on SPAR

[jira] [Updated] (SPARK-6619) Improve Jar caching on executors

2015-03-30 Thread Mingyu Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingyu Kim updated SPARK-6619: -- Description: Taking SPARK-2713 one step further so that - The cached jars can be used by multiple applic

[jira] [Updated] (SPARK-6619) Jar cache on Executors should use file content hash as the key

2015-03-30 Thread Mingyu Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingyu Kim updated SPARK-6619: -- Description: Taking SPARK-2713 one step further so that the cached jars can be used by multiple applica

[jira] [Created] (SPARK-6619) Jar cache on Executors should use file content hash as the key

2015-03-30 Thread Mingyu Kim (JIRA)
Mingyu Kim created SPARK-6619: - Summary: Jar cache on Executors should use file content hash as the key Key: SPARK-6619 URL: https://issues.apache.org/jira/browse/SPARK-6619 Project: Spark Issue

[jira] [Comment Edited] (SPARK-6239) Spark MLlib fpm#FPGrowth minSupport should use long instead

2015-03-30 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387748#comment-14387748 ] Littlestar edited comment on SPARK-6239 at 3/31/15 1:09 AM: >>

[jira] [Commented] (SPARK-6239) Spark MLlib fpm#FPGrowth minSupport should use long instead

2015-03-30 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387748#comment-14387748 ] Littlestar commented on SPARK-6239: --- >>If I want to set minCount=2, I must use.setMinSup

[jira] [Commented] (SPARK-6618) HiveMetastoreCatalog.lookupRelation should use fine-grained lock

2015-03-30 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387738#comment-14387738 ] Yin Huai commented on SPARK-6618: - cc [~marmbrus] and [~lian cheng]. I am going to addres

[jira] [Updated] (SPARK-6618) HiveMetastoreCatalog.lookupRelation should use fine-grained lock

2015-03-30 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6618: Target Version/s: 1.3.1 (was: 1.3.0) > HiveMetastoreCatalog.lookupRelation should use fine-grained lock > -

[jira] [Created] (SPARK-6618) HiveMetastoreCatalog.lookupRelation should use fine-grained lock

2015-03-30 Thread Yin Huai (JIRA)
Yin Huai created SPARK-6618: --- Summary: HiveMetastoreCatalog.lookupRelation should use fine-grained lock Key: SPARK-6618 URL: https://issues.apache.org/jira/browse/SPARK-6618 Project: Spark Issue T

[jira] [Updated] (SPARK-6617) Word2Vec is nondeterministic

2015-03-30 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6617: - Summary: Word2Vec is nondeterministic (was: Word2Vec is not deterministic) > Word2Vec is nondeter

[jira] [Created] (SPARK-6617) Word2Vec is not deterministic

2015-03-30 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-6617: Summary: Word2Vec is not deterministic Key: SPARK-6617 URL: https://issues.apache.org/jira/browse/SPARK-6617 Project: Spark Issue Type: Bug Compone

[jira] [Resolved] (SPARK-6369) InsertIntoHiveTable and Parquet Relation should use logic from SparkHadoopWriter

2015-03-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6369. --- Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Issue resolved by pull request

[jira] [Updated] (SPARK-6616) IsStopped set to true in before stop() is complete.

2015-03-30 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Ganelin updated SPARK-6616: Description: There are numerous instances throughout the code base of the following: {code} if (!st

[jira] [Updated] (SPARK-6616) IsStopped set to true in before stop() is complete.

2015-03-30 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Ganelin updated SPARK-6616: Description: There are numerous instances throughout the code base of the following: {code} if (!st

[jira] [Updated] (SPARK-6616) IsStopped set to true in before stop() is complete.

2015-03-30 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Ganelin updated SPARK-6616: Description: There are numerous instances throughout the code base of the following: {code} if (!st

[jira] [Updated] (SPARK-6616) IsStopped set to true in before stop() is complete.

2015-03-30 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Ganelin updated SPARK-6616: Description: There are numerous instances throughout the code base of the following: {{code}} if (!

[jira] [Updated] (SPARK-6616) IsStopped set to true in before stop() is complete.

2015-03-30 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Ganelin updated SPARK-6616: Description: There are numerous instances throughout the code base of the following: {{code}} if (!

[jira] [Updated] (SPARK-6616) IsStopped set to true in before stop() is complete.

2015-03-30 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Ganelin updated SPARK-6616: Description: There are numerous instances throughout the code base of the following: {{code}} if (!

[jira] [Created] (SPARK-6616) IsStopped set to true in before stop() is complete.

2015-03-30 Thread Ilya Ganelin (JIRA)
Ilya Ganelin created SPARK-6616: --- Summary: IsStopped set to true in before stop() is complete. Key: SPARK-6616 URL: https://issues.apache.org/jira/browse/SPARK-6616 Project: Spark Issue Type: I

[jira] [Created] (SPARK-6615) Python API for Word2Vec

2015-03-30 Thread Kai Sasaki (JIRA)
Kai Sasaki created SPARK-6615: - Summary: Python API for Word2Vec Key: SPARK-6615 URL: https://issues.apache.org/jira/browse/SPARK-6615 Project: Spark Issue Type: Task Components: MLlib,

[jira] [Commented] (SPARK-6492) SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop dies

2015-03-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387568#comment-14387568 ] Josh Rosen commented on SPARK-6492: --- Timeouts are one way to fix this, but I wonder if w

[jira] [Commented] (SPARK-5886) Add LabelIndexer

2015-03-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387567#comment-14387567 ] Joseph K. Bradley commented on SPARK-5886: -- Also, should this index native types

[jira] [Assigned] (SPARK-6492) SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop dies

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6492: --- Assignee: (was: Apache Spark) > SparkContext.stop() can deadlock when DAGSchedulerEventPr

[jira] [Assigned] (SPARK-5205) Inconsistent behaviour between Streaming job and others, when click kill link in WebUI

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5205: --- Assignee: Apache Spark > Inconsistent behaviour between Streaming job and others, when click

[jira] [Assigned] (SPARK-5205) Inconsistent behaviour between Streaming job and others, when click kill link in WebUI

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5205: --- Assignee: (was: Apache Spark) > Inconsistent behaviour between Streaming job and others,

[jira] [Assigned] (SPARK-6492) SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop dies

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6492: --- Assignee: Apache Spark > SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop d

[jira] [Commented] (SPARK-6492) SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop dies

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387558#comment-14387558 ] Apache Spark commented on SPARK-6492: - User 'ilganeli' has created a pull request for

[jira] [Commented] (SPARK-5886) Add LabelIndexer

2015-03-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387554#comment-14387554 ] Joseph K. Bradley commented on SPARK-5886: -- Was there any discussion about this i

[jira] [Resolved] (SPARK-6603) SQLContext.registerFunction -> SQLContext.udf.register

2015-03-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6603. Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 > SQLContext.registerFunction

[jira] [Commented] (SPARK-6251) Mark parts of LBFGS, GradientDescent as DeveloperApi

2015-03-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387541#comment-14387541 ] Joseph K. Bradley commented on SPARK-6251: -- I'm closing this since we need to rev

[jira] [Closed] (SPARK-6251) Mark parts of LBFGS, GradientDescent as DeveloperApi

2015-03-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-6251. Resolution: Won't Fix > Mark parts of LBFGS, GradientDescent as DeveloperApi > -

[jira] [Assigned] (SPARK-6251) Mark parts of LBFGS, GradientDescent as DeveloperApi

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6251: --- Assignee: Joseph K. Bradley (was: Apache Spark) > Mark parts of LBFGS, GradientDescent as De

[jira] [Assigned] (SPARK-6251) Mark parts of LBFGS, GradientDescent as DeveloperApi

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6251: --- Assignee: Apache Spark (was: Joseph K. Bradley) > Mark parts of LBFGS, GradientDescent as De

[jira] [Assigned] (SPARK-6614) OutputCommitCoordinator should clear authorized committers only after authorized committer fails, not after any failure

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6614: --- Assignee: Apache Spark (was: Josh Rosen) > OutputCommitCoordinator should clear authorized c

[jira] [Assigned] (SPARK-6614) OutputCommitCoordinator should clear authorized committers only after authorized committer fails, not after any failure

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6614: --- Assignee: Josh Rosen (was: Apache Spark) > OutputCommitCoordinator should clear authorized c

[jira] [Commented] (SPARK-6614) OutputCommitCoordinator should clear authorized committers only after authorized committer fails, not after any failure

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387534#comment-14387534 ] Apache Spark commented on SPARK-6614: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387532#comment-14387532 ] Apache Spark commented on SPARK-2883: - User 'zhzhan' has created a pull request for th

[jira] [Updated] (SPARK-6614) OutputCommitCoordinator should clear authorized committers only after authorized committer fails, not after any failure

2015-03-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6614: -- Affects Version/s: 1.4.0 1.3.1 > OutputCommitCoordinator should clear authorized

  1   2   3   >