[jira] [Commented] (SPARK-9213) Improve regular expression performance (via joni)

2015-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696596#comment-14696596 ] Reynold Xin commented on SPARK-9213: We just need to handle it in the analyzer to rewr

[jira] [Commented] (SPARK-9213) Improve regular expression performance (via joni)

2015-08-14 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696630#comment-14696630 ] Yadong Qi commented on SPARK-9213: -- I know, thanks! > Improve regular expression perform

[jira] [Assigned] (SPARK-9961) ML prediction abstractions should have defaultEvaluator fields

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9961: --- Assignee: Apache Spark > ML prediction abstractions should have defaultEvaluator fields > ---

[jira] [Commented] (SPARK-9961) ML prediction abstractions should have defaultEvaluator fields

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696640#comment-14696640 ] Apache Spark commented on SPARK-9961: - User 'mengxr' has created a pull request for th

[jira] [Assigned] (SPARK-9961) ML prediction abstractions should have defaultEvaluator fields

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9961: --- Assignee: (was: Apache Spark) > ML prediction abstractions should have defaultEvaluator f

[jira] [Commented] (SPARK-9661) ML 1.5 QA: API: Java compatibility, docs

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696641#comment-14696641 ] Apache Spark commented on SPARK-9661: - User 'mengxr' has created a pull request for th

[jira] [Reopened] (SPARK-9661) ML 1.5 QA: API: Java compatibility, docs

2015-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-9661: -- re-open for some clean-ups > ML 1.5 QA: API: Java compatibility, docs > ---

[jira] [Issue Comment Deleted] (SPARK-9961) ML prediction abstractions should have defaultEvaluator fields

2015-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-9961: - Comment: was deleted (was: User 'mengxr' has created a pull request for this issue: https://github

[jira] [Assigned] (SPARK-9899) JSON/Parquet writing on retry or speculation broken with direct output committer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9899: --- Assignee: Apache Spark (was: Cheng Lian) > JSON/Parquet writing on retry or speculation brok

[jira] [Assigned] (SPARK-9899) JSON/Parquet writing on retry or speculation broken with direct output committer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9899: --- Assignee: Cheng Lian (was: Apache Spark) > JSON/Parquet writing on retry or speculation brok

[jira] [Commented] (SPARK-9899) JSON/Parquet writing on retry or speculation broken with direct output committer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696672#comment-14696672 ] Apache Spark commented on SPARK-9899: - User 'liancheng' has created a pull request for

[jira] [Created] (SPARK-9969) Remove old Yarn MR classpath api support for Spark Yarn client

2015-08-14 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-9969: -- Summary: Remove old Yarn MR classpath api support for Spark Yarn client Key: SPARK-9969 URL: https://issues.apache.org/jira/browse/SPARK-9969 Project: Spark Iss

[jira] [Commented] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-08-14 Thread ding (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696677#comment-14696677 ] ding commented on SPARK-5556: - The code can be found https://github.com/intel-analytics/TopicM

[jira] [Assigned] (SPARK-9969) Remove old Yarn MR classpath api support for Spark Yarn client

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9969: --- Assignee: Apache Spark > Remove old Yarn MR classpath api support for Spark Yarn client > ---

[jira] [Commented] (SPARK-9969) Remove old Yarn MR classpath api support for Spark Yarn client

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696681#comment-14696681 ] Apache Spark commented on SPARK-9969: - User 'jerryshao' has created a pull request for

[jira] [Assigned] (SPARK-9969) Remove old Yarn MR classpath api support for Spark Yarn client

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9969: --- Assignee: (was: Apache Spark) > Remove old Yarn MR classpath api support for Spark Yarn c

[jira] [Created] (SPARK-9970) SQLContext.createDataFrame failed to properly determine column names

2015-08-14 Thread JIRA
Maciej Bryński created SPARK-9970: - Summary: SQLContext.createDataFrame failed to properly determine column names Key: SPARK-9970 URL: https://issues.apache.org/jira/browse/SPARK-9970 Project: Spark

[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

2015-08-14 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696684#comment-14696684 ] Zhongshuai Pei commented on SPARK-9725: --- In my environment,"offset" did not change a

[jira] [Updated] (SPARK-9970) SQLContext.createDataFrame failed to properly determine column names

2015-08-14 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-9970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-9970: -- Description: Hi, I'm trying to do "nested join" of tables. After first join everything is ok, bu

[jira] [Updated] (SPARK-9970) SQLContext.createDataFrame failed to properly determine column names

2015-08-14 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-9970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-9970: -- Description: Hi, I'm trying to do "nested join" of tables. After first join everything is ok, bu

[jira] [Commented] (SPARK-6624) Convert filters into CNF for data sources

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696700#comment-14696700 ] Apache Spark commented on SPARK-6624: - User 'yjshen' has created a pull request for th

[jira] [Assigned] (SPARK-6624) Convert filters into CNF for data sources

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6624: --- Assignee: (was: Apache Spark) > Convert filters into CNF for data sources > -

[jira] [Assigned] (SPARK-6624) Convert filters into CNF for data sources

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6624: --- Assignee: Apache Spark > Convert filters into CNF for data sources >

[jira] [Updated] (SPARK-9970) SQLContext.createDataFrame failed to properly determine column names

2015-08-14 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-9970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-9970: -- Description: Hi, I'm trying to do "nested join" of tables. After first join everything is ok, bu

[jira] [Commented] (SPARK-9871) Add expression functions into SparkR which have a variable parameter

2015-08-14 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696706#comment-14696706 ] Yu Ishikawa commented on SPARK-9871: I think it would be better to deal with {{struct}

[jira] [Created] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
Frank Rosner created SPARK-9971: --- Summary: MaxFunction not working correctly with columns containing Double.NaN Key: SPARK-9971 URL: https://issues.apache.org/jira/browse/SPARK-9971 Project: Spark

[jira] [Created] (SPARK-9973) wrong buffle size

2015-08-14 Thread xukun (JIRA)
xukun created SPARK-9973: Summary: wrong buffle size Key: SPARK-9973 URL: https://issues.apache.org/jira/browse/SPARK-9973 Project: Spark Issue Type: Bug Components: SQL Reporte

[jira] [Created] (SPARK-9972) Add `struct` function in SparkR

2015-08-14 Thread Yu Ishikawa (JIRA)
Yu Ishikawa created SPARK-9972: -- Summary: Add `struct` function in SparkR Key: SPARK-9972 URL: https://issues.apache.org/jira/browse/SPARK-9972 Project: Spark Issue Type: Sub-task Comp

[jira] [Updated] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Rosner updated SPARK-9971: Priority: Minor (was: Major) > MaxFunction not working correctly with columns containing Double.NaN

[jira] [Updated] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Rosner updated SPARK-9971: Description: h4. Problem Description When using the {{max}} function on a {{DoubleType}} column tha

[jira] [Updated] (SPARK-9973) wrong buffle size

2015-08-14 Thread xukun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xukun updated SPARK-9973: - Description: allocate wrong buffer size (4+ size * columnType.defaultSize * columnType.defaultSize), right buffer

[jira] [Assigned] (SPARK-9973) wrong buffle size

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9973: --- Assignee: Apache Spark > wrong buffle size > - > > Key: SPARK

[jira] [Commented] (SPARK-9973) wrong buffle size

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696714#comment-14696714 ] Apache Spark commented on SPARK-9973: - User 'viper-kun' has created a pull request for

[jira] [Assigned] (SPARK-9973) wrong buffle size

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9973: --- Assignee: (was: Apache Spark) > wrong buffle size > - > >

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696716#comment-14696716 ] Sean Owen commented on SPARK-9971: -- My instinct is that this should in fact result in NaN

[jira] [Commented] (SPARK-9973) wrong buffle size

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696717#comment-14696717 ] Sean Owen commented on SPARK-9973: -- [~xukun] This would be more useful if you added a des

[jira] [Assigned] (SPARK-9871) Add expression functions into SparkR which have a variable parameter

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9871: --- Assignee: (was: Apache Spark) > Add expression functions into SparkR which have a variabl

[jira] [Assigned] (SPARK-9871) Add expression functions into SparkR which have a variable parameter

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9871: --- Assignee: Apache Spark > Add expression functions into SparkR which have a variable parameter

[jira] [Commented] (SPARK-9871) Add expression functions into SparkR which have a variable parameter

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696720#comment-14696720 ] Apache Spark commented on SPARK-9871: - User 'yu-iskw' has created a pull request for t

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696760#comment-14696760 ] Frank Rosner commented on SPARK-9971: - I would like to provide a patch to make the fol

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696762#comment-14696762 ] Sean Owen commented on SPARK-9971: -- Hey Frank as I say I don't think that's the correct a

[jira] [Comment Edited] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696760#comment-14696760 ] Frank Rosner edited comment on SPARK-9971 at 8/14/15 9:34 AM: --

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696766#comment-14696766 ] Frank Rosner commented on SPARK-9971: - [~srowen] I get your point. But given your "wei

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696768#comment-14696768 ] Sean Owen commented on SPARK-9971: -- My argument is being consistent with Java really, and

[jira] [Resolved] (SPARK-9780) In case of invalid initialization of KafkaDirectStream, NPE is thrown

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9780. -- Resolution: Fixed Assignee: Cody Koeninger Fix Version/s: 1.5.0 > In case of invalid ini

[jira] [Created] (SPARK-9974) SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged into the assembly jar

2015-08-14 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-9974: - Summary: SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged into the assembly jar Key: SPARK-9974 URL: https://issues.apache.org/jira/browse/SPARK-9974 Proj

[jira] [Updated] (SPARK-9974) SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged into the assembly jar

2015-08-14 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-9974: -- Description: One of the consequence of this issue is that Parquet tables created in Hive are not access

[jira] [Created] (SPARK-9975) Add Normalized Closeness Centrality to Spark GraphX

2015-08-14 Thread Kenny Bastani (JIRA)
Kenny Bastani created SPARK-9975: Summary: Add Normalized Closeness Centrality to Spark GraphX Key: SPARK-9975 URL: https://issues.apache.org/jira/browse/SPARK-9975 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-9975) Add Normalized Closeness Centrality to Spark GraphX

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9975: --- Assignee: Apache Spark > Add Normalized Closeness Centrality to Spark GraphX > --

[jira] [Commented] (SPARK-9975) Add Normalized Closeness Centrality to Spark GraphX

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696799#comment-14696799 ] Apache Spark commented on SPARK-9975: - User 'kbastani' has created a pull request for

[jira] [Assigned] (SPARK-9975) Add Normalized Closeness Centrality to Spark GraphX

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9975: --- Assignee: (was: Apache Spark) > Add Normalized Closeness Centrality to Spark GraphX > ---

[jira] [Updated] (SPARK-8922) Add @since tags to mllib.evaluation

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8922: - Assignee: Xiangrui Meng > Add @since tags to mllib.evaluation > --- > >

[jira] [Updated] (SPARK-9726) PySpark Regression: DataFrame join no longer accepts None as join expression

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9726: - Assignee: Brennan Ashton > PySpark Regression: DataFrame join no longer accepts None as join expression >

[jira] [Updated] (SPARK-9826) Cannot use custom classes in log4j.properties

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9826: - Assignee: Michel Lemay > Cannot use custom classes in log4j.properties > -

[jira] [Updated] (SPARK-9936) decimal precision lost when loading DataFrame from RDD

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9936: - Assignee: Liang-Chi Hsieh > decimal precision lost when loading DataFrame from RDD > -

[jira] [Updated] (SPARK-9931) Flaky test: mllib/tests.py StreamingLogisticRegressionWithSGDTests. test_training_and_prediction

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9931: - Component/s: Tests PySpark > Flaky test: mllib/tests.py StreamingLogisticRegressionWithSG

[jira] [Updated] (SPARK-9934) Deprecate NIO ConnectionManager

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9934: - Component/s: Spark Core > Deprecate NIO ConnectionManager > --- > >

[jira] [Updated] (SPARK-9878) ReduceByKey + FullOuterJoin return 0 element if using an empty RDD

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9878: - Component/s: Spark Core > ReduceByKey + FullOuterJoin return 0 element if using an empty RDD > --

[jira] [Closed] (SPARK-9637) Add interface for implementing scheduling algorithm for standalone deployment

2015-08-14 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-9637. -- Resolution: Duplicate > Add interface for implementing scheduling algorithm for standalone deplo

[jira] [Resolved] (SPARK-9396) Spark yarn allocator does not call "removeContainerRequest" for allocated Container requests, resulting in bloated ask[] toYarn RM.

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9396. -- Resolution: Won't Fix This is fixed in 1.3+ by other changes; unless a decision is taken to make anothe

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696849#comment-14696849 ] Apache Spark commented on SPARK-8118: - User 'liancheng' has created a pull request for

[jira] [Closed] (SPARK-9826) Cannot use custom classes in log4j.properties

2015-08-14 Thread Michel Lemay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michel Lemay closed SPARK-9826. --- Verified working with a local build of branch-1.4 (1.4.2-snapshot) > Cannot use custom classes in log4j.p

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696889#comment-14696889 ] Frank Rosner commented on SPARK-9971: - Ok so shall we close it as a wontfix? > MaxFun

[jira] [Assigned] (SPARK-9906) User guide for LogisticRegressionSummary

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9906: --- Assignee: Manoj Kumar (was: Apache Spark) > User guide for LogisticRegressionSummary > -

[jira] [Assigned] (SPARK-9906) User guide for LogisticRegressionSummary

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9906: --- Assignee: Apache Spark (was: Manoj Kumar) > User guide for LogisticRegressionSummary > -

[jira] [Commented] (SPARK-9906) User guide for LogisticRegressionSummary

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696891#comment-14696891 ] Apache Spark commented on SPARK-9906: - User 'MechCoder' has created a pull request for

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696894#comment-14696894 ] Sean Owen commented on SPARK-9971: -- I'd always prefer to leave it open at least ~1 week t

[jira] [Assigned] (SPARK-9974) SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged into the assembly jar

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9974: --- Assignee: Apache Spark > SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged i

[jira] [Commented] (SPARK-9974) SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged into the assembly jar

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696919#comment-14696919 ] Apache Spark commented on SPARK-9974: - User 'liancheng' has created a pull request for

[jira] [Assigned] (SPARK-9974) SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged into the assembly jar

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9974: --- Assignee: (was: Apache Spark) > SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not

[jira] [Updated] (SPARK-6624) Convert filters into CNF for data sources

2015-08-14 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6624: -- Assignee: Yijie Shen > Convert filters into CNF for data sources > -

[jira] [Commented] (SPARK-9966) Handle a couple of corner cases in the PID rate estimator

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696928#comment-14696928 ] Apache Spark commented on SPARK-9966: - User 'tdas' has created a pull request for this

[jira] [Assigned] (SPARK-9966) Handle a couple of corner cases in the PID rate estimator

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9966: --- Assignee: Tathagata Das (was: Apache Spark) > Handle a couple of corner cases in the PID rat

[jira] [Assigned] (SPARK-9966) Handle a couple of corner cases in the PID rate estimator

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9966: --- Assignee: Apache Spark (was: Tathagata Das) > Handle a couple of corner cases in the PID rat

[jira] [Commented] (SPARK-6624) Convert filters into CNF for data sources

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696936#comment-14696936 ] Apache Spark commented on SPARK-6624: - User 'yjshen' has created a pull request for th

[jira] [Commented] (SPARK-8887) Explicitly define which data types can be used as dynamic partition columns

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696937#comment-14696937 ] Apache Spark commented on SPARK-8887: - User 'yjshen' has created a pull request for th

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696938#comment-14696938 ] Apache Spark commented on SPARK-8670: - User 'cloud-fan' has created a pull request for

[jira] [Updated] (SPARK-9970) SQLContext.createDataFrame failed to properly determine column names

2015-08-14 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-9970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-9970: -- Priority: Major (was: Minor) > SQLContext.createDataFrame failed to properly determine column n

[jira] [Created] (SPARK-9976) create function do not work

2015-08-14 Thread cen yuhai (JIRA)
cen yuhai created SPARK-9976: Summary: create function do not work Key: SPARK-9976 URL: https://issues.apache.org/jira/browse/SPARK-9976 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-9955) TPCDS Q8 failed in 1.5

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696975#comment-14696975 ] Apache Spark commented on SPARK-9955: - User 'cloud-fan' has created a pull request for

[jira] [Assigned] (SPARK-9955) TPCDS Q8 failed in 1.5

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9955: --- Assignee: Apache Spark (was: Wenchen Fan) > TPCDS Q8 failed in 1.5 > --

[jira] [Assigned] (SPARK-9955) TPCDS Q8 failed in 1.5

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9955: --- Assignee: Wenchen Fan (was: Apache Spark) > TPCDS Q8 failed in 1.5 > --

[jira] [Assigned] (SPARK-9968) BlockGenerator lock structure can cause lock starvation of the block updating thread

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9968: --- Assignee: Apache Spark (was: Tathagata Das) > BlockGenerator lock structure can cause lock s

[jira] [Assigned] (SPARK-9968) BlockGenerator lock structure can cause lock starvation of the block updating thread

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9968: --- Assignee: Tathagata Das (was: Apache Spark) > BlockGenerator lock structure can cause lock s

[jira] [Commented] (SPARK-9968) BlockGenerator lock structure can cause lock starvation of the block updating thread

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696987#comment-14696987 ] Apache Spark commented on SPARK-9968: - User 'tdas' has created a pull request for this

[jira] [Commented] (SPARK-6624) Convert filters into CNF for data sources

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696988#comment-14696988 ] Apache Spark commented on SPARK-6624: - User 'yjshen' has created a pull request for th

[jira] [Commented] (SPARK-9734) java.lang.IllegalArgumentException: Don't know how to save StructField(sal,DecimalType(7,2),true) to JDBC

2015-08-14 Thread Rama Mullapudi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697003#comment-14697003 ] Rama Mullapudi commented on SPARK-9734: --- Current 1.5 still gives error when creating

[jira] [Created] (SPARK-9977) The usage of a label generated by StringIndexer

2015-08-14 Thread Kai Sasaki (JIRA)
Kai Sasaki created SPARK-9977: - Summary: The usage of a label generated by StringIndexer Key: SPARK-9977 URL: https://issues.apache.org/jira/browse/SPARK-9977 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-9976) create function do not work

2015-08-14 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-9976: - Description: I use beeline to connect to ThriftServer, but add jar can not work, so I use create function

[jira] [Created] (SPARK-9978) Window functions require partitionBy to work as expected

2015-08-14 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-9978: - Summary: Window functions require partitionBy to work as expected Key: SPARK-9978 URL: https://issues.apache.org/jira/browse/SPARK-9978 Project: Spark

[jira] [Updated] (SPARK-9978) Window functions require partitionBy to work as expected

2015-08-14 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-9978: -- Description: I am trying to reproduce following SQL query: {code} df.registerTempTable(

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697098#comment-14697098 ] Cody Koeninger commented on SPARK-9947: --- You already have access to offsets and can

[jira] [Assigned] (SPARK-9977) The usage of a label generated by StringIndexer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9977: --- Assignee: Apache Spark > The usage of a label generated by StringIndexer > --

[jira] [Assigned] (SPARK-9977) The usage of a label generated by StringIndexer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9977: --- Assignee: (was: Apache Spark) > The usage of a label generated by StringIndexer > ---

[jira] [Commented] (SPARK-9977) The usage of a label generated by StringIndexer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697104#comment-14697104 ] Apache Spark commented on SPARK-9977: - User 'Lewuathe' has created a pull request for

[jira] [Created] (SPARK-9979) Unable to request addition to "powered by spark"

2015-08-14 Thread Jeff Palmucci (JIRA)
Jeff Palmucci created SPARK-9979: Summary: Unable to request addition to "powered by spark" Key: SPARK-9979 URL: https://issues.apache.org/jira/browse/SPARK-9979 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Dan Dutrow (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697157#comment-14697157 ] Dan Dutrow commented on SPARK-9947: --- The desire is to continue using checkpointing for e

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697166#comment-14697166 ] Cody Koeninger commented on SPARK-9947: --- You can't re-use checkpoint data across app

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Dan Dutrow (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697180#comment-14697180 ] Dan Dutrow commented on SPARK-9947: --- I want to maintain the data in updateStateByKey bet

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697193#comment-14697193 ] Cody Koeninger commented on SPARK-9947: --- Didn't you already say that you were saving

  1   2   3   >