[jira] [Closed] (SPARK-9826) Cannot use custom classes in log4j.properties

2015-08-14 Thread Michel Lemay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michel Lemay closed SPARK-9826. --- Verified working with a local build of branch-1.4 (1.4.2-snapshot) Cannot use custom classes in

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696889#comment-14696889 ] Frank Rosner commented on SPARK-9971: - Ok so shall we close it as a wontfix?

[jira] [Commented] (SPARK-9906) User guide for LogisticRegressionSummary

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696891#comment-14696891 ] Apache Spark commented on SPARK-9906: - User 'MechCoder' has created a pull request for

[jira] [Assigned] (SPARK-9906) User guide for LogisticRegressionSummary

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9906: --- Assignee: Manoj Kumar (was: Apache Spark) User guide for LogisticRegressionSummary

[jira] [Assigned] (SPARK-9906) User guide for LogisticRegressionSummary

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9906: --- Assignee: Apache Spark (was: Manoj Kumar) User guide for LogisticRegressionSummary

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696894#comment-14696894 ] Sean Owen commented on SPARK-9971: -- I'd always prefer to leave it open at least ~1 week

[jira] [Commented] (SPARK-9974) SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged into the assembly jar

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696919#comment-14696919 ] Apache Spark commented on SPARK-9974: - User 'liancheng' has created a pull request for

[jira] [Assigned] (SPARK-9974) SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged into the assembly jar

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9974: --- Assignee: (was: Apache Spark) SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not

[jira] [Assigned] (SPARK-9974) SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged into the assembly jar

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9974: --- Assignee: Apache Spark SBT build: com.twitter:parquet-hadoop-bundle:1.6.0 is not packaged

[jira] [Commented] (SPARK-9960) run-example SparkPi fails on Mac

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696546#comment-14696546 ] Apache Spark commented on SPARK-9960: - User 'farseer90718' has created a pull request

[jira] [Commented] (SPARK-9213) Improve regular expression performance (via joni)

2015-08-14 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696561#comment-14696561 ] Yadong Qi commented on SPARK-9213: -- [~rxin] Yes, I will do it as below: ``` case class

[jira] [Created] (SPARK-9966) Handle a couple of corner cases in the PID rate estimator

2015-08-14 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-9966: Summary: Handle a couple of corner cases in the PID rate estimator Key: SPARK-9966 URL: https://issues.apache.org/jira/browse/SPARK-9966 Project: Spark

[jira] [Created] (SPARK-9967) Rename the SparkConf property to spark.streaming.backpressure.{enable -- enabled}

2015-08-14 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-9967: Summary: Rename the SparkConf property to spark.streaming.backpressure.{enable -- enabled} Key: SPARK-9967 URL: https://issues.apache.org/jira/browse/SPARK-9967

[jira] [Commented] (SPARK-9213) Improve regular expression performance (via joni)

2015-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696596#comment-14696596 ] Reynold Xin commented on SPARK-9213: We just need to handle it in the analyzer to

[jira] [Commented] (SPARK-9213) Improve regular expression performance (via joni)

2015-08-14 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696630#comment-14696630 ] Yadong Qi commented on SPARK-9213: -- I know, thanks! Improve regular expression

[jira] [Updated] (SPARK-9958) HiveThriftServer2Listener is not thread-safe

2015-08-14 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-9958: -- Assignee: Shixiong Zhu HiveThriftServer2Listener is not thread-safe

[jira] [Resolved] (SPARK-9958) HiveThriftServer2Listener is not thread-safe

2015-08-14 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-9958. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8185

[jira] [Commented] (SPARK-9961) ML prediction abstractions should have defaultEvaluator fields

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696640#comment-14696640 ] Apache Spark commented on SPARK-9961: - User 'mengxr' has created a pull request for

[jira] [Assigned] (SPARK-9961) ML prediction abstractions should have defaultEvaluator fields

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9961: --- Assignee: (was: Apache Spark) ML prediction abstractions should have defaultEvaluator

[jira] [Commented] (SPARK-9661) ML 1.5 QA: API: Java compatibility, docs

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696641#comment-14696641 ] Apache Spark commented on SPARK-9661: - User 'mengxr' has created a pull request for

[jira] [Assigned] (SPARK-9961) ML prediction abstractions should have defaultEvaluator fields

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9961: --- Assignee: Apache Spark ML prediction abstractions should have defaultEvaluator fields

[jira] [Reopened] (SPARK-9661) ML 1.5 QA: API: Java compatibility, docs

2015-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-9661: -- re-open for some clean-ups ML 1.5 QA: API: Java compatibility, docs

[jira] [Created] (SPARK-9964) PySpark DataFrameReader accept RDD of String for JSON

2015-08-14 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-9964: Summary: PySpark DataFrameReader accept RDD of String for JSON Key: SPARK-9964 URL: https://issues.apache.org/jira/browse/SPARK-9964 Project: Spark

[jira] [Created] (SPARK-9963) ML RandomForest cleanup: replace predictNodeIndex with predictImpl

2015-08-14 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-9963: Summary: ML RandomForest cleanup: replace predictNodeIndex with predictImpl Key: SPARK-9963 URL: https://issues.apache.org/jira/browse/SPARK-9963 Project:

[jira] [Created] (SPARK-9965) Scala, Python SQLContext input methods' deprecation statuses do not match

2015-08-14 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-9965: Summary: Scala, Python SQLContext input methods' deprecation statuses do not match Key: SPARK-9965 URL: https://issues.apache.org/jira/browse/SPARK-9965

[jira] [Comment Edited] (SPARK-9213) Improve regular expression performance (via joni)

2015-08-14 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696561#comment-14696561 ] Yadong Qi edited comment on SPARK-9213 at 8/14/15 6:34 AM: ---

[jira] [Updated] (SPARK-9968) BlockGenerator lock structure can cause lock starvation of the block updating thread

2015-08-14 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-9968: - Description: When the rate limiter is actually limiting the rate at which data is inserted into

[jira] [Created] (SPARK-9968) BlockGenerator lock structure can cause lock starvation of the block updating thread

2015-08-14 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-9968: Summary: BlockGenerator lock structure can cause lock starvation of the block updating thread Key: SPARK-9968 URL: https://issues.apache.org/jira/browse/SPARK-9968

[jira] [Issue Comment Deleted] (SPARK-9961) ML prediction abstractions should have defaultEvaluator fields

2015-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-9961: - Comment: was deleted (was: User 'mengxr' has created a pull request for this issue:

[jira] [Created] (SPARK-9961) ML prediction abstractions should have defaultEvaluator fields

2015-08-14 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-9961: Summary: ML prediction abstractions should have defaultEvaluator fields Key: SPARK-9961 URL: https://issues.apache.org/jira/browse/SPARK-9961 Project: Spark

[jira] [Created] (SPARK-9962) Decision Tree training: prevNodeIdsForInstances.unpersist() at end of training

2015-08-14 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-9962: Summary: Decision Tree training: prevNodeIdsForInstances.unpersist() at end of training Key: SPARK-9962 URL: https://issues.apache.org/jira/browse/SPARK-9962

[jira] [Comment Edited] (SPARK-9213) Improve regular expression performance (via joni)

2015-08-14 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696561#comment-14696561 ] Yadong Qi edited comment on SPARK-9213 at 8/14/15 6:33 AM: ---

[jira] [Commented] (SPARK-9213) Improve regular expression performance (via joni)

2015-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696574#comment-14696574 ] Reynold Xin commented on SPARK-9213: I'm thinking just have Like for Joni, and then

[jira] [Commented] (SPARK-9213) Improve regular expression performance (via joni)

2015-08-14 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696595#comment-14696595 ] Yadong Qi commented on SPARK-9213: -- [~rxin] There're many place use Like, It does not

[jira] [Commented] (SPARK-6624) Convert filters into CNF for data sources

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696988#comment-14696988 ] Apache Spark commented on SPARK-6624: - User 'yjshen' has created a pull request for

[jira] [Assigned] (SPARK-9968) BlockGenerator lock structure can cause lock starvation of the block updating thread

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9968: --- Assignee: Apache Spark (was: Tathagata Das) BlockGenerator lock structure can cause lock

[jira] [Commented] (SPARK-9968) BlockGenerator lock structure can cause lock starvation of the block updating thread

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696987#comment-14696987 ] Apache Spark commented on SPARK-9968: - User 'tdas' has created a pull request for this

[jira] [Assigned] (SPARK-9968) BlockGenerator lock structure can cause lock starvation of the block updating thread

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9968: --- Assignee: Tathagata Das (was: Apache Spark) BlockGenerator lock structure can cause lock

[jira] [Commented] (SPARK-9734) java.lang.IllegalArgumentException: Don't know how to save StructField(sal,DecimalType(7,2),true) to JDBC

2015-08-14 Thread Rama Mullapudi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697003#comment-14697003 ] Rama Mullapudi commented on SPARK-9734: --- Current 1.5 still gives error when creating

[jira] [Created] (SPARK-9977) The usage of a label generated by StringIndexer

2015-08-14 Thread Kai Sasaki (JIRA)
Kai Sasaki created SPARK-9977: - Summary: The usage of a label generated by StringIndexer Key: SPARK-9977 URL: https://issues.apache.org/jira/browse/SPARK-9977 Project: Spark Issue Type:

[jira] [Updated] (SPARK-9976) create function do not work

2015-08-14 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-9976: - Description: I use beeline to connect to ThriftServer, but add jar can not work, so I use create

[jira] [Created] (SPARK-9978) Window functions require partitionBy to work as expected

2015-08-14 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-9978: - Summary: Window functions require partitionBy to work as expected Key: SPARK-9978 URL: https://issues.apache.org/jira/browse/SPARK-9978 Project: Spark

[jira] [Updated] (SPARK-9978) Window functions require partitionBy to work as expected

2015-08-14 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-9978: -- Description: I am trying to reproduce following SQL query: {code}

[jira] [Commented] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-08-14 Thread ding (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696677#comment-14696677 ] ding commented on SPARK-5556: - The code can be found

[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

2015-08-14 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696684#comment-14696684 ] Zhongshuai Pei commented on SPARK-9725: --- In my environment,offset did not change

[jira] [Updated] (SPARK-9970) SQLContext.createDataFrame failed to properly determine column names

2015-08-14 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-9970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-9970: -- Description: Hi, I'm trying to do nested join of tables. After first join everything is ok, but

[jira] [Assigned] (SPARK-9899) JSON/Parquet writing on retry or speculation broken with direct output committer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9899: --- Assignee: Cheng Lian (was: Apache Spark) JSON/Parquet writing on retry or speculation

[jira] [Commented] (SPARK-9899) JSON/Parquet writing on retry or speculation broken with direct output committer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696672#comment-14696672 ] Apache Spark commented on SPARK-9899: - User 'liancheng' has created a pull request for

[jira] [Assigned] (SPARK-9899) JSON/Parquet writing on retry or speculation broken with direct output committer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9899: --- Assignee: Apache Spark (was: Cheng Lian) JSON/Parquet writing on retry or speculation

[jira] [Created] (SPARK-9970) SQLContext.createDataFrame failed to properly determine column names

2015-08-14 Thread JIRA
Maciej Bryński created SPARK-9970: - Summary: SQLContext.createDataFrame failed to properly determine column names Key: SPARK-9970 URL: https://issues.apache.org/jira/browse/SPARK-9970 Project: Spark

[jira] [Updated] (SPARK-9970) SQLContext.createDataFrame failed to properly determine column names

2015-08-14 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-9970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-9970: -- Description: Hi, I'm trying to do nested join of tables. After first join everything is ok, but

[jira] [Commented] (SPARK-9871) Add expression functions into SparkR which have a variable parameter

2015-08-14 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696706#comment-14696706 ] Yu Ishikawa commented on SPARK-9871: I think it would be better to deal with

[jira] [Created] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
Frank Rosner created SPARK-9971: --- Summary: MaxFunction not working correctly with columns containing Double.NaN Key: SPARK-9971 URL: https://issues.apache.org/jira/browse/SPARK-9971 Project: Spark

[jira] [Created] (SPARK-9972) Add `struct` function in SparkR

2015-08-14 Thread Yu Ishikawa (JIRA)
Yu Ishikawa created SPARK-9972: -- Summary: Add `struct` function in SparkR Key: SPARK-9972 URL: https://issues.apache.org/jira/browse/SPARK-9972 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-9973) wrong buffle size

2015-08-14 Thread xukun (JIRA)
xukun created SPARK-9973: Summary: wrong buffle size Key: SPARK-9973 URL: https://issues.apache.org/jira/browse/SPARK-9973 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-9973) wrong buffle size

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696717#comment-14696717 ] Sean Owen commented on SPARK-9973: -- [~xukun] This would be more useful if you added a

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696716#comment-14696716 ] Sean Owen commented on SPARK-9971: -- My instinct is that this should in fact result in

[jira] [Commented] (SPARK-9871) Add expression functions into SparkR which have a variable parameter

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696720#comment-14696720 ] Apache Spark commented on SPARK-9871: - User 'yu-iskw' has created a pull request for

[jira] [Assigned] (SPARK-9871) Add expression functions into SparkR which have a variable parameter

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9871: --- Assignee: (was: Apache Spark) Add expression functions into SparkR which have a

[jira] [Assigned] (SPARK-9871) Add expression functions into SparkR which have a variable parameter

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9871: --- Assignee: Apache Spark Add expression functions into SparkR which have a variable parameter

[jira] [Created] (SPARK-9969) Remove old Yarn MR classpath api support for Spark Yarn client

2015-08-14 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-9969: -- Summary: Remove old Yarn MR classpath api support for Spark Yarn client Key: SPARK-9969 URL: https://issues.apache.org/jira/browse/SPARK-9969 Project: Spark

[jira] [Commented] (SPARK-9969) Remove old Yarn MR classpath api support for Spark Yarn client

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696681#comment-14696681 ] Apache Spark commented on SPARK-9969: - User 'jerryshao' has created a pull request for

[jira] [Assigned] (SPARK-9969) Remove old Yarn MR classpath api support for Spark Yarn client

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9969: --- Assignee: Apache Spark Remove old Yarn MR classpath api support for Spark Yarn client

[jira] [Assigned] (SPARK-9969) Remove old Yarn MR classpath api support for Spark Yarn client

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9969: --- Assignee: (was: Apache Spark) Remove old Yarn MR classpath api support for Spark Yarn

[jira] [Updated] (SPARK-9970) SQLContext.createDataFrame failed to properly determine column names

2015-08-14 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-9970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-9970: -- Description: Hi, I'm trying to do nested join of tables. After first join everything is ok, but

[jira] [Commented] (SPARK-6624) Convert filters into CNF for data sources

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696700#comment-14696700 ] Apache Spark commented on SPARK-6624: - User 'yjshen' has created a pull request for

[jira] [Assigned] (SPARK-6624) Convert filters into CNF for data sources

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6624: --- Assignee: (was: Apache Spark) Convert filters into CNF for data sources

[jira] [Assigned] (SPARK-6624) Convert filters into CNF for data sources

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6624: --- Assignee: Apache Spark Convert filters into CNF for data sources

[jira] [Updated] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Rosner updated SPARK-9971: Priority: Minor (was: Major) MaxFunction not working correctly with columns containing Double.NaN

[jira] [Updated] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Rosner updated SPARK-9971: Description: h4. Problem Description When using the {{max}} function on a {{DoubleType}} column

[jira] [Updated] (SPARK-9973) wrong buffle size

2015-08-14 Thread xukun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xukun updated SPARK-9973: - Description: allocate wrong buffer size (4+ size * columnType.defaultSize * columnType.defaultSize), right

[jira] [Commented] (SPARK-9973) wrong buffle size

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696714#comment-14696714 ] Apache Spark commented on SPARK-9973: - User 'viper-kun' has created a pull request for

[jira] [Assigned] (SPARK-9973) wrong buffle size

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9973: --- Assignee: (was: Apache Spark) wrong buffle size -

[jira] [Assigned] (SPARK-9973) wrong buffle size

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9973: --- Assignee: Apache Spark wrong buffle size - Key:

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

2015-08-14 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696760#comment-14696760 ] Frank Rosner commented on SPARK-9971: - I would like to provide a patch to make the

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697098#comment-14697098 ] Cody Koeninger commented on SPARK-9947: --- You already have access to offsets and can

[jira] [Created] (SPARK-9979) Unable to request addition to powered by spark

2015-08-14 Thread Jeff Palmucci (JIRA)
Jeff Palmucci created SPARK-9979: Summary: Unable to request addition to powered by spark Key: SPARK-9979 URL: https://issues.apache.org/jira/browse/SPARK-9979 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Dan Dutrow (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697157#comment-14697157 ] Dan Dutrow commented on SPARK-9947: --- The desire is to continue using checkpointing for

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697166#comment-14697166 ] Cody Koeninger commented on SPARK-9947: --- You can't re-use checkpoint data across

[jira] [Assigned] (SPARK-9977) The usage of a label generated by StringIndexer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9977: --- Assignee: Apache Spark The usage of a label generated by StringIndexer

[jira] [Assigned] (SPARK-9977) The usage of a label generated by StringIndexer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9977: --- Assignee: (was: Apache Spark) The usage of a label generated by StringIndexer

[jira] [Commented] (SPARK-9977) The usage of a label generated by StringIndexer

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697104#comment-14697104 ] Apache Spark commented on SPARK-9977: - User 'Lewuathe' has created a pull request for

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Dan Dutrow (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697180#comment-14697180 ] Dan Dutrow commented on SPARK-9947: --- I want to maintain the data in updateStateByKey

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697193#comment-14697193 ] Cody Koeninger commented on SPARK-9947: --- Didn't you already say that you were saving

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

2015-08-14 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697203#comment-14697203 ] Cheng Lian commented on SPARK-8118: --- Unfortunately no. Turn off noisy log output

[jira] [Commented] (SPARK-6249) Get Kafka offsets from consumer group in ZK when using direct stream

2015-08-14 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697188#comment-14697188 ] Cody Koeninger commented on SPARK-6249: --- If you want an api that has imprecise

[jira] [Commented] (SPARK-9947) Separate Metadata and State Checkpoint Data

2015-08-14 Thread Dan Dutrow (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697220#comment-14697220 ] Dan Dutrow commented on SPARK-9947: --- Ok, right, we're saving the same state that's

[jira] [Updated] (SPARK-9972) Add `struct` function in SparkR

2015-08-14 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-9972: --- Target Version/s: 1.6.0 (was: 1.5.0) Add `struct` function in SparkR

[jira] [Commented] (SPARK-9972) Add `struct` function in SparkR

2015-08-14 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697232#comment-14697232 ] Shivaram Venkataraman commented on SPARK-9972: -- Yeah this can be marked as

[jira] [Resolved] (SPARK-9946) NPE in TaskMemoryManager

2015-08-14 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9946. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8177

[jira] [Resolved] (SPARK-9828) Should not share `{}` among instances

2015-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-9828. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8110

[jira] [Created] (SPARK-9984) Create local physical operator interface

2015-08-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9984: -- Summary: Create local physical operator interface Key: SPARK-9984 URL: https://issues.apache.org/jira/browse/SPARK-9984 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-9985) DataFrameWriter jdbc method ignore options that have been set

2015-08-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9985: Assignee: Shixiong Zhu DataFrameWriter jdbc method ignore options that have been set

[jira] [Resolved] (SPARK-9828) Should not share `{}` among instances

2015-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-9828. -- Resolution: Fixed Fix Version/s: 1.4.2 Should not share `{}` among instances

[jira] [Created] (SPARK-9986) Create a simple test framework for local operators

2015-08-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9986: -- Summary: Create a simple test framework for local operators Key: SPARK-9986 URL: https://issues.apache.org/jira/browse/SPARK-9986 Project: Spark Issue Type:

[jira] [Created] (SPARK-9994) Create local TopK operator

2015-08-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9994: -- Summary: Create local TopK operator Key: SPARK-9994 URL: https://issues.apache.org/jira/browse/SPARK-9994 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-9993) Create local union operator

2015-08-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9993: -- Summary: Create local union operator Key: SPARK-9993 URL: https://issues.apache.org/jira/browse/SPARK-9993 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-9589) Flaky test: HiveCompatibilitySuite.groupby8

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9589: --- Assignee: Apache Spark (was: Josh Rosen) Flaky test: HiveCompatibilitySuite.groupby8

[jira] [Assigned] (SPARK-9561) Enable BroadcastJoinSuite

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9561: --- Assignee: Andrew Or (was: Apache Spark) Enable BroadcastJoinSuite

[jira] [Commented] (SPARK-9561) Enable BroadcastJoinSuite

2015-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697425#comment-14697425 ] Apache Spark commented on SPARK-9561: - User 'andrewor14' has created a pull request

  1   2   3   >