[jira] [Commented] (SPARK-5770) Use addJar() to upload a new jar file to executor, it can't be added to classloader

2016-06-23 Thread huangning (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347750#comment-15347750 ] huangning commented on SPARK-5770: -- Because it is not support deleteJar or updataJar, so I can't update

[jira] [Assigned] (SPARK-16179) UDF explosion yielding empty dataframe fails

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-16179: -- Assignee: Davies Liu > UDF explosion yielding empty dataframe fails >

[jira] [Commented] (SPARK-16000) Make model loading backward compatible with saved models using old vector columns

2016-06-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347705#comment-15347705 ] Xiangrui Meng commented on SPARK-16000: --- We can distribute the work easily if you can help create

[jira] [Updated] (SPARK-16179) UDF explosion yielding empty dataframe fails

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16179: --- Affects Version/s: 2.0.0 > UDF explosion yielding empty dataframe fails >

[jira] [Assigned] (SPARK-16181) Incorrect behavior for isNull filter

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16181: Assignee: (was: Apache Spark) > Incorrect behavior for isNull filter >

[jira] [Assigned] (SPARK-16181) Incorrect behavior for isNull filter

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16181: Assignee: Apache Spark > Incorrect behavior for isNull filter >

[jira] [Assigned] (SPARK-16179) UDF explosion yielding empty dataframe fails

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16179: Assignee: Apache Spark > UDF explosion yielding empty dataframe fails >

[jira] [Commented] (SPARK-16181) Incorrect behavior for isNull filter

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347704#comment-15347704 ] Apache Spark commented on SPARK-16181: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-16164) CombineFilters should keep the ordering in the logical plan

2016-06-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347701#comment-15347701 ] Cheng Lian commented on SPARK-16164: I'm posting a summary of our offline and GitHub discussion about

[jira] [Updated] (SPARK-15784) Add Power Iteration Clustering to spark.ml

2016-06-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15784: -- Target Version/s: 2.1.0 > Add Power Iteration Clustering to spark.ml >

[jira] [Updated] (SPARK-16133) model loading backward compatibility for ml.feature

2016-06-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16133: -- Assignee: yuhao yang > model loading backward compatibility for ml.feature >

[jira] [Resolved] (SPARK-16133) model loading backward compatibility for ml.feature

2016-06-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16133. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by

[jira] [Assigned] (SPARK-16179) UDF explosion yielding empty dataframe fails

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16179: Assignee: (was: Apache Spark) > UDF explosion yielding empty dataframe fails >

[jira] [Commented] (SPARK-16179) UDF explosion yielding empty dataframe fails

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347692#comment-15347692 ] Apache Spark commented on SPARK-16179: -- User 'davies' has created a pull request for this issue:

[jira] [Resolved] (SPARK-16142) Group naive Bayes methods in generated doc

2016-06-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16142. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by

[jira] [Updated] (SPARK-16177) model loading backward compatibility for ml.regression

2016-06-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16177: -- Assignee: yuhao yang > model loading backward compatibility for ml.regression >

[jira] [Resolved] (SPARK-16177) model loading backward compatibility for ml.regression

2016-06-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16177. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by

[jira] [Created] (SPARK-16183) Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql

2016-06-23 Thread Matthew Porter (JIRA)
Matthew Porter created SPARK-16183: -- Summary: Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql Key: SPARK-16183 URL: https://issues.apache.org/jira/browse/SPARK-16183

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-06-23 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347576#comment-15347576 ] Yin Huai commented on SPARK-15345: -- [~m1lan] Do you use hive-site.xml to store the url to your

[jira] [Commented] (SPARK-16168) Spark sql can not read ORC table

2016-06-23 Thread AnfengYuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347566#comment-15347566 ] AnfengYuan commented on SPARK-16168: Ok, to make it clear, I can reproduce it with following steps:

[jira] [Comment Edited] (SPARK-14172) Hive table partition predicate not passed down correctly

2016-06-23 Thread Pedro Osorio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347556#comment-15347556 ] Pedro Osorio edited comment on SPARK-14172 at 6/24/16 2:10 AM: --- [~lucasmf]

[jira] [Commented] (SPARK-14172) Hive table partition predicate not passed down correctly

2016-06-23 Thread Pedro Osorio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347556#comment-15347556 ] Pedro Osorio commented on SPARK-14172: -- [~lucasmf] I have been able to reproduce the issue by

[jira] [Resolved] (SPARK-16123) Avoid NegativeArraySizeException while reserving additional capacity in VectorizedColumnReader

2016-06-23 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16123. --- Resolution: Resolved Assignee: Sameer Agarwal > Avoid

[jira] [Created] (SPARK-16182) companion object's shutdownhook not being invoked

2016-06-23 Thread Christian Chua (JIRA)
Christian Chua created SPARK-16182: -- Summary: companion object's shutdownhook not being invoked Key: SPARK-16182 URL: https://issues.apache.org/jira/browse/SPARK-16182 Project: Spark Issue

[jira] [Commented] (SPARK-16181) Incorrect behavior for isNull filter

2016-06-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347509#comment-15347509 ] Reynold Xin commented on SPARK-16181: - cc [~cloud_fan] > Incorrect behavior for isNull filter >

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-06-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347497#comment-15347497 ] Wenchen Fan commented on SPARK-15345: - hi, can you open a new JIRA for the remaining problem? thanks!

[jira] [Assigned] (SPARK-3723) DecisionTree, RandomForest: Add more instrumentation

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3723: --- Assignee: (was: Apache Spark) > DecisionTree, RandomForest: Add more instrumentation >

[jira] [Assigned] (SPARK-3723) DecisionTree, RandomForest: Add more instrumentation

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3723: --- Assignee: Apache Spark > DecisionTree, RandomForest: Add more instrumentation >

[jira] [Commented] (SPARK-3723) DecisionTree, RandomForest: Add more instrumentation

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347489#comment-15347489 ] Apache Spark commented on SPARK-3723: - User 'smurching' has created a pull request for this issue:

[jira] [Created] (SPARK-16181) Incorrect behavior for isNull filter

2016-06-23 Thread Kevin Chen (JIRA)
Kevin Chen created SPARK-16181: -- Summary: Incorrect behavior for isNull filter Key: SPARK-16181 URL: https://issues.apache.org/jira/browse/SPARK-16181 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16168) Spark sql can not read ORC table

2016-06-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347481#comment-15347481 ] Hyukjin Kwon commented on SPARK-16168: -- I think it would be nicer if there are some cores to

[jira] [Closed] (SPARK-16146) Spark application failed by Yarn preempting

2016-06-23 Thread Cong Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Feng closed SPARK-16146. - Resolution: Fixed > Spark application failed by Yarn preempting >

[jira] [Commented] (SPARK-16146) Spark application failed by Yarn preempting

2016-06-23 Thread Cong Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347479#comment-15347479 ] Cong Feng commented on SPARK-16146: --- Finally it turned out to be the Zeppelin issue, we run the same

[jira] [Commented] (SPARK-16172) SQL Context's

2016-06-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347477#comment-15347477 ] Hyukjin Kwon commented on SPARK-16172: -- Do you mind updating the title to be in more details and

[jira] [Commented] (SPARK-15516) Schema merging in driver fails for parquet when merging LongType and IntegerType

2016-06-23 Thread MIN-FU YANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347465#comment-15347465 ] MIN-FU YANG commented on SPARK-15516: - OK, I've reproduced the problem with following code: ```

[jira] [Issue Comment Deleted] (SPARK-15516) Schema merging in driver fails for parquet when merging LongType and IntegerType

2016-06-23 Thread MIN-FU YANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MIN-FU YANG updated SPARK-15516: Comment: was deleted (was: I would like to look into this issue) > Schema merging in driver fails

[jira] [Assigned] (SPARK-16178) SQL - Hive writer should not require partition names to match table partitions

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16178: Assignee: (was: Apache Spark) > SQL - Hive writer should not require partition names

[jira] [Commented] (SPARK-16178) SQL - Hive writer should not require partition names to match table partitions

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347462#comment-15347462 ] Apache Spark commented on SPARK-16178: -- User 'rdblue' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16178) SQL - Hive writer should not require partition names to match table partitions

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16178: Assignee: Apache Spark > SQL - Hive writer should not require partition names to match

[jira] [Updated] (SPARK-16180) Task hang on fetching blocks (cached RDD)

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16180: --- Description: Here is the stackdump of executor: {code} sun.misc.Unsafe.park(Native Method)

[jira] [Updated] (SPARK-16180) Task hang on fetching blocks (cached RDD)

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16180: --- Affects Version/s: 1.6.1 > Task hang on fetching blocks (cached RDD) >

[jira] [Created] (SPARK-16180) Task hang on fetching blocks (cached RDD)

2016-06-23 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16180: -- Summary: Task hang on fetching blocks (cached RDD) Key: SPARK-16180 URL: https://issues.apache.org/jira/browse/SPARK-16180 Project: Spark Issue Type:

[jira] [Commented] (SPARK-15516) Schema merging in driver fails for parquet when merging LongType and IntegerType

2016-06-23 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347454#comment-15347454 ] Hossein Falaki commented on SPARK-15516: Here is an example: {code} create table mytable using

[jira] [Created] (SPARK-16179) UDF explosion yielding empty dataframe fails

2016-06-23 Thread Vladimir Feinberg (JIRA)
Vladimir Feinberg created SPARK-16179: - Summary: UDF explosion yielding empty dataframe fails Key: SPARK-16179 URL: https://issues.apache.org/jira/browse/SPARK-16179 Project: Spark Issue

[jira] [Comment Edited] (SPARK-14351) Optimize ImpurityAggregator for decision trees

2016-06-23 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335165#comment-15335165 ] Manoj Kumar edited comment on SPARK-14351 at 6/23/16 11:43 PM: --- OK, so here

[jira] [Comment Edited] (SPARK-14351) Optimize ImpurityAggregator for decision trees

2016-06-23 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335165#comment-15335165 ] Manoj Kumar edited comment on SPARK-14351 at 6/23/16 11:43 PM: --- OK, so here

[jira] [Created] (SPARK-16178) SQL - Hive writer should not require partition names to match table partitions

2016-06-23 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-16178: - Summary: SQL - Hive writer should not require partition names to match table partitions Key: SPARK-16178 URL: https://issues.apache.org/jira/browse/SPARK-16178 Project:

[jira] [Commented] (SPARK-15516) Schema merging in driver fails for parquet when merging LongType and IntegerType

2016-06-23 Thread MIN-FU YANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347428#comment-15347428 ] MIN-FU YANG commented on SPARK-15516: - Do you have any sample code? > Schema merging in driver fails

[jira] [Resolved] (SPARK-16165) Fix the update logic for InMemoryTableScanExec.readBatches accumulator

2016-06-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16165. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Updated] (SPARK-16165) Fix the update logic for InMemoryTableScanExec.readBatches accumulator

2016-06-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16165: --- Assignee: Dongjoon Hyun > Fix the update logic for InMemoryTableScanExec.readBatches accumulator >

[jira] [Updated] (SPARK-16174) Improve OptimizeIn optimizer to remove deterministic repetitions

2016-06-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16174: -- Description: This issue improves `OptimizeIn` optimizer to remove the deterministic

[jira] [Updated] (SPARK-15663) SparkSession.catalog.listFunctions shouldn't include the list of built-in functions

2016-06-23 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-15663: - Labels: release_notes releasenotes (was: ) > SparkSession.catalog.listFunctions shouldn't include the

[jira] [Assigned] (SPARK-16177) model loading backward compatibility for ml.regression

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16177: Assignee: Apache Spark > model loading backward compatibility for ml.regression >

[jira] [Commented] (SPARK-16177) model loading backward compatibility for ml.regression

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347399#comment-15347399 ] Apache Spark commented on SPARK-16177: -- User 'hhbyyh' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16177) model loading backward compatibility for ml.regression

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16177: Assignee: (was: Apache Spark) > model loading backward compatibility for

[jira] [Comment Edited] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-06-23 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347172#comment-15347172 ] Ryan Blue edited comment on SPARK-16032 at 6/23/16 10:59 PM: - bq. I am not

[jira] [Updated] (SPARK-15443) Properly explain the streaming queries

2016-06-23 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-15443: - Fix Version/s: (was: 2.0.0) 2.1.0 2.0.1 > Properly explain the

[jira] [Resolved] (SPARK-15443) Properly explain the streaming queries

2016-06-23 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-15443. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13815

[jira] [Created] (SPARK-16177) model loading backward compatibility for ml.regression

2016-06-23 Thread yuhao yang (JIRA)
yuhao yang created SPARK-16177: -- Summary: model loading backward compatibility for ml.regression Key: SPARK-16177 URL: https://issues.apache.org/jira/browse/SPARK-16177 Project: Spark Issue

[jira] [Resolved] (SPARK-15854) Spark History server gets null pointer exception

2016-06-23 Thread Yesha Vora (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora resolved SPARK-15854. Resolution: Cannot Reproduce > Spark History server gets null pointer exception >

[jira] [Commented] (SPARK-15955) Failed Spark application returns with exitcode equals to zero

2016-06-23 Thread Yesha Vora (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347360#comment-15347360 ] Yesha Vora commented on SPARK-15955: [~sowen], I'm checking exit code of the process which started

[jira] [Commented] (SPARK-13288) [1.6.0] Memory leak in Spark streaming

2016-06-23 Thread roberto hashioka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347355#comment-15347355 ] roberto hashioka commented on SPARK-13288: -- Do you really need to create multiple streams with

[jira] [Commented] (SPARK-13288) [1.6.0] Memory leak in Spark streaming

2016-06-23 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347338#comment-15347338 ] JESSE CHEN commented on SPARK-13288: [~AlexSparkJiang]The code is: val multiTweetStreams=(1 to

[jira] [Updated] (SPARK-16164) CombineFilters should keep the ordering in the logical plan

2016-06-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16164: -- Summary: CombineFilters should keep the ordering in the logical plan (was: Filter pushdown

[jira] [Created] (SPARK-16176) model loading backward compatibility for ml.recommendation

2016-06-23 Thread yuhao yang (JIRA)
yuhao yang created SPARK-16176: -- Summary: model loading backward compatibility for ml.recommendation Key: SPARK-16176 URL: https://issues.apache.org/jira/browse/SPARK-16176 Project: Spark Issue

[jira] [Resolved] (SPARK-16164) Filter pushdown should keep the ordering in the logical plan

2016-06-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16164. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by

[jira] [Updated] (SPARK-16164) Filter pushdown should keep the ordering in the logical plan

2016-06-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16164: -- Assignee: Dongjoon Hyun > Filter pushdown should keep the ordering in the logical plan >

[jira] [Commented] (SPARK-11744) bin/pyspark --version doesn't return version and exit

2016-06-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347297#comment-15347297 ] Nicholas Chammas commented on SPARK-11744: -- This is not the appropriate place to ask random

[jira] [Assigned] (SPARK-16175) Handle None for all Python UDT

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16175: Assignee: (was: Apache Spark) > Handle None for all Python UDT >

[jira] [Updated] (SPARK-16175) Handle None for all Python UDT

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-16175: --- Affects Version/s: 2.0.0 1.6.1 > Handle None for all Python UDT >

[jira] [Assigned] (SPARK-16175) Handle None for all Python UDT

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16175: Assignee: Apache Spark > Handle None for all Python UDT > --

[jira] [Commented] (SPARK-16175) Handle None for all Python UDT

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347287#comment-15347287 ] Apache Spark commented on SPARK-16175: -- User 'davies' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-23 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347255#comment-15347255 ] Bo Meng edited comment on SPARK-16173 at 6/23/16 9:58 PM: -- Use the latest

[jira] [Updated] (SPARK-16004) Improve CatalogTable information

2016-06-23 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Meng updated SPARK-16004: Description: A few issues found when running "describe extended | formatted [tableName]" command: 1. The

[jira] [Updated] (SPARK-15606) Driver hang in o.a.s.DistributedSuite on 2 core machine

2016-06-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-15606: - Fix Version/s: (was: 1.6.2) 1.6.3 > Driver hang in o.a.s.DistributedSuite

[jira] [Commented] (SPARK-13288) [1.6.0] Memory leak in Spark streaming

2016-06-23 Thread roberto hashioka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347265#comment-15347265 ] roberto hashioka commented on SPARK-13288: -- I'm using the createDirectStream. > [1.6.0] Memory

[jira] [Comment Edited] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-23 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347255#comment-15347255 ] Bo Meng edited comment on SPARK-16173 at 6/23/16 9:38 PM: -- Use the latest

[jira] [Comment Edited] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-23 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347255#comment-15347255 ] Bo Meng edited comment on SPARK-16173 at 6/23/16 9:37 PM: -- Use the latest

[jira] [Commented] (SPARK-16173) Can't join describe() of DataFrame in Scala 2.10

2016-06-23 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347255#comment-15347255 ] Bo Meng commented on SPARK-16173: - Use the latest master, I was not able to reproduce, here is my code:

[jira] [Commented] (SPARK-15565) The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

2016-06-23 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347247#comment-15347247 ] Bikas Saha commented on SPARK-15565: SPARK-15565 is pulled into Erie Spark2 by the last refresh. So I

[jira] [Issue Comment Deleted] (SPARK-15565) The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

2016-06-23 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated SPARK-15565: --- Comment: was deleted (was: SPARK-15565 is pulled into Erie Spark2 by the last refresh. So I am

[jira] [Comment Edited] (SPARK-15565) The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

2016-06-23 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347247#comment-15347247 ] Bikas Saha edited comment on SPARK-15565 at 6/23/16 9:34 PM: - SPARK-15565 is

[jira] [Updated] (SPARK-16175) Handle None for all Python UDT

2016-06-23 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-16175: -- Attachment: nullvector.dbc databricks nb demonstrating the issue > Handle None for

[jira] [Created] (SPARK-16175) Handle None for all Python UDT

2016-06-23 Thread Davies Liu (JIRA)
Davies Liu created SPARK-16175: -- Summary: Handle None for all Python UDT Key: SPARK-16175 URL: https://issues.apache.org/jira/browse/SPARK-16175 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-13697) TransformFunctionSerializer.loads doesn't restore the function's module name if it's '__main__'

2016-06-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-13697: - Fix Version/s: (was: 1.6.1) 1.6.2 > TransformFunctionSerializer.loads

[jira] [Commented] (SPARK-16142) Group naive Bayes methods in generated doc

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347205#comment-15347205 ] Apache Spark commented on SPARK-16142: -- User 'mengxr' has created a pull request for this issue:

[jira] [Updated] (SPARK-13601) Invoke task failure callbacks before calling outputstream.close()

2016-06-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-13601: - Fix Version/s: 1.6.2 > Invoke task failure callbacks before calling outputstream.close() >

[jira] [Updated] (SPARK-13465) Add a task failure listener to TaskContext

2016-06-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-13465: - Fix Version/s: 1.6.2 > Add a task failure listener to TaskContext >

[jira] [Commented] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-06-23 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347172#comment-15347172 ] Ryan Blue commented on SPARK-16032: --- bq. I am not sure apply by-name resolution just to partition

[jira] [Commented] (SPARK-9478) Add class weights to Random Forest

2016-06-23 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347138#comment-15347138 ] Seth Hendrickson commented on SPARK-9478: - [~mengxr] Thanks for your feedback. Originally I did

[jira] [Commented] (SPARK-16163) Statistics of logical plan is super slow on large query

2016-06-23 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347136#comment-15347136 ] Davies Liu commented on SPARK-16163: [~srowen] yes, thanks for correct it. > Statistics of logical

[jira] [Assigned] (SPARK-16174) Add RemoveLiteralRepetitionFromIn optimizer

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16174: Assignee: (was: Apache Spark) > Add RemoveLiteralRepetitionFromIn optimizer >

[jira] [Assigned] (SPARK-16174) Add RemoveLiteralRepetitionFromIn optimizer

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16174: Assignee: Apache Spark > Add RemoveLiteralRepetitionFromIn optimizer >

[jira] [Commented] (SPARK-16174) Add RemoveLiteralRepetitionFromIn optimizer

2016-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347131#comment-15347131 ] Apache Spark commented on SPARK-16174: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-15069) GSoC 2016: Exposing more R and Python APIs for MLlib

2016-06-23 Thread Kai Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347121#comment-15347121 ] Kai Jiang commented on SPARK-15069: --- *6/22/2016 - Week 5* To-do items * Keep investigating more

[jira] [Created] (SPARK-16174) Add RemoveLiteralRepetitionFromIn optimizer

2016-06-23 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-16174: - Summary: Add RemoveLiteralRepetitionFromIn optimizer Key: SPARK-16174 URL: https://issues.apache.org/jira/browse/SPARK-16174 Project: Spark Issue Type:

[jira] [Commented] (SPARK-13216) Spark streaming application not honoring --num-executors in restarting of an application from a checkpoint

2016-06-23 Thread Neelesh Srinivas Salian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347067#comment-15347067 ] Neelesh Srinivas Salian commented on SPARK-13216: - [~srowen] are we moving forward with

[jira] [Updated] (SPARK-15725) Dynamic allocation hangs YARN app when executors time out

2016-06-23 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-15725: -- Fix Version/s: (was: 2.0.0) 2.0.1 > Dynamic allocation hangs YARN app

[jira] [Updated] (SPARK-13723) YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true

2016-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13723: -- Fix Version/s: (was: 2.0.0) 2.0.1 > YARN - Change behavior of --num-executors

[jira] [Updated] (SPARK-15230) Back quoted column with dot in it fails when running distinct on dataframe

2016-06-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15230: Component/s: (was: Examples) SQL > Back quoted column with dot in it fails

  1   2   3   >