[jira] [Updated] (SPARK-16286) Implement stack table generating function

2016-06-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16286: Summary: Implement stack table generating function (was: Implement stack SQL function) >

[jira] [Commented] (SPARK-16281) Implement parse_url SQL function

2016-06-28 Thread Peter Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354572#comment-15354572 ] Peter Lee commented on SPARK-16281: --- I can work on this one too. > Implement parse_url SQL function >

[jira] [Commented] (SPARK-16286) Implement stack SQL function

2016-06-28 Thread Peter Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354571#comment-15354571 ] Peter Lee commented on SPARK-16286: --- This is actually a table generating function. > Implement stack

[jira] [Commented] (SPARK-16277) Implement java_method SQL function

2016-06-28 Thread Peter Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354568#comment-15354568 ] Peter Lee commented on SPARK-16277: --- I can work on this. > Implement java_method SQL function >

[jira] [Commented] (SPARK-16284) Implement reflect SQL function

2016-06-28 Thread Peter Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354569#comment-15354569 ] Peter Lee commented on SPARK-16284: --- I can work on this. > Implement reflect SQL function >

[jira] [Assigned] (SPARK-16276) Implement elt SQL function

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16276: Assignee: (was: Apache Spark) > Implement elt SQL function >

[jira] [Commented] (SPARK-16276) Implement elt SQL function

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354565#comment-15354565 ] Apache Spark commented on SPARK-16276: -- User 'petermaxlee' has created a pull request for this

[jira] [Assigned] (SPARK-16276) Implement elt SQL function

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16276: Assignee: Apache Spark > Implement elt SQL function > -- > >

[jira] [Commented] (SPARK-16276) Implement elt SQL function

2016-06-28 Thread Peter Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354551#comment-15354551 ] Peter Lee commented on SPARK-16276: --- I can work on this. > Implement elt SQL function >

[jira] [Created] (SPARK-16286) Implement stack SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16286: --- Summary: Implement stack SQL function Key: SPARK-16286 URL: https://issues.apache.org/jira/browse/SPARK-16286 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16285) Implement sentences SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16285: --- Summary: Implement sentences SQL function Key: SPARK-16285 URL: https://issues.apache.org/jira/browse/SPARK-16285 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16288) Implement inline table generating function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16288: --- Summary: Implement inline table generating function Key: SPARK-16288 URL: https://issues.apache.org/jira/browse/SPARK-16288 Project: Spark Issue Type:

[jira] [Created] (SPARK-16289) Implement posexplode table generating function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16289: --- Summary: Implement posexplode table generating function Key: SPARK-16289 URL: https://issues.apache.org/jira/browse/SPARK-16289 Project: Spark Issue Type:

[jira] [Created] (SPARK-16287) Implement str_to_map SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16287: --- Summary: Implement str_to_map SQL function Key: SPARK-16287 URL: https://issues.apache.org/jira/browse/SPARK-16287 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16284) Implement reflect SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16284: --- Summary: Implement reflect SQL function Key: SPARK-16284 URL: https://issues.apache.org/jira/browse/SPARK-16284 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16283) Implement percentile_approx SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16283: --- Summary: Implement percentile_approx SQL function Key: SPARK-16283 URL: https://issues.apache.org/jira/browse/SPARK-16283 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16282) Implement percentile SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16282: --- Summary: Implement percentile SQL function Key: SPARK-16282 URL: https://issues.apache.org/jira/browse/SPARK-16282 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16281) Implement parse_url SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16281: --- Summary: Implement parse_url SQL function Key: SPARK-16281 URL: https://issues.apache.org/jira/browse/SPARK-16281 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16279) Implement map_values SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16279: --- Summary: Implement map_values SQL function Key: SPARK-16279 URL: https://issues.apache.org/jira/browse/SPARK-16279 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16278) Implement map_keys SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16278: --- Summary: Implement map_keys SQL function Key: SPARK-16278 URL: https://issues.apache.org/jira/browse/SPARK-16278 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16280) Implement histogram_numeric SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16280: --- Summary: Implement histogram_numeric SQL function Key: SPARK-16280 URL: https://issues.apache.org/jira/browse/SPARK-16280 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16276) Implement elt SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16276: --- Summary: Implement elt SQL function Key: SPARK-16276 URL: https://issues.apache.org/jira/browse/SPARK-16276 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16277) Implement java_method SQL function

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16277: --- Summary: Implement java_method SQL function Key: SPARK-16277 URL: https://issues.apache.org/jira/browse/SPARK-16277 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-16236) Add Path Option back to Load API in DataFrameReader

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354532#comment-15354532 ] Apache Spark commented on SPARK-16236: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Created] (SPARK-16275) Implement all the Hive fallback functions

2016-06-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16275: --- Summary: Implement all the Hive fallback functions Key: SPARK-16275 URL: https://issues.apache.org/jira/browse/SPARK-16275 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-16095) Yarn cluster mode should return consistent result for command line and SparkLauncher

2016-06-28 Thread Peng Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354509#comment-15354509 ] Peng Zhang commented on SPARK-16095: [~tgraves] [~sowen] I have added a test case for explaining this

[jira] [Commented] (SPARK-16274) Implement xpath_boolean

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354504#comment-15354504 ] Apache Spark commented on SPARK-16274: -- User 'petermaxlee' has created a pull request for this

[jira] [Assigned] (SPARK-16274) Implement xpath_boolean

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16274: Assignee: (was: Apache Spark) > Implement xpath_boolean > --- > >

[jira] [Assigned] (SPARK-16274) Implement xpath_boolean

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16274: Assignee: Apache Spark > Implement xpath_boolean > --- > >

[jira] [Resolved] (SPARK-16271) Implement Hive's UDFXPathUtil

2016-06-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16271. - Resolution: Fixed Assignee: Peter Lee Fix Version/s: 2.1.0 > Implement Hive's

[jira] [Commented] (SPARK-16250) Can't use escapeQuotes option in DataFrameWriter.csv()

2016-06-28 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354496#comment-15354496 ] Hyukjin Kwon commented on SPARK-16250: -- This was fixed in SPARK-16259 together. > Can't use

[jira] [Commented] (SPARK-16273) NoSuchMethodError on SparkContext.rddToPairRDDFunctions

2016-06-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354486#comment-15354486 ] Shixiong Zhu commented on SPARK-16273: -- This method has been moved into RDD object. You need to

[jira] [Closed] (SPARK-16168) Spark sql can not read ORC table

2016-06-28 Thread AnfengYuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] AnfengYuan closed SPARK-16168. -- Resolution: Duplicate > Spark sql can not read ORC table > > >

[jira] [Commented] (SPARK-16168) Spark sql can not read ORC table

2016-06-28 Thread AnfengYuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354478#comment-15354478 ] AnfengYuan commented on SPARK-16168: I'm closing this now since it is an known issue

[jira] [Comment Edited] (SPARK-16168) Spark sql can not read ORC table

2016-06-28 Thread AnfengYuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354478#comment-15354478 ] AnfengYuan edited comment on SPARK-16168 at 6/29/16 3:33 AM: - I'm closing

[jira] [Created] (SPARK-16274) Implement xpath_boolean

2016-06-28 Thread Peter Lee (JIRA)
Peter Lee created SPARK-16274: - Summary: Implement xpath_boolean Key: SPARK-16274 URL: https://issues.apache.org/jira/browse/SPARK-16274 Project: Spark Issue Type: Sub-task Components:

[jira] [Commented] (SPARK-16274) Implement xpath_boolean

2016-06-28 Thread Peter Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354465#comment-15354465 ] Peter Lee commented on SPARK-16274: --- I will have a pull request as soon as

[jira] [Created] (SPARK-16273) NoSuchMethodError on SparkContext.rddToPairRDDFunctions

2016-06-28 Thread Fu Zhouwang (JIRA)
Fu Zhouwang created SPARK-16273: --- Summary: NoSuchMethodError on SparkContext.rddToPairRDDFunctions Key: SPARK-16273 URL: https://issues.apache.org/jira/browse/SPARK-16273 Project: Spark Issue

[jira] [Updated] (SPARK-16245) model loading backward compatibility for ml.feature.PCA

2016-06-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16245: -- Assignee: Yanbo Liang > model loading backward compatibility for ml.feature.PCA >

[jira] [Resolved] (SPARK-16245) model loading backward compatibility for ml.feature.PCA

2016-06-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16245. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by

[jira] [Assigned] (SPARK-16095) Yarn cluster mode should return consistent result for command line and SparkLauncher

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16095: Assignee: (was: Apache Spark) > Yarn cluster mode should return consistent result for

[jira] [Commented] (SPARK-16095) Yarn cluster mode should return consistent result for command line and SparkLauncher

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354425#comment-15354425 ] Apache Spark commented on SPARK-16095: -- User 'renozhang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16095) Yarn cluster mode should return consistent result for command line and SparkLauncher

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16095: Assignee: Apache Spark > Yarn cluster mode should return consistent result for command

[jira] [Resolved] (SPARK-16248) Whitelist the list of Hive fallback functions

2016-06-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16248. - Resolution: Fixed Fix Version/s: 2.0.0 > Whitelist the list of Hive fallback functions >

[jira] [Commented] (SPARK-16230) Executors self-killing after being assigned tasks while still in init

2016-06-28 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354280#comment-15354280 ] Saisai Shao commented on SPARK-16230: - I'm not sure which version Spark are you using? Here is a

[jira] [Commented] (SPARK-16272) Allow configs to reference other configs, env and system properties

2016-06-28 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354211#comment-15354211 ] Marcelo Vanzin commented on SPARK-16272: BTW, I have the code mostly ready, just needs more

[jira] [Created] (SPARK-16272) Allow configs to reference other configs, env and system properties

2016-06-28 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-16272: -- Summary: Allow configs to reference other configs, env and system properties Key: SPARK-16272 URL: https://issues.apache.org/jira/browse/SPARK-16272 Project:

[jira] [Commented] (SPARK-16265) Add option to SparkSubmit to ship driver JRE to YARN

2016-06-28 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354207#comment-15354207 ] Saisai Shao commented on SPARK-16265: - If you want to run Spark on different JVM other than YARN, you

[jira] [Commented] (SPARK-16271) Implement Hive's UDFXPathUtil

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354198#comment-15354198 ] Apache Spark commented on SPARK-16271: -- User 'petermaxlee' has created a pull request for this

[jira] [Assigned] (SPARK-16271) Implement Hive's UDFXPathUtil

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16271: Assignee: Apache Spark > Implement Hive's UDFXPathUtil > - >

[jira] [Assigned] (SPARK-16271) Implement Hive's UDFXPathUtil

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16271: Assignee: (was: Apache Spark) > Implement Hive's UDFXPathUtil >

[jira] [Created] (SPARK-16271) Implement Hive's UDFXPathUtil

2016-06-28 Thread Peter Lee (JIRA)
Peter Lee created SPARK-16271: - Summary: Implement Hive's UDFXPathUtil Key: SPARK-16271 URL: https://issues.apache.org/jira/browse/SPARK-16271 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16270) Implement xpath user defined functions

2016-06-28 Thread Peter Lee (JIRA)
Peter Lee created SPARK-16270: - Summary: Implement xpath user defined functions Key: SPARK-16270 URL: https://issues.apache.org/jira/browse/SPARK-16270 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-16269) Support null handling for vectorized hashmap during hash aggregate

2016-06-28 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qifan Pu updated SPARK-16269: - External issue URL: https://github.com/apache/spark/pull/13960 > Support null handling for vectorized

[jira] [Commented] (SPARK-16269) Support null handling for vectorized hashmap during hash aggregate

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354187#comment-15354187 ] Apache Spark commented on SPARK-16269: -- User 'ooq' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16269) Support null handling for vectorized hashmap during hash aggregate

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16269: Assignee: (was: Apache Spark) > Support null handling for vectorized hashmap during

[jira] [Assigned] (SPARK-16269) Support null handling for vectorized hashmap during hash aggregate

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16269: Assignee: Apache Spark > Support null handling for vectorized hashmap during hash

[jira] [Created] (SPARK-16269) Support null handling for vectorized hashmap during hash aggregate

2016-06-28 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-16269: Summary: Support null handling for vectorized hashmap during hash aggregate Key: SPARK-16269 URL: https://issues.apache.org/jira/browse/SPARK-16269 Project: Spark

[jira] [Resolved] (SPARK-16268) SQLContext should import DataStreamReader

2016-06-28 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-16268. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13958

[jira] [Closed] (SPARK-16188) Spark sql create a lot of small files

2016-06-28 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust closed SPARK-16188. Resolution: Not A Bug This is by design and changes would likely be too disruptive. The

[jira] [Commented] (SPARK-14351) Optimize ImpurityAggregator for decision trees

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353985#comment-15353985 ] Apache Spark commented on SPARK-14351: -- User 'MechCoder' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14351) Optimize ImpurityAggregator for decision trees

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14351: Assignee: (was: Apache Spark) > Optimize ImpurityAggregator for decision trees >

[jira] [Assigned] (SPARK-14351) Optimize ImpurityAggregator for decision trees

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14351: Assignee: Apache Spark > Optimize ImpurityAggregator for decision trees >

[jira] [Commented] (SPARK-16239) SQL issues with cast from date to string around daylight savings time

2016-06-28 Thread Glen Maisey (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353984#comment-15353984 ] Glen Maisey commented on SPARK-16239: - As a layman myself it is not expected that a Date type would

[jira] [Commented] (SPARK-16235) "evaluateEachIteration" is returning wrong results when calculated for classification model.

2016-06-28 Thread Mahmoud Rawas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353979#comment-15353979 ] Mahmoud Rawas commented on SPARK-16235: --- I don't fully agree with the statement 'it has no clear

[jira] [Assigned] (SPARK-16268) SQLContext should import DataStreamReader

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16268: Assignee: Apache Spark (was: Shixiong Zhu) > SQLContext should import DataStreamReader >

[jira] [Assigned] (SPARK-16268) SQLContext should import DataStreamReader

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16268: Assignee: Shixiong Zhu (was: Apache Spark) > SQLContext should import DataStreamReader >

[jira] [Commented] (SPARK-16268) SQLContext should import DataStreamReader

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353975#comment-15353975 ] Apache Spark commented on SPARK-16268: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Created] (SPARK-16268) SQLContext should import DataStreamReader

2016-06-28 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-16268: Summary: SQLContext should import DataStreamReader Key: SPARK-16268 URL: https://issues.apache.org/jira/browse/SPARK-16268 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-14083) Analyze JVM bytecode and turn closures into Catalyst expressions

2016-06-28 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353969#comment-15353969 ] Sean Zhong edited comment on SPARK-14083 at 6/29/16 12:17 AM: -- For typed

[jira] [Commented] (SPARK-14083) Analyze JVM bytecode and turn closures into Catalyst expressions

2016-06-28 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353969#comment-15353969 ] Sean Zhong commented on SPARK-14083: For typed operation like map, it will first de-serialize

[jira] [Updated] (SPARK-13944) Separate out local linear algebra as a standalone module without Spark dependency

2016-06-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-13944: -- Description: Separate out linear algebra as a standalone module without Spark

[jira] [Commented] (SPARK-16267) Replace deprecated `CREATE TEMPORARY TABLE` from testsuites

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353939#comment-15353939 ] Apache Spark commented on SPARK-16267: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-16267) Replace deprecated `CREATE TEMPORARY TABLE` from testsuites

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16267: Assignee: Apache Spark > Replace deprecated `CREATE TEMPORARY TABLE` from testsuites >

[jira] [Assigned] (SPARK-16267) Replace deprecated `CREATE TEMPORARY TABLE` from testsuites

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16267: Assignee: (was: Apache Spark) > Replace deprecated `CREATE TEMPORARY TABLE` from

[jira] [Updated] (SPARK-16267) Replace deprecated `CREATE TEMPORARY TABLE` from testsuites

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16267: -- Description: After SPARK-15674, `DDLStrategy` prints out the following deprecation messages

[jira] [Updated] (SPARK-16267) Replace deprecate `CREATE TEMPORARY TABLE` from testsuites

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16267: -- Summary: Replace deprecate `CREATE TEMPORARY TABLE` from testsuites (was: Remove deprecate

[jira] [Created] (SPARK-16267) Remove deprecate `CREATE TEMPORARY TABLE` from testsuites

2016-06-28 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-16267: - Summary: Remove deprecate `CREATE TEMPORARY TABLE` from testsuites Key: SPARK-16267 URL: https://issues.apache.org/jira/browse/SPARK-16267 Project: Spark

[jira] [Updated] (SPARK-16267) Replace deprecated `CREATE TEMPORARY TABLE` from testsuites

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16267: -- Summary: Replace deprecated `CREATE TEMPORARY TABLE` from testsuites (was: Replace deprecate

[jira] [Commented] (SPARK-16006) Attemping to write empty DataFrame with no fields throw non-intuitive exception

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353914#comment-15353914 ] Dongjoon Hyun commented on SPARK-16006: --- Hi, [~tdas]. Could you review the PR again? > Attemping

[jira] [Resolved] (SPARK-16114) Add network word count example

2016-06-28 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-16114. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13816

[jira] [Resolved] (SPARK-16100) Aggregator fails with Tungsten error when complex types are used for results and partial sum

2016-06-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16100. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-16266) Move python DataStreamReader/Writer from pyspark.sql to pyspark.sql.streaming package

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353815#comment-15353815 ] Apache Spark commented on SPARK-16266: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16266) Move python DataStreamReader/Writer from pyspark.sql to pyspark.sql.streaming package

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16266: Assignee: Tathagata Das (was: Apache Spark) > Move python DataStreamReader/Writer from

[jira] [Assigned] (SPARK-16266) Move python DataStreamReader/Writer from pyspark.sql to pyspark.sql.streaming package

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16266: Assignee: Apache Spark (was: Tathagata Das) > Move python DataStreamReader/Writer from

[jira] [Created] (SPARK-16266) Move python DataStreamReader/Writer from pyspark.sql to pyspark.sql.streaming package

2016-06-28 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-16266: - Summary: Move python DataStreamReader/Writer from pyspark.sql to pyspark.sql.streaming package Key: SPARK-16266 URL: https://issues.apache.org/jira/browse/SPARK-16266

[jira] [Resolved] (SPARK-16236) Add Path Option back to Load API in DataFrameReader

2016-06-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16236. - Resolution: Fixed Assignee: Xiao Li Fix Version/s: 2.0.0 > Add Path Option back

[jira] [Commented] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-06-28 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353790#comment-15353790 ] Yin Huai commented on SPARK-16032: -- Thank you [~rdblue] for the detailed reply. Those are super helpful.

[jira] [Closed] (SPARK-16081) Disallow using "l" as variable name

2016-06-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-16081. --- Resolution: Won't Fix See discussion on pr https://github.com/apache/spark/pull/13915 > Disallow

[jira] [Updated] (SPARK-16174) Improve `OptimizeIn` optimizer to remove deterministic repetitions

2016-06-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16174: -- Summary: Improve `OptimizeIn` optimizer to remove deterministic repetitions (was: Improve

[jira] [Resolved] (SPARK-16175) Handle None for all Python UDT

2016-06-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16175. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13878

[jira] [Commented] (SPARK-16265) Add option to SparkSubmit to ship driver JRE to YARN

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353709#comment-15353709 ] Apache Spark commented on SPARK-16265: -- User 'andreweduffy' has created a pull request for this

[jira] [Assigned] (SPARK-16265) Add option to SparkSubmit to ship driver JRE to YARN

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16265: Assignee: Apache Spark > Add option to SparkSubmit to ship driver JRE to YARN >

[jira] [Assigned] (SPARK-16265) Add option to SparkSubmit to ship driver JRE to YARN

2016-06-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16265: Assignee: (was: Apache Spark) > Add option to SparkSubmit to ship driver JRE to YARN

[jira] [Commented] (SPARK-16264) Allow the user to use operators on the received DataFrame

2016-06-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353695#comment-15353695 ] Shixiong Zhu commented on SPARK-16264: -- Yep, probably this one will be "Won't Fix" > Allow the user

[jira] [Commented] (SPARK-16175) Handle None for all Python UDT

2016-06-28 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353684#comment-15353684 ] Josh Rosen commented on SPARK-16175: Here's a published copy of the error reproduction notebook which

[jira] [Commented] (SPARK-16265) Add option to SparkSubmit to ship driver JRE to YARN

2016-06-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353682#comment-15353682 ] Sean Owen commented on SPARK-16265: --- I don't think this is in scope for Spark, since I assume you mean

[jira] [Resolved] (SPARK-16259) Cleanup options for DataFrame reader API in Python

2016-06-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16259. - Resolution: Fixed Fix Version/s: 2.1.0 > Cleanup options for DataFrame reader API in

[jira] [Created] (SPARK-16265) Add option to SparkSubmit to ship driver JRE to YARN

2016-06-28 Thread Andrew Duffy (JIRA)
Andrew Duffy created SPARK-16265: Summary: Add option to SparkSubmit to ship driver JRE to YARN Key: SPARK-16265 URL: https://issues.apache.org/jira/browse/SPARK-16265 Project: Spark Issue

[jira] [Updated] (SPARK-16148) TaskLocation does not allow for Executor ID's with underscores

2016-06-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-16148: - Assignee: Tom Magrino > TaskLocation does not allow for Executor ID's with underscores >

  1   2   3   >