[jira] [Created] (SPARK-20365) Inaccurate classpath format for AM and Containers

2017-04-18 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-20365: --- Summary: Inaccurate classpath format for AM and Containers Key: SPARK-20365 URL: https://issues.apache.org/jira/browse/SPARK-20365 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-20286) dynamicAllocation.executorIdleTimeout is ignored after unpersist

2017-04-18 Thread Umesh Chaudhary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972231#comment-15972231 ] Umesh Chaudhary commented on SPARK-20286: - Yep, +1 to the UI changes. However, I

[jira] [Updated] (SPARK-20365) Not so accurate classpath format for AM and Containers

2017-04-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-20365: Summary: Not so accurate classpath format for AM and Containers (was: Inaccurate classpath format

[jira] [Created] (SPARK-20366) Fix recursive join reordering: inside joins are not reordered

2017-04-18 Thread Zhenhua Wang (JIRA)
Zhenhua Wang created SPARK-20366: Summary: Fix recursive join reordering: inside joins are not reordered Key: SPARK-20366 URL: https://issues.apache.org/jira/browse/SPARK-20366 Project: Spark

[jira] [Assigned] (SPARK-20366) Fix recursive join reordering: inside joins are not reordered

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20366: Assignee: Apache Spark > Fix recursive join reordering: inside joins are not reordered > -

[jira] [Commented] (SPARK-20366) Fix recursive join reordering: inside joins are not reordered

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972243#comment-15972243 ] Apache Spark commented on SPARK-20366: -- User 'wzhfy' has created a pull request for

[jira] [Assigned] (SPARK-20366) Fix recursive join reordering: inside joins are not reordered

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20366: Assignee: (was: Apache Spark) > Fix recursive join reordering: inside joins are not re

[jira] [Updated] (SPARK-20366) Fix recursive join reordering: inside joins are not reordered

2017-04-18 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20366: - Description: If a plan has multi-level successive joins, e.g.: ``` Join / \

[jira] [Updated] (SPARK-20366) Fix recursive join reordering: inside joins are not reordered

2017-04-18 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20366: - Description: If a plan has multi-level successive joins, e.g.: Join / \

[jira] [Commented] (SPARK-20320) AnalysisException: Columns of grouping_id (count(value#17L)) does not match grouping columns (count(value#17L))

2017-04-18 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972249#comment-15972249 ] Jacek Laskowski commented on SPARK-20320: - I'm playing with Spark SQL and multi-d

[jira] [Commented] (SPARK-20286) dynamicAllocation.executorIdleTimeout is ignored after unpersist

2017-04-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-20286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972253#comment-15972253 ] Miguel Pérez commented on SPARK-20286: -- Thank you! I'll check it again and close the

[jira] [Updated] (SPARK-20366) Fix recursive join reordering: inside joins are not reordered

2017-04-18 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-20366: - Description: If a plan has multi-level successive joins, e.g.: {noformat} Join

[jira] [Commented] (SPARK-20169) Groupby Bug with Sparksql

2017-04-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972270#comment-15972270 ] Hyukjin Kwon commented on SPARK-20169: -- Yea, I was confused too when I tried to repr

[jira] [Comment Edited] (SPARK-20169) Groupby Bug with Sparksql

2017-04-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972270#comment-15972270 ] Hyukjin Kwon edited comment on SPARK-20169 at 4/18/17 7:50 AM:

[jira] [Commented] (SPARK-20169) Groupby Bug with Sparksql

2017-04-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972300#comment-15972300 ] Takeshi Yamamuro commented on SPARK-20169: -- oh, ... good work. > Groupby Bug wi

[jira] [Commented] (SPARK-20174) Analyzer gives mysterious AnalysisException when posexplode used in withColumn

2017-04-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972359#comment-15972359 ] Takeshi Yamamuro commented on SPARK-20174: -- You could fix this like https://git

[jira] [Comment Edited] (SPARK-20174) Analyzer gives mysterious AnalysisException when posexplode used in withColumn

2017-04-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972359#comment-15972359 ] Takeshi Yamamuro edited comment on SPARK-20174 at 4/18/17 9:09 AM:

[jira] [Commented] (SPARK-19995) Using real user to connect HiveMetastore in HiveClientImpl

2017-04-18 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972373#comment-15972373 ] meiyoula commented on SPARK-19995: -- Will the token be expired? > Using real user to con

[jira] [Resolved] (SPARK-20363) sessionstate.get is get the same object in hive project, when I use spark-beeline

2017-04-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20363. --- Resolution: Invalid This is very unclear. Add a comment if you can significantly clarify what this i

[jira] [Comment Edited] (SPARK-20364) Parquet predicate pushdown on columns with dots return empty results

2017-04-18 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972400#comment-15972400 ] Robert Kruszewski edited comment on SPARK-20364 at 4/18/17 9:36 AM: ---

[jira] [Commented] (SPARK-20169) Groupby Bug with Sparksql

2017-04-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972401#comment-15972401 ] Takeshi Yamamuro commented on SPARK-20169: -- I also could reproduce this on /bin/

[jira] [Commented] (SPARK-20364) Parquet predicate pushdown on columns with dots return empty results

2017-04-18 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972400#comment-15972400 ] Robert Kruszewski commented on SPARK-20364: --- Looks like parquet doesn't differe

[jira] [Resolved] (SPARK-20361) JVM locale affects SQL type names

2017-04-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20361. --- Resolution: Duplicate > JVM locale affects SQL type names > -- > >

[jira] [Reopened] (SPARK-20361) JVM locale affects SQL type names

2017-04-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-20361: --- > JVM locale affects SQL type names > -- > > Key: SPARK-

[jira] [Resolved] (SPARK-20344) Duplicate call in FairSchedulableBuilder.addTaskSetManager

2017-04-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20344. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17647 [https://github.co

[jira] [Assigned] (SPARK-20344) Duplicate call in FairSchedulableBuilder.addTaskSetManager

2017-04-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-20344: - Assignee: Robert Stupp > Duplicate call in FairSchedulableBuilder.addTaskSetManager > --

[jira] [Commented] (SPARK-1548) Add Partial Random Forest algorithm to MLlib

2017-04-18 Thread Mohamed Baddar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972474#comment-15972474 ] Mohamed Baddar commented on SPARK-1548: --- [~srowen] [~josephkb] any updates on the po

[jira] [Commented] (SPARK-20343) SBT master build for Hadoop 2.6 in Jenkins fails due to Avro version resolution

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972490#comment-15972490 ] Apache Spark commented on SPARK-20343: -- User 'HyukjinKwon' has created a pull reques

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Ed Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972544#comment-15972544 ] Ed Lee commented on SPARK-20356: really quite dangerous bug > Spark sql group by returns

[jira] [Resolved] (SPARK-20366) Fix recursive join reordering: inside joins are not reordered

2017-04-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20366. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17668 [https://githu

[jira] [Assigned] (SPARK-20366) Fix recursive join reordering: inside joins are not reordered

2017-04-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20366: --- Assignee: Zhenhua Wang > Fix recursive join reordering: inside joins are not reordered > ---

[jira] [Commented] (SPARK-6509) MDLP discretizer

2017-04-18 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972744#comment-15972744 ] Barry Becker commented on SPARK-6509: - As further proof of relevance, I will be giving

[jira] [Commented] (SPARK-20281) Table-valued function range in SQL should use the same number of partitions as spark.range

2017-04-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972756#comment-15972756 ] Takeshi Yamamuro commented on SPARK-20281: -- IIUC they internally use the same va

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972753#comment-15972753 ] Herman van Hovell commented on SPARK-20356: --- Here is a reproduction in scala: {

[jira] [Assigned] (SPARK-20281) Table-valued function range in SQL should use the same number of partitions as spark.range

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20281: Assignee: (was: Apache Spark) > Table-valued function range in SQL should use the same

[jira] [Commented] (SPARK-20281) Table-valued function range in SQL should use the same number of partitions as spark.range

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972758#comment-15972758 ] Apache Spark commented on SPARK-20281: -- User 'maropu' has created a pull request for

[jira] [Assigned] (SPARK-20281) Table-valued function range in SQL should use the same number of partitions as spark.range

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20281: Assignee: Apache Spark > Table-valued function range in SQL should use the same number of

[jira] [Created] (SPARK-20367) Spark silently escapes partition column names

2017-04-18 Thread Juliusz Sompolski (JIRA)
Juliusz Sompolski created SPARK-20367: - Summary: Spark silently escapes partition column names Key: SPARK-20367 URL: https://issues.apache.org/jira/browse/SPARK-20367 Project: Spark Issue

[jira] [Commented] (SPARK-6509) MDLP discretizer

2017-04-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972806#comment-15972806 ] Sergio Ramírez commented on SPARK-6509: --- Thanks again Barry for your support. I hope

[jira] [Commented] (SPARK-20343) SBT master build for Hadoop 2.6 in Jenkins fails due to Avro version resolution

2017-04-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972842#comment-15972842 ] Hyukjin Kwon commented on SPARK-20343: -- Please let me know if anyone is able to repr

[jira] [Commented] (SPARK-20367) Spark silently escapes partition column names

2017-04-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972850#comment-15972850 ] Hyukjin Kwon commented on SPARK-20367: -- I guess probably this is not a CSV datasourc

[jira] [Commented] (SPARK-20367) Spark silently escapes partition column names

2017-04-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972865#comment-15972865 ] Hyukjin Kwon commented on SPARK-20367: -- Actually, I did while trying to reproduce th

[jira] [Commented] (SPARK-20367) Spark silently escapes partition column names

2017-04-18 Thread Juliusz Sompolski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972867#comment-15972867 ] Juliusz Sompolski commented on SPARK-20367: --- Hi [~hyukjin.kwon]. I tested also

[jira] [Commented] (SPARK-20367) Spark silently escapes partition column names

2017-04-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972871#comment-15972871 ] Hyukjin Kwon commented on SPARK-20367: -- Doh. I rushed reading ... > Spark silently

[jira] [Created] (SPARK-20368) Support Sentry on PySpark workers

2017-04-18 Thread Alexander Shorin (JIRA)
Alexander Shorin created SPARK-20368: Summary: Support Sentry on PySpark workers Key: SPARK-20368 URL: https://issues.apache.org/jira/browse/SPARK-20368 Project: Spark Issue Type: New Fea

[jira] [Created] (SPARK-20369) pyspark: Dynamic configuration with SparkConf does not work

2017-04-18 Thread Matthew McClain (JIRA)
Matthew McClain created SPARK-20369: --- Summary: pyspark: Dynamic configuration with SparkConf does not work Key: SPARK-20369 URL: https://issues.apache.org/jira/browse/SPARK-20369 Project: Spark

[jira] [Assigned] (SPARK-20368) Support Sentry on PySpark workers

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20368: Assignee: Apache Spark > Support Sentry on PySpark workers > -

[jira] [Assigned] (SPARK-20368) Support Sentry on PySpark workers

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20368: Assignee: (was: Apache Spark) > Support Sentry on PySpark workers > --

[jira] [Commented] (SPARK-20368) Support Sentry on PySpark workers

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972914#comment-15972914 ] Apache Spark commented on SPARK-20368: -- User 'kxepal' has created a pull request for

[jira] [Created] (SPARK-20370) create external table on read only location fails

2017-04-18 Thread Gaurav Shah (JIRA)
Gaurav Shah created SPARK-20370: --- Summary: create external table on read only location fails Key: SPARK-20370 URL: https://issues.apache.org/jira/browse/SPARK-20370 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-15816) SQL server based on Postgres protocol

2017-04-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972971#comment-15972971 ] Takeshi Yamamuro commented on SPARK-15816: -- I put the prototype a little forward

[jira] [Created] (SPARK-20371) R wrappers for collect_list and collect_set

2017-04-18 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20371: -- Summary: R wrappers for collect_list and collect_set Key: SPARK-20371 URL: https://issues.apache.org/jira/browse/SPARK-20371 Project: Spark Issue

[jira] [Resolved] (SPARK-20354) When I request access to the 'http: //ip:port/api/v1/applications' link, return 'sparkUser' is empty in REST API.

2017-04-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-20354. Resolution: Fixed Assignee: guoxiaolongzte Fix Version/s: 2.2.0 > When I re

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Job Evers-Meltzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973100#comment-15973100 ] Job Evers-Meltzer commented on SPARK-20356: --- Commit 5ed397baa758c29c54a853d3f8f

[jira] [Commented] (SPARK-20263) create empty dataframes in sparkR

2017-04-18 Thread Ott Toomet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973157#comment-15973157 ] Ott Toomet commented on SPARK-20263: for instance, by rbinding in a loop. I have sev

[jira] [Comment Edited] (SPARK-20263) create empty dataframes in sparkR

2017-04-18 Thread Ott Toomet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973157#comment-15973157 ] Ott Toomet edited comment on SPARK-20263 at 4/18/17 5:52 PM: -

[jira] [Comment Edited] (SPARK-20263) create empty dataframes in sparkR

2017-04-18 Thread Ott Toomet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973157#comment-15973157 ] Ott Toomet edited comment on SPARK-20263 at 4/18/17 5:51 PM: -

[jira] [Assigned] (SPARK-20371) R wrappers for collect_list and collect_set

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20371: Assignee: (was: Apache Spark) > R wrappers for collect_list and collect_set >

[jira] [Assigned] (SPARK-20371) R wrappers for collect_list and collect_set

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20371: Assignee: Apache Spark > R wrappers for collect_list and collect_set > ---

[jira] [Updated] (SPARK-20367) Spark silently escapes partition column names

2017-04-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20367: - Component/s: (was: Spark Core) SQL > Spark silently escapes partition column

[jira] [Updated] (SPARK-20370) create external table on read only location fails

2017-04-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20370: - Component/s: (was: Spark Core) SQL > create external table on read only loca

[jira] [Commented] (SPARK-20371) R wrappers for collect_list and collect_set

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973165#comment-15973165 ] Apache Spark commented on SPARK-20371: -- User 'zero323' has created a pull request fo

[jira] [Created] (SPARK-20372) Word2Vec Continuous Bag Of Words model

2017-04-18 Thread Shubham Chopra (JIRA)
Shubham Chopra created SPARK-20372: -- Summary: Word2Vec Continuous Bag Of Words model Key: SPARK-20372 URL: https://issues.apache.org/jira/browse/SPARK-20372 Project: Spark Issue Type: Improv

[jira] [Updated] (SPARK-20371) R wrappers for collect_list and collect_set

2017-04-18 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-20371: --- Affects Version/s: 2.0.0 > R wrappers for collect_list and collect_set >

[jira] [Assigned] (SPARK-20372) Word2Vec Continuous Bag Of Words model

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20372: Assignee: Apache Spark > Word2Vec Continuous Bag Of Words model >

[jira] [Assigned] (SPARK-20372) Word2Vec Continuous Bag Of Words model

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20372: Assignee: (was: Apache Spark) > Word2Vec Continuous Bag Of Words model > -

[jira] [Commented] (SPARK-20372) Word2Vec Continuous Bag Of Words model

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973187#comment-15973187 ] Apache Spark commented on SPARK-20372: -- User 'shubhamchopra' has created a pull requ

[jira] [Updated] (SPARK-19968) Use a cached instance of KafkaProducer for writing to kafka via KafkaSink.

2017-04-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19968: - Target Version/s: 2.3.0 (was: 2.2.0) > Use a cached instance of KafkaProducer for writing to kaf

[jira] [Commented] (SPARK-20341) Support BigIngeger values > 19 precision

2017-04-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973227#comment-15973227 ] Kazuaki Ishizaki commented on SPARK-20341: -- Master branch also throws an excepti

[jira] [Commented] (SPARK-20341) Support BigIngeger values > 19 precision

2017-04-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973343#comment-15973343 ] Kazuaki Ishizaki commented on SPARK-20341: -- This exception is thrown since {{Big

[jira] [Resolved] (SPARK-20360) Create repr functions for interpreters to use

2017-04-18 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-20360. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17662 [https://github.com/ap

[jira] [Assigned] (SPARK-20360) Create repr functions for interpreters to use

2017-04-18 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-20360: --- Assignee: Kyle Kelley > Create repr functions for interpreters to use >

[jira] [Created] (SPARK-20373) Batch queries with 'Dataset/DataFrame.withWatermark()` does not execute

2017-04-18 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-20373: - Summary: Batch queries with 'Dataset/DataFrame.withWatermark()` does not execute Key: SPARK-20373 URL: https://issues.apache.org/jira/browse/SPARK-20373 Project: Sp

[jira] [Created] (SPARK-20374) Encoder generated using Java beans causes corruption in MapGroupsWithState

2017-04-18 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-20374: - Summary: Encoder generated using Java beans causes corruption in MapGroupsWithState Key: SPARK-20374 URL: https://issues.apache.org/jira/browse/SPARK-20374 Project:

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973455#comment-15973455 ] Dilip Biswal commented on SPARK-20356: -- [~viirya] [~hvanhovell] [~cloud_fan] [~smile

[jira] [Comment Edited] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973455#comment-15973455 ] Dilip Biswal edited comment on SPARK-20356 at 4/18/17 8:46 PM:

[jira] [Comment Edited] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973455#comment-15973455 ] Dilip Biswal edited comment on SPARK-20356 at 4/18/17 8:47 PM:

[jira] [Updated] (SPARK-20208) Document R fpGrowth support in vignettes, programming guide and code example

2017-04-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20208: -- Issue Type: Documentation (was: Bug) > Document R fpGrowth support in vignettes, progr

[jira] [Updated] (SPARK-20192) SparkR 2.2.0 release note

2017-04-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20192: -- Issue Type: Documentation (was: Bug) > SparkR 2.2.0 release note > ---

[jira] [Commented] (SPARK-20015) Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide

2017-04-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973461#comment-15973461 ] Joseph K. Bradley commented on SPARK-20015: --- [~felixcheung] I see several R iss

[jira] [Updated] (SPARK-20374) Encoder generated using Java beans causes corruption in MapGroupsWithState

2017-04-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-20374: -- Description: Running the example https://github.com/apache/spark/blob/master/examples/src/main

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Ryan Stack (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973497#comment-15973497 ] Ryan Stack commented on SPARK-20036: I had the same issue. Removing the explicit dep

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973507#comment-15973507 ] Cody Koeninger commented on SPARK-20036: I'll submit a PR to add a note to the do

[jira] [Created] (SPARK-20375) R wrappers for array and map

2017-04-18 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20375: -- Summary: R wrappers for array and map Key: SPARK-20375 URL: https://issues.apache.org/jira/browse/SPARK-20375 Project: Spark Issue Type: Improvem

[jira] [Assigned] (SPARK-20375) R wrappers for array and map

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20375: Assignee: (was: Apache Spark) > R wrappers for array and map > ---

[jira] [Commented] (SPARK-20375) R wrappers for array and map

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973524#comment-15973524 ] Apache Spark commented on SPARK-20375: -- User 'zero323' has created a pull request fo

[jira] [Assigned] (SPARK-20375) R wrappers for array and map

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20375: Assignee: Apache Spark > R wrappers for array and map > > >

[jira] [Assigned] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20036: Assignee: (was: Apache Spark) > impossible to read a whole kafka topic using kafka 0.1

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973538#comment-15973538 ] Apache Spark commented on SPARK-20036: -- User 'koeninger' has created a pull request

[jira] [Assigned] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20036: Assignee: Apache Spark > impossible to read a whole kafka topic using kafka 0.10 and spark

[jira] [Commented] (SPARK-20153) Support Multiple aws credentials in order to access multiple Hive on S3 table in spark application

2017-04-18 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973541#comment-15973541 ] Steve Loughran commented on SPARK-20153: [~tafra...@gmail.com] : thanks for disco

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Daniel Nuriyev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973544#comment-15973544 ] Daniel Nuriyev commented on SPARK-20036: I placed the explicit dependencies in my

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973554#comment-15973554 ] Cody Koeninger commented on SPARK-20036: [~danielnuriyev] what actually happened

[jira] [Updated] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-04-18 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-12297: - Description: Spark copied Hive's behavior for parquet, but this was inconsistent with other file

[jira] [Closed] (SPARK-20374) Encoder generated using Java beans causes corruption in MapGroupsWithState

2017-04-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das closed SPARK-20374. - Resolution: Duplicate > Encoder generated using Java beans causes corruption in MapGroupsWithStat

[jira] [Created] (SPARK-20376) Make StateStoreProvider plugable

2017-04-18 Thread Yogesh Mahajan (JIRA)
Yogesh Mahajan created SPARK-20376: -- Summary: Make StateStoreProvider plugable Key: SPARK-20376 URL: https://issues.apache.org/jira/browse/SPARK-20376 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-20377) Fix JavaStructuredSessionization example

2017-04-18 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-20377: - Summary: Fix JavaStructuredSessionization example Key: SPARK-20377 URL: https://issues.apache.org/jira/browse/SPARK-20377 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-20377) Fix JavaStructuredSessionization example

2017-04-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-20377: -- Description: Extra accessors in java bean class causes incorrect encoder generation. This led t

[jira] [Updated] (SPARK-20378) StreamSinkProvider should provide schema in createSink.

2017-04-18 Thread Yogesh Mahajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogesh Mahajan updated SPARK-20378: --- Description: We have our own Sink implementation based on our in memory store and this sink

[jira] [Created] (SPARK-20378) StreamSinkProvider should provide schema in createSink.

2017-04-18 Thread Yogesh Mahajan (JIRA)
Yogesh Mahajan created SPARK-20378: -- Summary: StreamSinkProvider should provide schema in createSink. Key: SPARK-20378 URL: https://issues.apache.org/jira/browse/SPARK-20378 Project: Spark

  1   2   >