[jira] [Commented] (SPARK-20912) map function with columns as strings

2017-05-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029496#comment-16029496 ] Apache Spark commented on SPARK-20912: -- User 'liyichao' has created a pull request for this issue:

[jira] [Commented] (SPARK-20920) ForkJoinPool pools are leaked when writing hive tables with many partitions

2017-05-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029492#comment-16029492 ] Sean Owen commented on SPARK-20920: --- Yeah, we can't use a ForkJoinPool like this. It has to be started,

[jira] [Updated] (SPARK-20869) Master should clear failed apps when worker down

2017-05-30 Thread lyc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lyc updated SPARK-20869: Summary: Master should clear failed apps when worker down (was: Master may clear failed apps when worker down) >

[jira] [Commented] (SPARK-20922) Unsafe deserialization in Spark LauncherConnection

2017-05-30 Thread Aditya Sharad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029455#comment-16029455 ] Aditya Sharad commented on SPARK-20922: --- Yes, this is different from SPARK-11652, which focused on

[jira] [Commented] (SPARK-20178) Improve Scheduler fetch failures

2017-05-30 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029425#comment-16029425 ] Thomas Graves commented on SPARK-20178: --- Yeah I think we should do something here. I never looked

[jira] [Assigned] (SPARK-20916) Improve error message for unaliased subqueries in FROM clause

2017-05-30 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20916: --- Assignee: Liang-Chi Hsieh > Improve error message for unaliased subqueries in FROM clause >

[jira] [Resolved] (SPARK-20916) Improve error message for unaliased subqueries in FROM clause

2017-05-30 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20916. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18141

[jira] [Commented] (SPARK-20920) ForkJoinPool pools are leaked when writing hive tables with many partitions

2017-05-30 Thread Rares Mirica (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029393#comment-16029393 ] Rares Mirica commented on SPARK-20920: -- Yes, as per my comment, it's a related but distinct problem,

[jira] [Commented] (SPARK-20922) Unsafe deserialization in Spark LauncherConnection

2017-05-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029344#comment-16029344 ] Sean Owen commented on SPARK-20922: --- This is not the same as

[jira] [Commented] (SPARK-20799) Unable to infer schema for ORC on S3N when secrets are in the URL

2017-05-30 Thread Jork Zijlstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029297#comment-16029297 ] Jork Zijlstra commented on SPARK-20799: --- Hi [~dongjoon], Sorry that is took some time to test the

[jira] [Comment Edited] (SPARK-20799) Unable to infer schema for ORC on S3N when secrets are in the URL

2017-05-30 Thread Jork Zijlstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029297#comment-16029297 ] Jork Zijlstra edited comment on SPARK-20799 at 5/30/17 11:50 AM: - Hi

[jira] [Commented] (SPARK-20914) Javadoc contains code that is invalid

2017-05-30 Thread Cristian Teodor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029287#comment-16029287 ] Cristian Teodor commented on SPARK-20914: - I'll look and update the ticket. Until now this was

[jira] [Reopened] (SPARK-11083) insert overwrite table failed when beeline reconnect

2017-05-30 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński reopened SPARK-11083: Problem reappeared in Spark 2.1.1 > insert overwrite table failed when beeline reconnect >

[jira] [Commented] (SPARK-11083) insert overwrite table failed when beeline reconnect

2017-05-30 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029280#comment-16029280 ] Maciej Bryński commented on SPARK-11083: Same here. > insert overwrite table failed when beeline

[jira] [Commented] (SPARK-20920) ForkJoinPool pools are leaked when writing hive tables with many partitions

2017-05-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029277#comment-16029277 ] Sean Owen commented on SPARK-20920: --- Duplicate of SPARK-20848, it seems, or others related to

[jira] [Resolved] (SPARK-20816) MetricsConfig doen't trim the properties file cause the exception very confused

2017-05-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20816. --- Resolution: Not A Problem > MetricsConfig doen't trim the properties file cause the exception very

[jira] [Updated] (SPARK-20922) Unsafe deserialization in Spark LauncherConnection

2017-05-30 Thread Aditya Sharad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Sharad updated SPARK-20922: -- Description: The {{run()}} method of the class

[jira] [Issue Comment Deleted] (SPARK-20922) Unsafe deserialization in Spark LauncherConnection

2017-05-30 Thread Aditya Sharad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Sharad updated SPARK-20922: -- Comment: was deleted (was: Proof-of-concept with simple {{SparkLauncher}} application and

[jira] [Updated] (SPARK-20922) Unsafe deserialization in Spark LauncherConnection

2017-05-30 Thread Aditya Sharad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Sharad updated SPARK-20922: -- Attachment: spark-deserialize-master.zip Proof-of-concept with simple {{SparkLauncher}}

[jira] [Updated] (SPARK-20922) Unsafe deserialization in Spark LauncherConnection

2017-05-30 Thread Aditya Sharad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Sharad updated SPARK-20922: -- Description: The {{run()}} method of the class

[jira] [Created] (SPARK-20922) Unsafe deserialization in Spark LauncherConnection

2017-05-30 Thread Aditya Sharad (JIRA)
Aditya Sharad created SPARK-20922: - Summary: Unsafe deserialization in Spark LauncherConnection Key: SPARK-20922 URL: https://issues.apache.org/jira/browse/SPARK-20922 Project: Spark Issue

[jira] [Created] (SPARK-20921) While reading from oracle database, it converts to wrong type.

2017-05-30 Thread vivek dixit (JIRA)
vivek dixit created SPARK-20921: --- Summary: While reading from oracle database, it converts to wrong type. Key: SPARK-20921 URL: https://issues.apache.org/jira/browse/SPARK-20921 Project: Spark

[jira] [Commented] (SPARK-20914) Javadoc contains code that is invalid

2017-05-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029180#comment-16029180 ] Sean Owen commented on SPARK-20914: --- Agree, open a PR to fix them. Maybe do a search for similar

[jira] [Resolved] (SPARK-20913) Reference is ambiguous exception during dropDuplicates

2017-05-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20913. --- Resolution: Not A Problem JiRA isn't tech support and your urgency doesn't change that. >

[jira] [Created] (SPARK-20920) ForkJoinPool pools are leaked when writing hive tables with many partitions

2017-05-30 Thread Rares Mirica (JIRA)
Rares Mirica created SPARK-20920: Summary: ForkJoinPool pools are leaked when writing hive tables with many partitions Key: SPARK-20920 URL: https://issues.apache.org/jira/browse/SPARK-20920 Project:

[jira] [Comment Edited] (SPARK-20913) Reference is ambiguous exception during dropDuplicates

2017-05-30 Thread Saurabh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028816#comment-16028816 ] Saurabh edited comment on SPARK-20913 at 5/30/17 7:47 AM: -- Thanks Takeshi for

[jira] [Commented] (SPARK-20913) Reference is ambiguous exception during dropDuplicates

2017-05-30 Thread Saurabh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028816#comment-16028816 ] Saurabh commented on SPARK-20913: - Thanks Takeshi for your comment but sadly the response time for

[jira] [Updated] (SPARK-17689) _temporary files breaks the Spark SQL streaming job.

2017-05-30 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-17689: Affects Version/s: (was: 2.3.0) 2.2.0 > _temporary files breaks

[jira] [Updated] (SPARK-17689) _temporary files breaks the Spark SQL streaming job.

2017-05-30 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-17689: Target Version/s: 2.3.0 (was: 2.2.0) > _temporary files breaks the Spark SQL streaming

[jira] [Updated] (SPARK-17689) _temporary files breaks the Spark SQL streaming job.

2017-05-30 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-17689: Affects Version/s: 2.3.0 > _temporary files breaks the Spark SQL streaming job. >

[jira] [Resolved] (SPARK-20909) Build-in SQL Function Support - DAYOFWEEK

2017-05-30 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-20909. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18134

[jira] [Assigned] (SPARK-20919) Simplificaiton of CachedKafkaConsumer using guava cache.

2017-05-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20919: Assignee: Apache Spark > Simplificaiton of CachedKafkaConsumer using guava cache. >

[jira] [Assigned] (SPARK-20919) Simplificaiton of CachedKafkaConsumer using guava cache.

2017-05-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20919: Assignee: (was: Apache Spark) > Simplificaiton of CachedKafkaConsumer using guava

[jira] [Commented] (SPARK-20919) Simplificaiton of CachedKafkaConsumer using guava cache.

2017-05-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028780#comment-16028780 ] Apache Spark commented on SPARK-20919: -- User 'ScrapCodes' has created a pull request for this issue:

[jira] [Created] (SPARK-20919) Simplificaiton of CachedKafkaConsumer using guava cache.

2017-05-30 Thread Prashant Sharma (JIRA)
Prashant Sharma created SPARK-20919: --- Summary: Simplificaiton of CachedKafkaConsumer using guava cache. Key: SPARK-20919 URL: https://issues.apache.org/jira/browse/SPARK-20919 Project: Spark

<    1   2