[jira] [Commented] (SPARK-19206) Update outdated parameter descriptions in external-kafka module

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164256#comment-16164256 ] Apache Spark commented on SPARK-19206: -- User 'Chaos-Ju' has created a pull request f

[jira] [Commented] (SPARK-21987) Spark 2.3 cannot read 2.2 event logs

2017-09-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164260#comment-16164260 ] Wenchen Fan commented on SPARK-21987: - Adding back the `SparkPlanGraph.metadata` is o

[jira] [Issue Comment Deleted] (SPARK-19206) Update outdated parameter descriptions in external-kafka module

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19206: -- Comment: was deleted (was: User 'Chaos-Ju' has created a pull request for this issue: https://github.co

[jira] [Commented] (SPARK-19206) Update outdated parameter descriptions in external-kafka module

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164266#comment-16164266 ] Apache Spark commented on SPARK-19206: -- User 'Chaos-Ju' has created a pull request f

[jira] [Resolved] (SPARK-21989) createDataset and the schema of encoder class

2017-09-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21989. -- Resolution: Invalid That's due to missing setter and getter for Java Bean. It works if both are

[jira] [Created] (SPARK-21990) QueryPlanConstraints misses some constraints can be recursively inferred

2017-09-13 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-21990: --- Summary: QueryPlanConstraints misses some constraints can be recursively inferred Key: SPARK-21990 URL: https://issues.apache.org/jira/browse/SPARK-21990 Projec

[jira] [Updated] (SPARK-21990) QueryPlanConstraints misses some constraints that can be recursively inferred

2017-09-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-21990: Summary: QueryPlanConstraints misses some constraints that can be recursively inferred (wa

[jira] [Commented] (SPARK-21990) QueryPlanConstraints misses some constraints that can be recursively inferred

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164327#comment-16164327 ] Apache Spark commented on SPARK-21990: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-21990) QueryPlanConstraints misses some constraints that can be recursively inferred

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21990: Assignee: Apache Spark > QueryPlanConstraints misses some constraints that can be recursiv

[jira] [Assigned] (SPARK-21990) QueryPlanConstraints misses some constraints that can be recursively inferred

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21990: Assignee: (was: Apache Spark) > QueryPlanConstraints misses some constraints that can

[jira] [Resolved] (SPARK-21982) Set Locale to US in order to pass UtilsSuite when your jvm Locale is not US

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21982. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19205 [https://github.co

[jira] [Assigned] (SPARK-21982) Set Locale to US in order to pass UtilsSuite when your jvm Locale is not US

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21982: - Assignee: German Schiavon Matteo > Set Locale to US in order to pass UtilsSuite when your jvm Lo

[jira] [Created] (SPARK-21991) [LAUNCHER] LauncherServer acceptConnections thread sometime dies if machine has very high load

2017-09-13 Thread Andrea Zito (JIRA)
Andrea Zito created SPARK-21991: --- Summary: [LAUNCHER] LauncherServer acceptConnections thread sometime dies if machine has very high load Key: SPARK-21991 URL: https://issues.apache.org/jira/browse/SPARK-21991

[jira] [Commented] (SPARK-21989) createDataset and the schema of encoder class

2017-09-13 Thread taiho choi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164350#comment-16164350 ] taiho choi commented on SPARK-21989: thanks! > createDataset and the schema of encod

[jira] [Updated] (SPARK-21933) Spark Streaming request more executors than excepted without DynamicAllocation

2017-09-13 Thread Congxian Qiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Congxian Qiu updated SPARK-21933: - Description: Spark Streaming applications run on Yarn without DynamicAllocation, Yarn enable HA.

[jira] [Resolved] (SPARK-21893) Put Kafka 0.8 behind a profile

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21893. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19134 [https://github.co

[jira] [Assigned] (SPARK-21893) Put Kafka 0.8 behind a profile

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21893: - Assignee: Sean Owen > Put Kafka 0.8 behind a profile > -- > >

[jira] [Assigned] (SPARK-21991) [LAUNCHER] LauncherServer acceptConnections thread sometime dies if machine has very high load

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21991: Assignee: Apache Spark > [LAUNCHER] LauncherServer acceptConnections thread sometime dies

[jira] [Assigned] (SPARK-21991) [LAUNCHER] LauncherServer acceptConnections thread sometime dies if machine has very high load

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21991: Assignee: (was: Apache Spark) > [LAUNCHER] LauncherServer acceptConnections thread som

[jira] [Commented] (SPARK-21991) [LAUNCHER] LauncherServer acceptConnections thread sometime dies if machine has very high load

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164372#comment-16164372 ] Apache Spark commented on SPARK-21991: -- User 'nivox' has created a pull request for

[jira] [Commented] (SPARK-21786) The 'spark.sql.parquet.compression.codec' configuration doesn't take effect on tables with partition field(s)

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164382#comment-16164382 ] Apache Spark commented on SPARK-21786: -- User 'fjh100456' has created a pull request

[jira] [Created] (SPARK-21992) Json read in Spark 2.2.0

2017-09-13 Thread Utkarsh Saraf (JIRA)
Utkarsh Saraf created SPARK-21992: - Summary: Json read in Spark 2.2.0 Key: SPARK-21992 URL: https://issues.apache.org/jira/browse/SPARK-21992 Project: Spark Issue Type: Bug Componen

[jira] [Resolved] (SPARK-21992) Json read in Spark 2.2.0

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21992. --- Resolution: Invalid Questions should go to the mailing list. This isn't a bug; that's not the kind o

[jira] [Commented] (SPARK-21992) Json read in Spark 2.2.0

2017-09-13 Thread Utkarsh Saraf (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164390#comment-16164390 ] Utkarsh Saraf commented on SPARK-21992: --- Thanks Sean.KIndly provide me link to mail

[jira] [Commented] (SPARK-20462) Spark-Kinesis Direct Connector

2017-09-13 Thread Gaurav Shah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164429#comment-16164429 ] Gaurav Shah commented on SPARK-20462: - Flink has implemented in similar fashion http

[jira] [Created] (SPARK-21993) Close CliSessionState in shutdown hook of SparkSQLCLIDriver

2017-09-13 Thread jin xing (JIRA)
jin xing created SPARK-21993: Summary: Close CliSessionState in shutdown hook of SparkSQLCLIDriver Key: SPARK-21993 URL: https://issues.apache.org/jira/browse/SPARK-21993 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-21993) Close CliSessionState in shutdown hook of SparkSQLCLIDriver

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21993: Assignee: Apache Spark > Close CliSessionState in shutdown hook of SparkSQLCLIDriver > ---

[jira] [Commented] (SPARK-21993) Close CliSessionState in shutdown hook of SparkSQLCLIDriver

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164447#comment-16164447 ] Apache Spark commented on SPARK-21993: -- User 'jinxing64' has created a pull request

[jira] [Assigned] (SPARK-21993) Close CliSessionState in shutdown hook of SparkSQLCLIDriver

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21993: Assignee: (was: Apache Spark) > Close CliSessionState in shutdown hook of SparkSQLCLID

[jira] [Commented] (SPARK-18608) Spark ML algorithms that check RDD cache level for internal caching double-cache data

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164519#comment-16164519 ] Apache Spark commented on SPARK-18608: -- User 'yanboliang' has created a pull request

[jira] [Resolved] (SPARK-21963) create temp file should be delete after use

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21963. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19174 [https://github.co

[jira] [Assigned] (SPARK-21963) create temp file should be delete after use

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21963: - Assignee: caoxuewen > create temp file should be delete after use >

[jira] [Comment Edited] (SPARK-11248) Spark hivethriftserver is using the wrong user to while getting HDFS permissions

2017-09-13 Thread wuchang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164559#comment-16164559 ] wuchang edited comment on SPARK-11248 at 9/13/17 12:14 PM: --- the

[jira] [Commented] (SPARK-11248) Spark hivethriftserver is using the wrong user to while getting HDFS permissions

2017-09-13 Thread wuchang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164559#comment-16164559 ] wuchang commented on SPARK-11248: - the spark thrift server have security bugs , cause the

[jira] [Created] (SPARK-21994) Spark 2.2 can not read Parquet table created by itself

2017-09-13 Thread Jurgis Pods (JIRA)
Jurgis Pods created SPARK-21994: --- Summary: Spark 2.2 can not read Parquet table created by itself Key: SPARK-21994 URL: https://issues.apache.org/jira/browse/SPARK-21994 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-21970) Do a Project Wide Sweep for Redundant Throws Declarations

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21970. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19182 [https://github.co

[jira] [Assigned] (SPARK-21970) Do a Project Wide Sweep for Redundant Throws Declarations

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21970: - Assignee: Armin Braun Issue Type: Improvement (was: Bug) > Do a Project Wide Sweep for Re

[jira] [Created] (SPARK-21995) Accessing s3 bucket from read.df

2017-09-13 Thread Himanshu Arora (JIRA)
Himanshu Arora created SPARK-21995: -- Summary: Accessing s3 bucket from read.df Key: SPARK-21995 URL: https://issues.apache.org/jira/browse/SPARK-21995 Project: Spark Issue Type: IT Help

[jira] [Resolved] (SPARK-21995) Accessing s3 bucket from read.df

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21995. --- Resolution: Invalid There's no detail here that would indicate it's a Spark issue, and this isn't wh

[jira] [Created] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
Ivan Sharamet created SPARK-21996: - Summary: Streaming ignores files with spaces in the file names Key: SPARK-21996 URL: https://issues.apache.org/jira/browse/SPARK-21996 Project: Spark Issue

[jira] [Commented] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164711#comment-16164711 ] Sean Owen commented on SPARK-21996: --- Your code doesn't show a path with a space. Are yo

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Description: I tried to stream text files from folder I noticed that files with spaces in the

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Description: I tried to stream text files from folder I noticed that files with spaces in the

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Description: I tried to stream text files from folder and noticed that files with spaces in th

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Description: I tried to stream text files from folder and noticed that files with spaces in th

[jira] [Commented] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164722#comment-16164722 ] Ivan Sharamet commented on SPARK-21996: --- Hi Sean, As I wrote in the description th

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Description: I tried to stream text files from folder and noticed that files inside this folde

[jira] [Commented] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164727#comment-16164727 ] Sean Owen commented on SPARK-21996: --- /dir does not occur in your code snippet. I'm ques

[jira] [Comment Edited] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164722#comment-16164722 ] Ivan Sharamet edited comment on SPARK-21996 at 9/13/17 2:26 PM: ---

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Description: I tried to stream text files from folder and noticed that files inside this folde

[jira] [Commented] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164741#comment-16164741 ] Ivan Sharamet commented on SPARK-21996: --- Sure, it's generic example: the folder nam

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Attachment: spark-streaming.zip Sample application to reproduce this issue. > Streaming ignore

[jira] [Comment Edited] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164780#comment-16164780 ] Ivan Sharamet edited comment on SPARK-21996 at 9/13/17 3:00 PM: ---

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Description: I tried to stream text files from folder and noticed that files inside this folde

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Description: I tried to stream text files from folder and noticed that files inside this folde

[jira] [Updated] (SPARK-21987) Spark 2.3 cannot read 2.2 event logs

2017-09-13 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-21987: --- Target Version/s: 2.3.0 > Spark 2.3 cannot read 2.2 event logs >

[jira] [Resolved] (SPARK-21980) References in grouping functions should be indexed with resolver

2017-09-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21980. - Resolution: Fixed Assignee: Feng Zhu Fix Version/s: 2.3.0 > References in grouping functi

[jira] [Updated] (SPARK-21980) References in grouping functions should be indexed with resolver

2017-09-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21980: Fix Version/s: 2.2.1 > References in grouping functions should be indexed with resolver > -

[jira] [Commented] (SPARK-4131) Support "Writing data into the filesystem from queries"

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165013#comment-16165013 ] Apache Spark commented on SPARK-4131: - User 'janewangfb' has created a pull request fo

[jira] [Commented] (SPARK-10399) Off Heap Memory Access for non-JVM libraries (C++)

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165055#comment-16165055 ] Apache Spark commented on SPARK-10399: -- User 'kiszk' has created a pull request for

[jira] [Commented] (SPARK-21514) Hive has updated with new support for S3 and InsertIntoHiveTable.scala should update also

2017-09-13 Thread Dongwook Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165057#comment-16165057 ] Dongwook Kwon commented on SPARK-21514: --- The impact of staging on S3(same FS of des

[jira] [Created] (SPARK-21997) Spark shows different results on Hive char/varchar columns

2017-09-13 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-21997: - Summary: Spark shows different results on Hive char/varchar columns Key: SPARK-21997 URL: https://issues.apache.org/jira/browse/SPARK-21997 Project: Spark

[jira] [Updated] (SPARK-21997) Spark shows different results on Hive char/varchar columns on Parquet

2017-09-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21997: -- Summary: Spark shows different results on Hive char/varchar columns on Parquet (was: Spark sho

[jira] [Updated] (SPARK-21997) Spark shows different results on Hive char/varchar columns on Parquet/ORC

2017-09-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21997: -- Description: SPARK-19459 resolves CHAR/VARCHAR issues in general, but Spark shows different re

[jira] [Updated] (SPARK-21997) Spark shows different results on Hive char/varchar columns on Parquet/ORC

2017-09-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21997: -- Summary: Spark shows different results on Hive char/varchar columns on Parquet/ORC (was: Spark

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Description: I tried to stream text files from folder and noticed that files inside this folde

[jira] [Updated] (SPARK-21996) Streaming ignores files with spaces in the file names

2017-09-13 Thread Ivan Sharamet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sharamet updated SPARK-21996: -- Description: I tried to stream text files from folder and noticed that files inside this folde

[jira] [Updated] (SPARK-21997) Spark shows different results on Hive char/varchar columns on Parquet/ORC

2017-09-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21997: -- Affects Version/s: 2.1.1 > Spark shows different results on Hive char/varchar columns on Parque

[jira] [Updated] (SPARK-21997) Spark shows different results on Hive char/varchar columns on Parquet/ORC

2017-09-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21997: -- Affects Version/s: 2.0.2 > Spark shows different results on Hive char/varchar columns on Parque

[jira] [Commented] (SPARK-21997) Spark shows different results on Hive char/varchar columns on Parquet/ORC

2017-09-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165096#comment-16165096 ] Dongjoon Hyun commented on SPARK-21997: --- Hi, [~smilegator] and [~cloud_fan]. I'm wo

[jira] [Created] (SPARK-21998) SortMergeJoinExec should calculate its outputOrdering independent of its children's outputOrdering

2017-09-13 Thread Maryann Xue (JIRA)
Maryann Xue created SPARK-21998: --- Summary: SortMergeJoinExec should calculate its outputOrdering independent of its children's outputOrdering Key: SPARK-21998 URL: https://issues.apache.org/jira/browse/SPARK-21998

[jira] [Commented] (SPARK-18591) Replace hash-based aggregates with sort-based ones if inputs already sorted

2017-09-13 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165148#comment-16165148 ] Maryann Xue commented on SPARK-18591: - Hey, I came up with this idea of doing sort-ag

[jira] [Updated] (SPARK-21997) Spark shows different results on Hive char/varchar columns on Parquet

2017-09-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21997: -- Summary: Spark shows different results on Hive char/varchar columns on Parquet (was: Spark sho

[jira] [Commented] (SPARK-21997) Spark shows different results on Hive char/varchar columns on Parquet

2017-09-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165150#comment-16165150 ] Dongjoon Hyun commented on SPARK-21997: --- I update the title to focus on Parquet fir

[jira] [Updated] (SPARK-21997) Spark shows different results on Hive char/varchar columns on Parquet

2017-09-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21997: -- Description: SPARK-19459 resolves CHAR/VARCHAR issues in general, but Spark shows different re

[jira] [Commented] (SPARK-21513) SQL to_json should support all column types

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165189#comment-16165189 ] Apache Spark commented on SPARK-21513: -- User 'goldmedal' has created a pull request

[jira] [Commented] (SPARK-20990) Multi-line support for JSON

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165340#comment-16165340 ] Apache Spark commented on SPARK-20990: -- User 'mgaido91' has created a pull request f

[jira] [Updated] (SPARK-21985) PySpark PairDeserializer is broken for double-zipped RDDs

2017-09-13 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-21985: Target Version/s: 2.1.2 > PySpark PairDeserializer is broken for double-zipped RDDs > -

[jira] [Commented] (SPARK-21998) SortMergeJoinExec should calculate its outputOrdering independent of its children's outputOrdering

2017-09-13 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165403#comment-16165403 ] Takeshi Yamamuro commented on SPARK-21998: -- I think the orders depend on their c

[jira] [Commented] (SPARK-21985) PySpark PairDeserializer is broken for double-zipped RDDs

2017-09-13 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165408#comment-16165408 ] holdenk commented on SPARK-21985: - CC [~a1ray] moving the discussion back here from githu

[jira] [Created] (SPARK-21999) ConcurrentModificationException - Error sending message [message = Heartbeat

2017-09-13 Thread Michael N (JIRA)
Michael N created SPARK-21999: - Summary: ConcurrentModificationException - Error sending message [message = Heartbeat Key: SPARK-21999 URL: https://issues.apache.org/jira/browse/SPARK-21999 Project: Spark

[jira] [Commented] (SPARK-21418) NoSuchElementException: None.get in DataSourceScanExec with sun.io.serialization.extendedDebugInfo=true

2017-09-13 Thread Dale Richardson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165448#comment-16165448 ] Dale Richardson commented on SPARK-21418: - I'm getting the same stackdump in 2.2.

[jira] [Commented] (SPARK-4131) Support "Writing data into the filesystem from queries"

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165451#comment-16165451 ] Apache Spark commented on SPARK-4131: - User 'gatorsmile' has created a pull request fo

[jira] [Comment Edited] (SPARK-21418) NoSuchElementException: None.get in DataSourceScanExec with sun.io.serialization.extendedDebugInfo=true

2017-09-13 Thread Dale Richardson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165448#comment-16165448 ] Dale Richardson edited comment on SPARK-21418 at 9/13/17 11:18 PM:

[jira] [Updated] (SPARK-21999) ConcurrentModificationException - Error sending message [message = Heartbeat

2017-09-13 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael N updated SPARK-21999: -- Description: Hi, I am using Spark Streaming v2.1.0 with Kafka 0.8. I am getting ConcurrentModificati

[jira] [Resolved] (SPARK-20427) Issue with Spark interpreting Oracle datatype NUMBER

2017-09-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20427. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 2.3.0 > Issue with Spark interpret

[jira] [Updated] (SPARK-21999) ConcurrentModificationException - Error sending message [message = Heartbeat

2017-09-13 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael N updated SPARK-21999: -- Description: Hi, I am using Spark Streaming v2.1.0 with Kafka 0.8. I am getting ConcurrentModificati

[jira] [Updated] (SPARK-21999) ConcurrentModificationException - Error sending message [message = Heartbeat

2017-09-13 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael N updated SPARK-21999: -- Description: Hi, I am using Spark Streaming v2.1.0 with Kafka 0.8. I am getting ConcurrentModificati

[jira] [Updated] (SPARK-21999) ConcurrentModificationException - Error sending message [message = Heartbeat

2017-09-13 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael N updated SPARK-21999: -- Description: Hi, I am using Spark Streaming v2.1.0 with Kafka 0.8. I am getting ConcurrentModificati

[jira] [Assigned] (SPARK-21985) PySpark PairDeserializer is broken for double-zipped RDDs

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21985: Assignee: (was: Apache Spark) > PySpark PairDeserializer is broken for double-zipped R

[jira] [Commented] (SPARK-21985) PySpark PairDeserializer is broken for double-zipped RDDs

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165600#comment-16165600 ] Apache Spark commented on SPARK-21985: -- User 'aray' has created a pull request for t

[jira] [Assigned] (SPARK-21985) PySpark PairDeserializer is broken for double-zipped RDDs

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21985: Assignee: Apache Spark > PySpark PairDeserializer is broken for double-zipped RDDs > -

[jira] [Created] (SPARK-22000) org.codehaus.commons.compiler.CompileException: toString method is not declared

2017-09-13 Thread taiho choi (JIRA)
taiho choi created SPARK-22000: -- Summary: org.codehaus.commons.compiler.CompileException: toString method is not declared Key: SPARK-22000 URL: https://issues.apache.org/jira/browse/SPARK-22000 Project:

[jira] [Commented] (SPARK-20060) Support Standalone visiting secured HDFS

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165634#comment-16165634 ] Apache Spark commented on SPARK-20060: -- User 'jerryshao' has created a pull request

[jira] [Updated] (SPARK-21990) QueryPlanConstraints misses some constraints that can be recursively inferred

2017-09-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-21990: Description: When I inspected the latest change of SPARK-21979, I found we could miss few

[jira] [Resolved] (SPARK-21990) QueryPlanConstraints misses some constraints that can be recursively inferred

2017-09-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh resolved SPARK-21990. - Resolution: Not A Problem > QueryPlanConstraints misses some constraints that can be recu

[jira] [Commented] (SPARK-21990) QueryPlanConstraints misses some constraints that can be recursively inferred

2017-09-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165639#comment-16165639 ] Liang-Chi Hsieh commented on SPARK-21990: - After inspecting it, because {{InferFi

[jira] [Commented] (SPARK-22000) org.codehaus.commons.compiler.CompileException: toString method is not declared

2017-09-13 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165667#comment-16165667 ] Takeshi Yamamuro commented on SPARK-22000: -- What's the query? > org.codehaus.co

[jira] [Commented] (SPARK-21985) PySpark PairDeserializer is broken for double-zipped RDDs

2017-09-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165671#comment-16165671 ] Apache Spark commented on SPARK-21985: -- User 'HyukjinKwon' has created a pull reques

[jira] [Commented] (SPARK-21985) PySpark PairDeserializer is broken for double-zipped RDDs

2017-09-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165672#comment-16165672 ] Hyukjin Kwon commented on SPARK-21985: -- Oh, I am sorry, [~holdenk] and [~a1ray]. I j

  1   2   >