[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Adrian Ionescu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358510#comment-15358510 ] Adrian Ionescu commented on SPARK-16329: Wow, you guys are moving fast :) Thanks!

[jira] [Resolved] (SPARK-16332) the history server of spark2.0-preview (may-24 build) consumes more than 1000% cpu

2016-06-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16332. --- Resolution: Duplicate Target Version/s: (was: 2.0.1) (Don't set target version please) >

[jira] [Commented] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

2016-06-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358508#comment-15358508 ] Sean Owen commented on SPARK-16333: --- No, I'm asking you to take a look at both files, a

[jira] [Updated] (SPARK-16341) [SQL] In regexp_replace function column and/or column expression should also allowed as replacement.

2016-06-30 Thread Mukul Garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Garg updated SPARK-16341: --- Summary: [SQL] In regexp_replace function column and/or column expression should also allowed as repl

[jira] [Updated] (SPARK-16340) In regexp_replace function column and/or column expression should also allowed as replacement.

2016-06-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16340: -- Priority: Minor (was: Critical) Fix Version/s: (was: 1.6.3) > In regexp_replace function

[jira] [Created] (SPARK-16340) In regexp_replace function column and/or column expression should also allowed as replacement.

2016-06-30 Thread Mukul Garg (JIRA)
Mukul Garg created SPARK-16340: -- Summary: In regexp_replace function column and/or column expression should also allowed as replacement. Key: SPARK-16340 URL: https://issues.apache.org/jira/browse/SPARK-16340

[jira] [Created] (SPARK-16341) In regexp_replace function column and/or column expression should also allowed as replacement.

2016-06-30 Thread Mukul Garg (JIRA)
Mukul Garg created SPARK-16341: -- Summary: In regexp_replace function column and/or column expression should also allowed as replacement. Key: SPARK-16341 URL: https://issues.apache.org/jira/browse/SPARK-16341

[jira] [Commented] (SPARK-16339) ScriptTransform does not print stderr when outstream is lost

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358473#comment-15358473 ] Apache Spark commented on SPARK-16339: -- User 'tejasapatil' has created a pull reques

[jira] [Assigned] (SPARK-16339) ScriptTransform does not print stderr when outstream is lost

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16339: Assignee: (was: Apache Spark) > ScriptTransform does not print stderr when outstream i

[jira] [Assigned] (SPARK-16339) ScriptTransform does not print stderr when outstream is lost

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16339: Assignee: Apache Spark > ScriptTransform does not print stderr when outstream is lost > --

[jira] [Created] (SPARK-16339) ScriptTransform does not print stderr when outstream is lost

2016-06-30 Thread Tejas Patil (JIRA)
Tejas Patil created SPARK-16339: --- Summary: ScriptTransform does not print stderr when outstream is lost Key: SPARK-16339 URL: https://issues.apache.org/jira/browse/SPARK-16339 Project: Spark I

[jira] [Commented] (SPARK-16144) Add a separate Rd for ML generic methods: read.ml, write.ml, summary, predict

2016-06-30 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358460#comment-15358460 ] Yanbo Liang commented on SPARK-16144: - Should we rename {{summary(model)}} to {{summa

[jira] [Commented] (SPARK-16311) Improve metadata refresh

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358456#comment-15358456 ] Apache Spark commented on SPARK-16311: -- User 'rxin' has created a pull request for t

[jira] [Commented] (SPARK-16281) Implement parse_url SQL function

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358423#comment-15358423 ] Apache Spark commented on SPARK-16281: -- User 'janplus' has created a pull request fo

[jira] [Commented] (SPARK-16334) [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

2016-06-30 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358424#comment-15358424 ] Liwei Lin commented on SPARK-16334: --- hi [~epahomov], by which tool were your parquet fi

[jira] [Resolved] (SPARK-16331) [SQL] Reduce code generation time

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16331. - Resolution: Fixed Assignee: Hiroshi Inoue Fix Version/s: 2.1.0 > [SQL] Reduce cod

[jira] [Closed] (SPARK-16247) Using pyspark dataframe with pipeline and cross validator

2016-06-30 Thread Edward Ma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ma closed SPARK-16247. - Misusage. Resolved. > Using pyspark dataframe with pipeline and cross validator > --

[jira] [Assigned] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16329: Assignee: Apache Spark > select * from temp_table_no_cols fails >

[jira] [Assigned] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16329: Assignee: (was: Apache Spark) > select * from temp_table_no_cols fails > -

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358306#comment-15358306 ] Apache Spark commented on SPARK-16329: -- User 'gatorsmile' has created a pull request

[jira] [Resolved] (SPARK-14608) transformSchema needs better documentation

2016-06-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14608. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12384 [h

[jira] [Updated] (SPARK-15820) Add Catalog.refreshTable into python API

2016-06-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15820: --- Fix Version/s: (was: 2.0.0) 2.1.0 2.0.1 > Add Catalog.refre

[jira] [Updated] (SPARK-15820) Add Catalog.refreshTable into python API

2016-06-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15820: --- Assignee: Weichen Xu > Add Catalog.refreshTable into python API > ---

[jira] [Resolved] (SPARK-15820) Add Catalog.refreshTable into python API

2016-06-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15820. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13558 [https://github.

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358236#comment-15358236 ] Takeshi Yamamuro commented on SPARK-16329: -- okay, thanks! > select * from temp_

[jira] [Commented] (SPARK-16317) Add file filtering interface for FileFormat

2016-06-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358235#comment-15358235 ] Takeshi Yamamuro commented on SPARK-16317: -- Does this intend a hadoop PathFilter

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358230#comment-15358230 ] Xiao Li commented on SPARK-16329: - If we support Dataframe with zero column, I think we s

[jira] [Resolved] (SPARK-15954) TestHive has issues being used in PySpark

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15954. - Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 2.0.0 > TestHive has issue

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358226#comment-15358226 ] Xiao Li commented on SPARK-16329: - We might hit multiple issues for supporting tables wit

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358221#comment-15358221 ] Takeshi Yamamuro commented on SPARK-16329: -- I found there is the similar issue i

[jira] [Updated] (SPARK-14138) Generated SpecificColumnarIterator code can exceed JVM size limit for cached DataFrames

2016-06-30 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-14138: - Fix Version/s: 1.6.2 > Generated SpecificColumnarIterator code can exceed JVM size limit for cach

[jira] [Comment Edited] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358212#comment-15358212 ] Takeshi Yamamuro edited comment on SPARK-16329 at 7/1/16 1:49 AM: -

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358212#comment-15358212 ] Takeshi Yamamuro commented on SPARK-16329: -- I also checked in mysql; {code} mysq

[jira] [Commented] (SPARK-15643) ML 2.0 QA: migration guide update

2016-06-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358118#comment-15358118 ] Joseph K. Bradley commented on SPARK-15643: --- I just resolved this, but let me k

[jira] [Resolved] (SPARK-15643) ML 2.0 QA: migration guide update

2016-06-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-15643. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13924 [h

[jira] [Resolved] (SPARK-16328) Implement conversion utility functions for single instances in Python

2016-06-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-16328. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13997 [h

[jira] [Resolved] (SPARK-16276) Implement elt SQL function

2016-06-30 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16276. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13966 [https://githu

[jira] [Updated] (SPARK-16276) Implement elt SQL function

2016-06-30 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16276: Assignee: Peter Lee > Implement elt SQL function > -- > > K

[jira] [Resolved] (SPARK-16313) Spark should not silently drop exceptions in file listing

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16313. - Resolution: Fixed Fix Version/s: 2.0.0 > Spark should not silently drop exceptions in file

[jira] [Resolved] (SPARK-16336) Suggest doing table refresh when encountering FileNotFoundException at runtime

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16336. - Resolution: Fixed Assignee: Peter Lee Fix Version/s: 2.0.0 > Suggest doing table

[jira] [Commented] (SPARK-13015) Replace example code in mllib-data-types.md using include_example

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358028#comment-15358028 ] Apache Spark commented on SPARK-13015: -- User 'yinxusen' has created a pull request f

[jira] [Commented] (SPARK-15954) TestHive has issues being used in PySpark

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358008#comment-15358008 ] Apache Spark commented on SPARK-15954: -- User 'rxin' has created a pull request for t

[jira] [Commented] (SPARK-16286) Implement stack table generating function

2016-06-30 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357983#comment-15357983 ] Dongjoon Hyun commented on SPARK-16286: --- Thank you! :) > Implement stack table gen

[jira] [Updated] (SPARK-16338) Streaming driver running on standalone cluster mode with supervise goes into bad state when application is killed from the UI

2016-06-30 Thread Rohit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Agarwal updated SPARK-16338: -- Attachment: error Attached the file with the driver log for a couple of batch durations. > Str

[jira] [Created] (SPARK-16338) Streaming driver running on standalone cluster mode with supervise goes into bad state when application is killed from the UI

2016-06-30 Thread Rohit Agarwal (JIRA)
Rohit Agarwal created SPARK-16338: - Summary: Streaming driver running on standalone cluster mode with supervise goes into bad state when application is killed from the UI Key: SPARK-16338 URL: https://issues.apach

[jira] [Commented] (SPARK-16286) Implement stack table generating function

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357972#comment-15357972 ] Reynold Xin commented on SPARK-16286: - Go for it! > Implement stack table generatin

[jira] [Commented] (SPARK-16286) Implement stack table generating function

2016-06-30 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357970#comment-15357970 ] Dongjoon Hyun commented on SPARK-16286: --- Hi, [~petermaxlee] and [~rxin]. If you do

[jira] [Updated] (SPARK-16208) Add `PropagateEmptyRelation` optimizer

2016-06-30 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16208: -- Priority: Major (was: Minor) Description: This issue adds a new logical optimizer, `Pro

[jira] [Assigned] (SPARK-16285) Implement sentences SQL function

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16285: Assignee: (was: Apache Spark) > Implement sentences SQL function > ---

[jira] [Commented] (SPARK-16285) Implement sentences SQL function

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357958#comment-15357958 ] Apache Spark commented on SPARK-16285: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-16285) Implement sentences SQL function

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16285: Assignee: Apache Spark > Implement sentences SQL function > --

[jira] [Assigned] (SPARK-16336) Suggest doing table refresh when encountering FileNotFoundException at runtime

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16336: Assignee: Apache Spark > Suggest doing table refresh when encountering FileNotFoundExcepti

[jira] [Commented] (SPARK-16336) Suggest doing table refresh when encountering FileNotFoundException at runtime

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357926#comment-15357926 ] Apache Spark commented on SPARK-16336: -- User 'petermaxlee' has created a pull reques

[jira] [Assigned] (SPARK-16336) Suggest doing table refresh when encountering FileNotFoundException at runtime

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16336: Assignee: (was: Apache Spark) > Suggest doing table refresh when encountering FileNotF

[jira] [Created] (SPARK-16337) Metadata refresh should work on temporary views

2016-06-30 Thread Peter Lee (JIRA)
Peter Lee created SPARK-16337: - Summary: Metadata refresh should work on temporary views Key: SPARK-16337 URL: https://issues.apache.org/jira/browse/SPARK-16337 Project: Spark Issue Type: Sub-tas

[jira] [Created] (SPARK-16336) Suggest doing table refresh when encountering FileNotFoundException at runtime

2016-06-30 Thread Peter Lee (JIRA)
Peter Lee created SPARK-16336: - Summary: Suggest doing table refresh when encountering FileNotFoundException at runtime Key: SPARK-16336 URL: https://issues.apache.org/jira/browse/SPARK-16336 Project: Spa

[jira] [Comment Edited] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357910#comment-15357910 ] Xiao Li edited comment on SPARK-16329 at 6/30/16 9:47 PM: -- I see

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357910#comment-15357910 ] Xiao Li commented on SPARK-16329: - I see. Just FYI, I tried it in DB2. db2 => create t

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357897#comment-15357897 ] Reynold Xin commented on SPARK-16329: - Hmmm I tend to like Postgres more :) It's a r

[jira] [Updated] (SPARK-16335) Structured streaming should fail if source directory does not exist

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16335: Summary: Structured streaming should fail if source directory does not exist (was: Streaming sourc

[jira] [Assigned] (SPARK-16335) Streaming source should fail if file does not exist

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16335: Assignee: Apache Spark (was: Reynold Xin) > Streaming source should fail if file does not

[jira] [Updated] (SPARK-16335) Streaming source should fail if file does not exist

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16335: Description: In structured streaming, Spark does not report errors when the specified directory do

[jira] [Assigned] (SPARK-16335) Streaming source should fail if file does not exist

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16335: Assignee: Reynold Xin (was: Apache Spark) > Streaming source should fail if file does not

[jira] [Commented] (SPARK-16335) Streaming source should fail if file does not exist

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357876#comment-15357876 ] Apache Spark commented on SPARK-16335: -- User 'rxin' has created a pull request for t

[jira] [Created] (SPARK-16335) Streaming source should fail if file does not exist

2016-06-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16335: --- Summary: Streaming source should fail if file does not exist Key: SPARK-16335 URL: https://issues.apache.org/jira/browse/SPARK-16335 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4240) Refine Tree Predictions in Gradient Boosting to Improve Prediction Accuracy.

2016-06-30 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357868#comment-15357868 ] Vladimir Feinberg commented on SPARK-4240: -- [~sethah] Hi Seth, it seems like your

[jira] [Created] (SPARK-16334) [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

2016-06-30 Thread Egor Pahomov (JIRA)
Egor Pahomov created SPARK-16334: Summary: [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException Key: SPARK-16334 URL: https://issues.apache.org/jira/browse/SPARK-16334 Project: Sp

[jira] [Updated] (SPARK-16334) [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

2016-06-30 Thread Egor Pahomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Egor Pahomov updated SPARK-16334: - Labels: sql (was: ) > [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

[jira] [Commented] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

2016-06-30 Thread Peter Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357855#comment-15357855 ] Peter Liu commented on SPARK-16333: --- is there anyway to upload the file (gzip the 5GB)?

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357845#comment-15357845 ] Takeshi Yamamuro commented on SPARK-16329: -- Tables with no columns make less sen

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357831#comment-15357831 ] Xiao Li commented on SPARK-16329: - In Hive, we are unable to create a table with 0 column

[jira] [Commented] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

2016-06-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357829#comment-15357829 ] Sean Owen commented on SPARK-16333: --- Likely related. Are you certain it's the same prog

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357826#comment-15357826 ] Takeshi Yamamuro commented on SPARK-16329: -- One idea to fix this is to follow th

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357824#comment-15357824 ] Xiao Li commented on SPARK-16329: - nvm, thank you for your confirmation! > select * from

[jira] [Commented] (SPARK-16256) Add Structured Streaming Programming Guide

2016-06-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357825#comment-15357825 ] Apache Spark commented on SPARK-16256: -- User 'tdas' has created a pull request for t

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357817#comment-15357817 ] Takeshi Yamamuro commented on SPARK-16329: -- Oh, my bad. {code} val rddNoCols = s

[jira] [Comment Edited] (SPARK-16332) the history server of spark2.0-preview (may-24 build) consumes more than 1000% cpu

2016-06-30 Thread Peter Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357811#comment-15357811 ] Peter Liu edited comment on SPARK-16332 at 6/30/16 8:41 PM: I

[jira] [Commented] (SPARK-16332) the history server of spark2.0-preview (may-24 build) consumes more than 1000% cpu

2016-06-30 Thread Peter Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357811#comment-15357811 ] Peter Liu commented on SPARK-16332: --- I think this is likely related to issue "SPARK-163

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357804#comment-15357804 ] Xiao Li commented on SPARK-16329: - [~maropu] I can reproduce it in the master. It reports

[jira] [Issue Comment Deleted] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16329: Comment: was deleted (was: [~maropu]I can reproduce it in the master. ) > select * from temp_table_no_cols

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357803#comment-15357803 ] Xiao Li commented on SPARK-16329: - Which behavior is preferred? > select * from temp_ta

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357800#comment-15357800 ] Xiao Li commented on SPARK-16329: - [~maropu]I can reproduce it in the master. > select

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357798#comment-15357798 ] Reynold Xin commented on SPARK-16329: - We can fix 1.6. > select * from temp_table_n

[jira] [Commented] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

2016-06-30 Thread Peter Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357797#comment-15357797 ] Peter Liu commented on SPARK-16333: --- please see if this answers your question: the his

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357792#comment-15357792 ] Takeshi Yamamuro commented on SPARK-16329: -- Additional info; the result of the c

[jira] [Resolved] (SPARK-16212) code cleanup of kafka-0-8 to match review feedback on 0-10

2016-06-30 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-16212. --- Resolution: Fixed Assignee: Cody Koeninger > code cleanup of kafka-0-8 to match review

[jira] [Commented] (SPARK-16332) the history server of spark2.0-preview (may-24 build) consumes more than 1000% cpu

2016-06-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357759#comment-15357759 ] Sean Owen commented on SPARK-16332: --- I'm not sure that's an issue per se. It might cons

[jira] [Commented] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

2016-06-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357757#comment-15357757 ] Sean Owen commented on SPARK-16333: --- Can you comment on what the data is before and aft

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357758#comment-15357758 ] Xiao Li commented on SPARK-16329: - [~rxin]What do you think about this? Should we just is

[jira] [Resolved] (SPARK-16247) Using pyspark dataframe with pipeline and cross validator

2016-06-30 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16247. --- Resolution: Not A Problem > Using pyspark dataframe with pipeline and cross validator > -

[jira] [Updated] (SPARK-15352) Topology aware block replication

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15352: Assignee: Shubham Chopra > Topology aware block replication > > >

[jira] [Commented] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Adrian Ionescu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357729#comment-15357729 ] Adrian Ionescu commented on SPARK-16329: Well, this is a simplified example. In r

[jira] [Comment Edited] (SPARK-16329) select * from temp_table_no_cols fails

2016-06-30 Thread Adrian Ionescu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357729#comment-15357729 ] Adrian Ionescu edited comment on SPARK-16329 at 6/30/16 7:42 PM: --

[jira] [Updated] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

2016-06-30 Thread Peter Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Liu updated SPARK-16333: -- Summary: Excessive Spark history event/json data size (5GB each) (was: Excessive Spark history event/j

[jira] [Updated] (SPARK-16332) the history server of spark2.0-preview (may-24 build) consumes more than 1000% cpu

2016-06-30 Thread Peter Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Liu updated SPARK-16332: -- Environment: this is seen on both x86 (Intel(R) Xeon(R), E5-2699 ) and ppc platform IBM Power8 Habanero

[jira] [Created] (SPARK-16333) Excessive Spark history event/json data (5GB!)

2016-06-30 Thread Peter Liu (JIRA)
Peter Liu created SPARK-16333: - Summary: Excessive Spark history event/json data (5GB!) Key: SPARK-16333 URL: https://issues.apache.org/jira/browse/SPARK-16333 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-16332) the history server of spark2.0-preview (may-24 build) consumes more than 1000% cpu

2016-06-30 Thread Peter Liu (JIRA)
Peter Liu created SPARK-16332: - Summary: the history server of spark2.0-preview (may-24 build) consumes more than 1000% cpu Key: SPARK-16332 URL: https://issues.apache.org/jira/browse/SPARK-16332 Project:

[jira] [Resolved] (SPARK-16289) Implement posexplode table generating function

2016-06-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16289. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.0.0 > Implement posexp

[jira] [Closed] (SPARK-15069) GSoC 2016: Exposing more R and Python APIs for MLlib

2016-06-30 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-15069. - Resolution: Done > GSoC 2016: Exposing more R and Python APIs for MLlib > ---

[jira] [Resolved] (SPARK-15865) Blacklist should not result in job hanging with less than 4 executors

2016-06-30 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-15865. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13603 [https://git

  1   2   >