[jira] [Commented] (SPARK-18430) Returned Message Null when Hitting an Invocation Exception of Function Lookup.

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669726#comment-15669726 ] Apache Spark commented on SPARK-18430: -- User 'gatorsmile' has created a pull request

[jira] [Created] (SPARK-18468) Flaky test: org.apache.spark.sql.hive.HiveSparkSubmitSuite.SPARK-9757 Persist Parquet relation with decimal column

2016-11-15 Thread Yin Huai (JIRA)
Yin Huai created SPARK-18468: Summary: Flaky test: org.apache.spark.sql.hive.HiveSparkSubmitSuite.SPARK-9757 Persist Parquet relation with decimal column Key: SPARK-18468 URL: https://issues.apache.org/jira/browse/SP

[jira] [Assigned] (SPARK-18467) Refactor StaticInvoke, Invoke and NewInstance.

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18467: Assignee: (was: Apache Spark) > Refactor StaticInvoke, Invoke and NewInstance. > -

[jira] [Commented] (SPARK-18467) Refactor StaticInvoke, Invoke and NewInstance.

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669598#comment-15669598 ] Apache Spark commented on SPARK-18467: -- User 'ueshin' has created a pull request for

[jira] [Assigned] (SPARK-18467) Refactor StaticInvoke, Invoke and NewInstance.

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18467: Assignee: Apache Spark > Refactor StaticInvoke, Invoke and NewInstance. >

[jira] [Created] (SPARK-18467) Refactor StaticInvoke, Invoke and NewInstance.

2016-11-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-18467: - Summary: Refactor StaticInvoke, Invoke and NewInstance. Key: SPARK-18467 URL: https://issues.apache.org/jira/browse/SPARK-18467 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-18464) Spark SQL fails to load tables created without providing a schema

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18464: Assignee: Apache Spark > Spark SQL fails to load tables created without providing a schema

[jira] [Commented] (SPARK-18464) Spark SQL fails to load tables created without providing a schema

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669575#comment-15669575 ] Apache Spark commented on SPARK-18464: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-18464) Spark SQL fails to load tables created without providing a schema

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18464: Assignee: (was: Apache Spark) > Spark SQL fails to load tables created without providi

[jira] [Assigned] (SPARK-18466) Missing withFilter method causes warnings when using for comprehensions

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18466: Assignee: Apache Spark > Missing withFilter method causes warnings when using for comprehe

[jira] [Assigned] (SPARK-18466) Missing withFilter method causes warnings when using for comprehensions

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18466: Assignee: (was: Apache Spark) > Missing withFilter method causes warnings when using f

[jira] [Commented] (SPARK-18466) Missing withFilter method causes warnings when using for comprehensions

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669545#comment-15669545 ] Apache Spark commented on SPARK-18466: -- User 'reggert' has created a pull request fo

[jira] [Updated] (SPARK-18466) Missing withFilter method causes warnings when using for comprehensions

2016-11-15 Thread Richard W. Eggert II (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard W. Eggert II updated SPARK-18466: - Description: The fact that the RDD class has a {{filter}} method but not a {{with

[jira] [Created] (SPARK-18466) Missing withFilter method causes warnings when using for comprehensions

2016-11-15 Thread Richard W. Eggert II (JIRA)
Richard W. Eggert II created SPARK-18466: Summary: Missing withFilter method causes warnings when using for comprehensions Key: SPARK-18466 URL: https://issues.apache.org/jira/browse/SPARK-18466

[jira] [Updated] (SPARK-18424) Improve Date Parsing Semantics & Functionality

2016-11-15 Thread Bill Chambers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Chambers updated SPARK-18424: -- Summary: Improve Date Parsing Semantics & Functionality (was: Improve Date Parsing Functionali

[jira] [Commented] (SPARK-18457) ORC and other columnar formats using HiveShim read all columns when doing a simple count

2016-11-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669369#comment-15669369 ] Hyukjin Kwon commented on SPARK-18457: -- Ah, I see {{HiveShim.appendReadColumns}} is

[jira] [Assigned] (SPARK-18457) ORC and other columnar formats using HiveShim read all columns when doing a simple count

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18457: Assignee: (was: Apache Spark) > ORC and other columnar formats using HiveShim read all

[jira] [Assigned] (SPARK-18457) ORC and other columnar formats using HiveShim read all columns when doing a simple count

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18457: Assignee: Apache Spark > ORC and other columnar formats using HiveShim read all columns wh

[jira] [Commented] (SPARK-18457) ORC and other columnar formats using HiveShim read all columns when doing a simple count

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669335#comment-15669335 ] Apache Spark commented on SPARK-18457: -- User 'aray' has created a pull request for t

[jira] [Updated] (SPARK-18377) warehouse path should be a static conf

2016-11-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18377: Labels: releasenotes (was: ) > warehouse path should be a static conf > --

[jira] [Resolved] (SPARK-18377) warehouse path should be a static conf

2016-11-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18377. - Resolution: Fixed Fix Version/s: 2.1.0 > warehouse path should be a static conf >

[jira] [Commented] (SPARK-17836) Use cross validation to determine the number of clusters for EM or KMeans algorithms

2016-11-15 Thread Lei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669182#comment-15669182 ] Lei Wang commented on SPARK-17836: -- Yes. Of course. Do you also have the same demand? >

[jira] [Comment Edited] (SPARK-18457) ORC and other columnar formats using HiveShim read all columns when doing a simple count

2016-11-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669150#comment-15669150 ] Hyukjin Kwon edited comment on SPARK-18457 at 11/16/16 2:39 AM: ---

[jira] [Comment Edited] (SPARK-18457) ORC and other columnar formats using HiveShim read all columns when doing a simple count

2016-11-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669150#comment-15669150 ] Hyukjin Kwon edited comment on SPARK-18457 at 11/16/16 2:39 AM: ---

[jira] [Commented] (SPARK-18457) ORC and other columnar formats using HiveShim read all columns when doing a simple count

2016-11-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669150#comment-15669150 ] Hyukjin Kwon commented on SPARK-18457: -- Hm, don't we push down no column when perfor

[jira] [Assigned] (SPARK-18461) Improve docs on StreamingQueryListener and StreamingQuery.status

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18461: Assignee: Tathagata Das (was: Apache Spark) > Improve docs on StreamingQueryListener and

[jira] [Commented] (SPARK-18461) Improve docs on StreamingQueryListener and StreamingQuery.status

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669128#comment-15669128 ] Apache Spark commented on SPARK-18461: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-18461) Improve docs on StreamingQueryListener and StreamingQuery.status

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18461: Assignee: Apache Spark (was: Tathagata Das) > Improve docs on StreamingQueryListener and

[jira] [Commented] (SPARK-18441) Add Smote in spark mlib and ml

2016-11-15 Thread lichenglin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669125#comment-15669125 ] lichenglin commented on SPARK-18441: Thanks ,It works now > Add Smote in spark mlib

[jira] [Commented] (SPARK-18441) Add Smote in spark mlib and ml

2016-11-15 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669110#comment-15669110 ] yuhao yang commented on SPARK-18441: It should work with Spark 2.0+. If you have modi

[jira] [Assigned] (SPARK-18465) Uncache Table shouldn't throw an exception when table doesn't exist

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18465: Assignee: Apache Spark > Uncache Table shouldn't throw an exception when table doesn't exi

[jira] [Assigned] (SPARK-18465) Uncache Table shouldn't throw an exception when table doesn't exist

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18465: Assignee: (was: Apache Spark) > Uncache Table shouldn't throw an exception when table

[jira] [Commented] (SPARK-18465) Uncache Table shouldn't throw an exception when table doesn't exist

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669104#comment-15669104 ] Apache Spark commented on SPARK-18465: -- User 'brkyvz' has created a pull request for

[jira] [Updated] (SPARK-18300) ClassCastException during count distinct

2016-11-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18300: Fix Version/s: 2.0.3 > ClassCastException during count distinct > -

[jira] [Created] (SPARK-18465) Uncache Table shouldn't throw an exception when table doesn't exist

2016-11-15 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-18465: --- Summary: Uncache Table shouldn't throw an exception when table doesn't exist Key: SPARK-18465 URL: https://issues.apache.org/jira/browse/SPARK-18465 Project: Spark

[jira] [Commented] (SPARK-18464) Spark SQL fails to load tables created without providing a schema

2016-11-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669057#comment-15669057 ] Yin Huai commented on SPARK-18464: -- cc [~cloud_fan] > Spark SQL fails to load tables cr

[jira] [Created] (SPARK-18464) Spark SQL fails to load tables created without providing a schema

2016-11-15 Thread Yin Huai (JIRA)
Yin Huai created SPARK-18464: Summary: Spark SQL fails to load tables created without providing a schema Key: SPARK-18464 URL: https://issues.apache.org/jira/browse/SPARK-18464 Project: Spark Is

[jira] [Commented] (SPARK-18441) Add Smote in spark mlib and ml

2016-11-15 Thread lichenglin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669050#comment-15669050 ] lichenglin commented on SPARK-18441: Thanks for your reply. May I ask what version of

[jira] [Commented] (SPARK-18319) ML, Graph 2.1 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-11-15 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669027#comment-15669027 ] yuhao yang commented on SPARK-18319: Experimental classes in ml: *probably ready to

[jira] [Commented] (SPARK-18462) SparkListenerDriverAccumUpdates event does not deserialize properly in history server

2016-11-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668990#comment-15668990 ] Josh Rosen commented on SPARK-18462: {code} [info] - roundtripping SparkListenerDrive

[jira] [Created] (SPARK-18463) I think it's necessary to have an overrided method of smaple

2016-11-15 Thread Jianfei Wang (JIRA)
Jianfei Wang created SPARK-18463: Summary: I think it's necessary to have an overrided method of smaple Key: SPARK-18463 URL: https://issues.apache.org/jira/browse/SPARK-18463 Project: Spark

[jira] [Created] (SPARK-18462) SparkListenerDriverAccumUpdates event does not deserialize properly in history server

2016-11-15 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-18462: -- Summary: SparkListenerDriverAccumUpdates event does not deserialize properly in history server Key: SPARK-18462 URL: https://issues.apache.org/jira/browse/SPARK-18462 Pro

[jira] [Assigned] (SPARK-18460) Include triggerDetails in StreamingQueryStatus.json

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18460: Assignee: Tathagata Das (was: Apache Spark) > Include triggerDetails in StreamingQuerySta

[jira] [Assigned] (SPARK-18460) Include triggerDetails in StreamingQueryStatus.json

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18460: Assignee: Apache Spark (was: Tathagata Das) > Include triggerDetails in StreamingQuerySta

[jira] [Created] (SPARK-18461) Improve docs on StreamingQueryListener and StreamingQuery.status

2016-11-15 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-18461: - Summary: Improve docs on StreamingQueryListener and StreamingQuery.status Key: SPARK-18461 URL: https://issues.apache.org/jira/browse/SPARK-18461 Project: Spark

[jira] [Commented] (SPARK-18460) Include triggerDetails in StreamingQueryStatus.json

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668954#comment-15668954 ] Apache Spark commented on SPARK-18460: -- User 'tdas' has created a pull request for t

[jira] [Commented] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668948#comment-15668948 ] Kazuaki Ishizaki commented on SPARK-18458: -- I work for this. > core dumped runn

[jira] [Created] (SPARK-18460) Include triggerDetails in StreamingQueryStatus.json

2016-11-15 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-18460: - Summary: Include triggerDetails in StreamingQueryStatus.json Key: SPARK-18460 URL: https://issues.apache.org/jira/browse/SPARK-18460 Project: Spark Issue T

[jira] [Assigned] (SPARK-18459) Rename triggerId to batchId in StreamingQueryStatus.triggerDetails

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18459: Assignee: Apache Spark (was: Tathagata Das) > Rename triggerId to batchId in StreamingQue

[jira] [Assigned] (SPARK-18459) Rename triggerId to batchId in StreamingQueryStatus.triggerDetails

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18459: Assignee: Tathagata Das (was: Apache Spark) > Rename triggerId to batchId in StreamingQue

[jira] [Commented] (SPARK-18459) Rename triggerId to batchId in StreamingQueryStatus.triggerDetails

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668919#comment-15668919 ] Apache Spark commented on SPARK-18459: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-18188) Add checksum for block in Spark

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18188: Assignee: Davies Liu (was: Apache Spark) > Add checksum for block in Spark >

[jira] [Commented] (SPARK-18188) Add checksum for block in Spark

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668908#comment-15668908 ] Apache Spark commented on SPARK-18188: -- User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-18188) Add checksum for block in Spark

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18188: Assignee: Apache Spark (was: Davies Liu) > Add checksum for block in Spark >

[jira] [Created] (SPARK-18459) Rename triggerId to batchId in StreamingQueryStatus.triggerDetails

2016-11-15 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-18459: - Summary: Rename triggerId to batchId in StreamingQueryStatus.triggerDetails Key: SPARK-18459 URL: https://issues.apache.org/jira/browse/SPARK-18459 Project: Spark

[jira] [Commented] (SPARK-18319) ML, Graph 2.1 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-11-15 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668892#comment-15668892 ] yuhao yang commented on SPARK-18319: sealed: org.apache.spark.ml.linalg.Matrix org.a

[jira] [Assigned] (SPARK-18440) Fix FileStreamSink with aggregation + watermark + append mode

2016-11-15 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-18440: - Assignee: Tathagata Das > Fix FileStreamSink with aggregation + watermark + append mode

[jira] [Updated] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-15 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-18458: --- Fix Version/s: (was: 2.0.0) > core dumped running Spark SQL on large data volume (100TB) > --

[jira] [Updated] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-15 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-18458: --- Description: Running a query on 100TB parquet database using the Spark master dated 11/04 dump cores

[jira] [Updated] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-15 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-18458: --- Labels: core dump (was: tpcds-result-mismatch) > core dumped running Spark SQL on large data volume

[jira] [Updated] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-15 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN updated SPARK-18458: --- Affects Version/s: (was: 1.6.0) 2.1.0 > core dumped running Spark SQL on l

[jira] [Updated] (SPARK-17732) ALTER TABLE DROP PARTITION should support comparators

2016-11-15 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17732: -- Fix Version/s: (was: 2.2.0) 2.1.0 > ALTER TABLE DROP PARTITION s

[jira] [Created] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-15 Thread JESSE CHEN (JIRA)
JESSE CHEN created SPARK-18458: -- Summary: core dumped running Spark SQL on large data volume (100TB) Key: SPARK-18458 URL: https://issues.apache.org/jira/browse/SPARK-18458 Project: Spark Issue

[jira] [Commented] (SPARK-18388) Running aggregation on many columns throws SOE

2016-11-15 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668805#comment-15668805 ] Herman van Hovell commented on SPARK-18388: --- [~bond_] Yeah this should be fixed

[jira] [Resolved] (SPARK-17732) ALTER TABLE DROP PARTITION should support comparators

2016-11-15 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17732. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.2.0 > ALTE

[jira] [Resolved] (SPARK-18423) ReceiverTracker should close checkpoint dir when stopped even if it was not started

2016-11-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-18423. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.1.0 > ReceiverTracker

[jira] [Commented] (SPARK-18319) ML, Graph 2.1 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-11-15 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668703#comment-15668703 ] yuhao yang commented on SPARK-18319: I'll start make a pass. > ML, Graph 2.1 QA: API

[jira] [Commented] (SPARK-18437) Inconsistent mark-down for `Note:`/`NOTE:`/`Note that` across Scala/Java/R/Python in API documentations

2016-11-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668697#comment-15668697 ] Hyukjin Kwon commented on SPARK-18437: -- Yup, I don't mind but let us be very sure on

[jira] [Assigned] (SPARK-18456) Use matrix abstraction for LogisitRegression coefficients during training

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18456: Assignee: (was: Apache Spark) > Use matrix abstraction for LogisitRegression coefficie

[jira] [Commented] (SPARK-18456) Use matrix abstraction for LogisitRegression coefficients during training

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668689#comment-15668689 ] Apache Spark commented on SPARK-18456: -- User 'sethah' has created a pull request for

[jira] [Resolved] (SPARK-18440) Fix FileStreamSink with aggregation + watermark + append mode

2016-11-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-18440. -- Resolution: Fixed Issue resolved by pull request 15885 [https://github.com/apache/spark

[jira] [Assigned] (SPARK-18456) Use matrix abstraction for LogisitRegression coefficients during training

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18456: Assignee: Apache Spark > Use matrix abstraction for LogisitRegression coefficients during

[jira] [Created] (SPARK-18457) ORC and other columnar formats using HiveShim read all columns when doing a simple count

2016-11-15 Thread Andrew Ray (JIRA)
Andrew Ray created SPARK-18457: -- Summary: ORC and other columnar formats using HiveShim read all columns when doing a simple count Key: SPARK-18457 URL: https://issues.apache.org/jira/browse/SPARK-18457

[jira] [Created] (SPARK-18456) Use matrix abstraction for LogisitRegression coefficients during training

2016-11-15 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-18456: Summary: Use matrix abstraction for LogisitRegression coefficients during training Key: SPARK-18456 URL: https://issues.apache.org/jira/browse/SPARK-18456 Pro

[jira] [Updated] (SPARK-18448) SparkSession should implement java.lang.AutoCloseable like JavaSparkContext

2016-11-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18448: -- Priority: Trivial (was: Major) Issue Type: Improvement (was: Bug) AutoCloseable is Java 7 so sh

[jira] [Comment Edited] (SPARK-18455) General support for subquery processing

2016-11-15 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668583#comment-15668583 ] Nattavut Sutyanyong edited comment on SPARK-18455 at 11/15/16 10:25 PM: ---

[jira] [Comment Edited] (SPARK-18455) General support for subquery processing

2016-11-15 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668577#comment-15668577 ] Nattavut Sutyanyong edited comment on SPARK-18455 at 11/15/16 10:24 PM: ---

[jira] [Commented] (SPARK-18455) General support for subquery processing

2016-11-15 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668583#comment-15668583 ] Nattavut Sutyanyong commented on SPARK-18455: - Limitations on Subquery suppor

[jira] [Updated] (SPARK-18419) Fix `JDBCOptions.asConnectionProperties` to be case-insensitive

2016-11-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18419: -- Summary: Fix `JDBCOptions.asConnectionProperties` to be case-insensitive (was: [SPARK-18419][S

[jira] [Commented] (SPARK-18455) General support for subquery processing

2016-11-15 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668577#comment-15668577 ] Nattavut Sutyanyong commented on SPARK-18455: - These are JIRAs on incorrect r

[jira] [Commented] (SPARK-18300) ClassCastException during count distinct

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668550#comment-15668550 ] Apache Spark commented on SPARK-18300: -- User 'hvanhovell' has created a pull request

[jira] [Commented] (SPARK-18455) General support for subquery processing

2016-11-15 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668521#comment-15668521 ] Nattavut Sutyanyong commented on SPARK-18455: - Two papers were discussed as p

[jira] [Created] (SPARK-18455) General support for subquery processing

2016-11-15 Thread Nattavut Sutyanyong (JIRA)
Nattavut Sutyanyong created SPARK-18455: --- Summary: General support for subquery processing Key: SPARK-18455 URL: https://issues.apache.org/jira/browse/SPARK-18455 Project: Spark Issue T

[jira] [Commented] (SPARK-18300) ClassCastException during count distinct

2016-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668465#comment-15668465 ] Apache Spark commented on SPARK-18300: -- User 'hvanhovell' has created a pull request

[jira] [Resolved] (SPARK-18337) Memory Sink should be able to recover from checkpoints in Complete OutputMode

2016-11-15 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-18337. --- Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-18392) LSH API, algorithm, and documentation follow-ups

2016-11-15 Thread Yun Ni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668372#comment-15668372 ] Yun Ni commented on SPARK-18392: Re-org the ticket dependencies for a little bit. PTAL.

[jira] [Updated] (SPARK-18453) Support Thrift UI to use SPNEGO

2016-11-15 Thread Shi Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shi Wang updated SPARK-18453: - External issue ID: (was: SPARK-18452) > Support Thrift UI to use SPNEGO > -

[jira] [Created] (SPARK-18454) Changes to fix Multi-Probe Nearest Neighbor Search for LSH

2016-11-15 Thread Yun Ni (JIRA)
Yun Ni created SPARK-18454: -- Summary: Changes to fix Multi-Probe Nearest Neighbor Search for LSH Key: SPARK-18454 URL: https://issues.apache.org/jira/browse/SPARK-18454 Project: Spark Issue Type: Im

[jira] [Updated] (SPARK-18453) Support Thrift UI to use SPNEGO

2016-11-15 Thread Shi Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shi Wang updated SPARK-18453: - External issue ID: SPARK-18452 > Support Thrift UI to use SPNEGO > --- > >

[jira] [Resolved] (SPARK-18417) Define 'spark.yarn.am.port' in yarn config object

2016-11-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-18417. Resolution: Fixed Assignee: Weiqing Yang Fix Version/s: 2.2.0 > Define 'spa

[jira] [Created] (SPARK-18453) Support Thrift UI to use SPNEGO

2016-11-15 Thread Shi Wang (JIRA)
Shi Wang created SPARK-18453: Summary: Support Thrift UI to use SPNEGO Key: SPARK-18453 URL: https://issues.apache.org/jira/browse/SPARK-18453 Project: Spark Issue Type: Task Affects Versions

[jira] [Updated] (SPARK-18453) Support Thrift UI to use SPNEGO

2016-11-15 Thread Shi Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shi Wang updated SPARK-18453: - Description: Currently almost all the hadoop component UI support SPNEGO, HADOOP, HBASE, OOIZE. SPARK U

[jira] [Updated] (SPARK-18452) Support History Server UI to use SPNEGO

2016-11-15 Thread Shi Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shi Wang updated SPARK-18452: - Description: Currently almost all the hadoop component UI support SPNEGO, HADOOP, HBASE, OOIZE. SPARK

[jira] [Updated] (SPARK-16808) History Server main page does not honor APPLICATION_WEB_PROXY_BASE

2016-11-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-16808: --- Fix Version/s: 2.0.3 > History Server main page does not honor APPLICATION_WEB_PROXY_BASE > -

[jira] [Created] (SPARK-18452) Support History Server UI to use SPNEGO

2016-11-15 Thread Shi Wang (JIRA)
Shi Wang created SPARK-18452: Summary: Support History Server UI to use SPNEGO Key: SPARK-18452 URL: https://issues.apache.org/jira/browse/SPARK-18452 Project: Spark Issue Type: Task Affects

[jira] [Created] (SPARK-18451) Always set -XX:+HeapDumpOnOutOfMemoryError for Spark tests

2016-11-15 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-18451: -- Summary: Always set -XX:+HeapDumpOnOutOfMemoryError for Spark tests Key: SPARK-18451 URL: https://issues.apache.org/jira/browse/SPARK-18451 Project: Spark Issue

[jira] [Created] (SPARK-18450) Add AND-amplification to Locality Sensitive Hashing

2016-11-15 Thread Yun Ni (JIRA)
Yun Ni created SPARK-18450: -- Summary: Add AND-amplification to Locality Sensitive Hashing Key: SPARK-18450 URL: https://issues.apache.org/jira/browse/SPARK-18450 Project: Spark Issue Type: Improveme

[jira] [Assigned] (SPARK-13027) Add API for updateStateByKey to provide batch time as input

2016-11-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-13027: Assignee: Shixiong Zhu > Add API for updateStateByKey to provide batch time as input > ---

[jira] [Updated] (SPARK-13027) Add API for updateStateByKey to provide batch time as input

2016-11-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-13027: - Assignee: Aaditya Ramesh (was: Shixiong Zhu) > Add API for updateStateByKey to provide batch tim

[jira] [Resolved] (SPARK-13027) Add API for updateStateByKey to provide batch time as input

2016-11-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-13027. -- Resolution: Fixed Fix Version/s: 2.1.0 > Add API for updateStateByKey to provide batch t

  1   2   >