[jira] [Commented] (SPARK-16452) basic INFORMATION_SCHEMA support

2016-07-11 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372261#comment-15372261 ] Dongjoon Hyun commented on SPARK-16452: --- Hi, [~rxin]. I have a question about VIEW

[jira] [Commented] (SPARK-13514) Spark Shuffle Service 1.6.0 issue in Yarn

2016-07-11 Thread Kevin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372252#comment-15372252 ] Kevin commented on SPARK-13514: --- I met this problem too these days. Are there other solutio

[jira] [Commented] (SPARK-16437) SparkR read.df() from parquet got error: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder"

2016-07-11 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372211#comment-15372211 ] Xin Ren commented on SPARK-16437: - But I still find some minor improvements during my deb

[jira] [Commented] (SPARK-16437) SparkR read.df() from parquet got error: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder"

2016-07-11 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372210#comment-15372210 ] Xin Ren commented on SPARK-16437: - I worked on this for couple days, and I found it's not

[jira] [Resolved] (SPARK-16199) Add a method to list the referenced columns in data source Filter

2016-07-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16199. - Resolution: Fixed Assignee: Peter Lee (was: Reynold Xin) Fix Version/s: 2.1.0 >

[jira] [Updated] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-07-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-12639: - Assignee: Russell Alexander Spitzer > Improve Explain for DataSources with Handled Predicate Pushdowns >

[jira] [Resolved] (SPARK-12639) Improve Explain for DataSources with Handled Predicate Pushdowns

2016-07-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-12639. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 11317 [https://github.com/

[jira] [Assigned] (SPARK-16482) If a table's schema is inferred at runtime, describe table command does not show the schema

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16482: Assignee: Apache Spark > If a table's schema is inferred at runtime, describe table comman

[jira] [Assigned] (SPARK-16482) If a table's schema is inferred at runtime, describe table command does not show the schema

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16482: Assignee: (was: Apache Spark) > If a table's schema is inferred at runtime, describe t

[jira] [Commented] (SPARK-16482) If a table's schema is inferred at runtime, describe table command does not show the schema

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372145#comment-15372145 ] Apache Spark commented on SPARK-16482: -- User 'gatorsmile' has created a pull request

[jira] [Resolved] (SPARK-15440) Add CSRF Filter for REST APIs to Spark

2016-07-11 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-15440. - Resolution: Won't Fix > Add CSRF Filter for REST APIs to Spark >

[jira] [Resolved] (SPARK-16488) Codegen variable namespace collision for pmod and partitionBy

2016-07-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16488. - Resolution: Fixed Assignee: Sameer Agarwal Fix Version/s: 2.0.0 > Codegen variabl

[jira] [Updated] (SPARK-16490) Python mllib example for chi-squared feature selector

2016-07-11 Thread Shuai Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuai Lin updated SPARK-16490: -- Labels: starter (was: ) > Python mllib example for chi-squared feature selector >

[jira] [Commented] (SPARK-16334) [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

2016-07-11 Thread Vladimir Ivanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372068#comment-15372068 ] Vladimir Ivanov commented on SPARK-16334: - Hi Herman, Thank you for reply! I wa

[jira] [Assigned] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14812: Assignee: Apache Spark (was: Joseph K. Bradley) > ML, Graph 2.0 QA: API: Experimental, De

[jira] [Commented] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372043#comment-15372043 ] Apache Spark commented on SPARK-14812: -- User 'jkbradley' has created a pull request

[jira] [Assigned] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14812: Assignee: Joseph K. Bradley (was: Apache Spark) > ML, Graph 2.0 QA: API: Experimental, De

[jira] [Assigned] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-14812: - Assignee: Joseph K. Bradley (was: DB Tsai) > ML, Graph 2.0 QA: API: Experimenta

[jira] [Created] (SPARK-16491) Crc32 should use different variable names (not "checksum")

2016-07-11 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16491: --- Summary: Crc32 should use different variable names (not "checksum") Key: SPARK-16491 URL: https://issues.apache.org/jira/browse/SPARK-16491 Project: Spark Issu

[jira] [Updated] (SPARK-16488) Codegen variable namespace collision for pmod and partitionBy

2016-07-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16488: Issue Type: Sub-task (was: Bug) Parent: SPARK-16489 > Codegen variable namespace collision

[jira] [Created] (SPARK-16490) Python mllib example for chi-squared feature selector

2016-07-11 Thread Shuai Lin (JIRA)
Shuai Lin created SPARK-16490: - Summary: Python mllib example for chi-squared feature selector Key: SPARK-16490 URL: https://issues.apache.org/jira/browse/SPARK-16490 Project: Spark Issue Type: T

[jira] [Updated] (SPARK-16489) Test harness to prevent expression code generation from reusing variable names

2016-07-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16489: Description: In code generation, it is incorrect for expressions to reuse variable names across di

[jira] [Resolved] (SPARK-16433) Improve StreamingQuery.explain when no data arrives

2016-07-11 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-16433. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 14100 [https://g

[jira] [Assigned] (SPARK-16489) Test harness to prevent expression code generation from reusing variable names

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16489: Assignee: Reynold Xin (was: Apache Spark) > Test harness to prevent expression code gener

[jira] [Commented] (SPARK-16489) Test harness to prevent expression code generation from reusing variable names

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372014#comment-15372014 ] Apache Spark commented on SPARK-16489: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-16489) Test harness to prevent expression code generation from reusing variable names

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16489: Assignee: Apache Spark (was: Reynold Xin) > Test harness to prevent expression code gener

[jira] [Created] (SPARK-16489) Test harness to prevent expression code generation from reusing variable names

2016-07-11 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16489: --- Summary: Test harness to prevent expression code generation from reusing variable names Key: SPARK-16489 URL: https://issues.apache.org/jira/browse/SPARK-16489 Project:

[jira] [Comment Edited] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371841#comment-15371841 ] Joseph K. Bradley edited comment on SPARK-14812 at 7/12/16 1:03 AM: ---

[jira] [Assigned] (SPARK-16487) Some batches might not get marked as fully processed in JobGenerator

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16487: Assignee: Apache Spark > Some batches might not get marked as fully processed in JobGenera

[jira] [Commented] (SPARK-16487) Some batches might not get marked as fully processed in JobGenerator

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372002#comment-15372002 ] Apache Spark commented on SPARK-16487: -- User 'ahmed-mahran' has created a pull reque

[jira] [Assigned] (SPARK-16487) Some batches might not get marked as fully processed in JobGenerator

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16487: Assignee: (was: Apache Spark) > Some batches might not get marked as fully processed i

[jira] [Assigned] (SPARK-16488) Codegen variable namespace collision for pmod and partitionBy

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16488: Assignee: Apache Spark > Codegen variable namespace collision for pmod and partitionBy > -

[jira] [Assigned] (SPARK-16488) Codegen variable namespace collision for pmod and partitionBy

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16488: Assignee: (was: Apache Spark) > Codegen variable namespace collision for pmod and part

[jira] [Commented] (SPARK-16488) Codegen variable namespace collision for pmod and partitionBy

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372000#comment-15372000 ] Apache Spark commented on SPARK-16488: -- User 'sameeragarwal' has created a pull requ

[jira] [Created] (SPARK-16488) Codegen variable namespace collision for pmod and partitionBy

2016-07-11 Thread Sameer Agarwal (JIRA)
Sameer Agarwal created SPARK-16488: -- Summary: Codegen variable namespace collision for pmod and partitionBy Key: SPARK-16488 URL: https://issues.apache.org/jira/browse/SPARK-16488 Project: Spark

[jira] [Updated] (SPARK-16487) Some batches might not get marked as fully processed in JobGenerator

2016-07-11 Thread Ahmed Mahran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Mahran updated SPARK-16487: - Description: In JobGenerator, the code reads like that some batches might not get marked as full

[jira] [Updated] (SPARK-16487) Some batches might not get marked as fully processed in JobGenerator

2016-07-11 Thread Ahmed Mahran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Mahran updated SPARK-16487: - Description: In JobGenerator, the code reads like that some batches might not get marked as full

[jira] [Updated] (SPARK-16487) Some batches might not get marked as fully processed in JobGenerator

2016-07-11 Thread Ahmed Mahran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Mahran updated SPARK-16487: - Description: In JobGenerator, the code reads like that some batches might not get marked as full

[jira] [Commented] (SPARK-16435) Behavior changes if initialExecutor is less than minExecutor for dynamic allocation

2016-07-11 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371975#comment-15371975 ] Saisai Shao commented on SPARK-16435: - OK, I will file a small patch to add the warni

[jira] [Created] (SPARK-16487) Some batches might not get marked as fully processed in JobGenerator

2016-07-11 Thread Ahmed Mahran (JIRA)
Ahmed Mahran created SPARK-16487: Summary: Some batches might not get marked as fully processed in JobGenerator Key: SPARK-16487 URL: https://issues.apache.org/jira/browse/SPARK-16487 Project: Spark

[jira] [Commented] (SPARK-15816) SQL server based on Postgres protocol

2016-07-11 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371952#comment-15371952 ] Takeshi Yamamuro commented on SPARK-15816: -- okay, I first take time on this prot

[jira] [Issue Comment Deleted] (SPARK-15816) SQL server based on Postgres protocol

2016-07-11 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-15816: - Comment: was deleted (was: okay, I first take time on this prototype. thanks.) > SQL ser

[jira] [Commented] (SPARK-15816) SQL server based on Postgres protocol

2016-07-11 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371950#comment-15371950 ] Takeshi Yamamuro commented on SPARK-15816: -- okay, I first take time on this prot

[jira] [Commented] (SPARK-16430) Add an option in file stream source to read 1 file at a time

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371887#comment-15371887 ] Apache Spark commented on SPARK-16430: -- User 'tdas' has created a pull request for t

[jira] [Commented] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371860#comment-15371860 ] Joseph K. Bradley commented on SPARK-14812: --- I'll start on a PR, but please let

[jira] [Resolved] (SPARK-14087) PySpark ML JavaModel does not properly own params after being fit

2016-07-11 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-14087. -- Resolution: Resolved Fix Version/s: 2.0.0 This is no longer an issue as the PySpark wrap

[jira] [Assigned] (SPARK-16439) Incorrect information in SQL Query details

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16439: Assignee: Apache Spark > Incorrect information in SQL Query details >

[jira] [Assigned] (SPARK-16439) Incorrect information in SQL Query details

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16439: Assignee: (was: Apache Spark) > Incorrect information in SQL Query details > -

[jira] [Commented] (SPARK-16439) Incorrect information in SQL Query details

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371845#comment-15371845 ] Apache Spark commented on SPARK-16439: -- User 'maver1ck' has created a pull request f

[jira] [Comment Edited] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371841#comment-15371841 ] Joseph K. Bradley edited comment on SPARK-14812 at 7/11/16 10:58 PM: --

[jira] [Comment Edited] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371841#comment-15371841 ] Joseph K. Bradley edited comment on SPARK-14812 at 7/11/16 10:57 PM: --

[jira] [Commented] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371841#comment-15371841 ] Joseph K. Bradley commented on SPARK-14812: --- [~thunterdb] has reviewed the publ

[jira] [Commented] (SPARK-6567) Large linear model parallelism via a join and reduceByKey

2016-07-11 Thread Ben McCann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371816#comment-15371816 ] Ben McCann commented on SPARK-6567: --- [~hucheng] can you share your code for this? > Lar

[jira] [Commented] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371804#comment-15371804 ] Joseph K. Bradley commented on SPARK-14812: --- First, I'll comment that I think w

[jira] [Commented] (SPARK-14812) ML, Graph 2.0 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371791#comment-15371791 ] Joseph K. Bradley commented on SPARK-14812: --- I'll make a pass over the docs now

[jira] [Commented] (SPARK-16439) Incorrect information in SQL Query details

2016-07-11 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371780#comment-15371780 ] Maciej BryƄski commented on SPARK-16439: I found that problem is locale dependent

[jira] [Updated] (SPARK-16349) IsolatedClientLoader ignores needed Hadoop classes not present in Spark's loader

2016-07-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16349: - Assignee: Marcelo Vanzin > IsolatedClientLoader ignores needed Hadoop classes not present in Spark's > l

[jira] [Resolved] (SPARK-16349) IsolatedClientLoader ignores needed Hadoop classes not present in Spark's loader

2016-07-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16349. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14020 [https://github.com/

[jira] [Assigned] (SPARK-16375) [Spark web UI]:The wrong value(numCompletedTasks) has been assigned to the variable numSkippedTasks

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16375: Assignee: (was: Apache Spark) > [Spark web UI]:The wrong value(numCompletedTasks) has

[jira] [Commented] (SPARK-16375) [Spark web UI]:The wrong value(numCompletedTasks) has been assigned to the variable numSkippedTasks

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371734#comment-15371734 ] Apache Spark commented on SPARK-16375: -- User 'ajbozarth' has created a pull request

[jira] [Assigned] (SPARK-16375) [Spark web UI]:The wrong value(numCompletedTasks) has been assigned to the variable numSkippedTasks

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16375: Assignee: Apache Spark > [Spark web UI]:The wrong value(numCompletedTasks) has been assign

[jira] [Updated] (SPARK-16486) Python API parity issues from 2.0 QA

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-16486: -- Target Version/s: (was: 2.0.0) > Python API parity issues from 2.0 QA > -

[jira] [Updated] (SPARK-16486) Python API parity issues from 2.0 QA

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-16486: -- Priority: Major (was: Blocker) > Python API parity issues from 2.0 QA > --

[jira] [Resolved] (SPARK-14813) ML 2.0 QA: API: Python API coverage

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14813. --- Resolution: Done Fix Version/s: 2.0.0 Closing. All open tasks have been copie

[jira] [Closed] (SPARK-15630) 2.0 python coverage ml root module

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-15630. - Resolution: Won't Fix > 2.0 python coverage ml root module >

[jira] [Updated] (SPARK-15630) 2.0 python coverage ml root module

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15630: -- Priority: Major (was: Blocker) > 2.0 python coverage ml root module >

[jira] [Commented] (SPARK-15630) 2.0 python coverage ml root module

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371725#comment-15371725 ] Joseph K. Bradley commented on SPARK-15630: --- I'm going to go ahead and close th

[jira] [Commented] (SPARK-15623) 2.0 python coverage ml.feature

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371721#comment-15371721 ] Joseph K. Bradley commented on SPARK-15623: --- I'll go ahead and close this. Tha

[jira] [Resolved] (SPARK-15623) 2.0 python coverage ml.feature

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-15623. --- Resolution: Done Assignee: Bryan Cutler Fix Version/s: 2.0.0

[jira] [Commented] (SPARK-3181) Add Robust Regression Algorithm with Huber Estimator

2016-07-11 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371715#comment-15371715 ] DB Tsai commented on SPARK-3181: I prefer to option 1) as well. We can extend it to elasti

[jira] [Commented] (SPARK-16484) Incremental Cardinality estimation operations with Hyperloglog

2016-07-11 Thread Yongjia Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371709#comment-15371709 ] Yongjia Wang commented on SPARK-16484: -- Yes, I agree all the building blocks are the

[jira] [Resolved] (SPARK-16144) Add a separate Rd for ML generic methods: read.ml, write.ml, summary, predict

2016-07-11 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-16144. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request

[jira] [Commented] (SPARK-16426) IsotonicRegression produces NaNs with certain data

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371664#comment-15371664 ] Apache Spark commented on SPARK-16426: -- User 'neggert' has created a pull request fo

[jira] [Assigned] (SPARK-16426) IsotonicRegression produces NaNs with certain data

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16426: Assignee: Apache Spark > IsotonicRegression produces NaNs with certain data >

[jira] [Assigned] (SPARK-16426) IsotonicRegression produces NaNs with certain data

2016-07-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16426: Assignee: (was: Apache Spark) > IsotonicRegression produces NaNs with certain data > -

[jira] [Commented] (SPARK-16455) Add a new hook in CoarseGrainedSchedulerBackend in order to stop scheduling new tasks when cluster is restarting

2016-07-11 Thread YangyangLiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371653#comment-15371653 ] YangyangLiu commented on SPARK-16455: - Oh, we are implementing a new feature in an in

[jira] [Commented] (SPARK-16482) If a table's schema is inferred at runtime, describe table command does not show the schema

2016-07-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371648#comment-15371648 ] Xiao Li commented on SPARK-16482: - Let me try it. : ) > If a table's schema is inferred

[jira] [Updated] (SPARK-16483) Unifying struct fields and columns

2016-07-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-16483: - Target Version/s: 2.1.0 > Unifying struct fields and columns > --

[jira] [Commented] (SPARK-16334) [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

2016-07-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371633#comment-15371633 ] Herman van Hovell commented on SPARK-16334: --- It would also be great if we can r

[jira] [Updated] (SPARK-16486) Python API parity issues from 2.0 QA

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-16486: -- Description: This is an umbrella for Python API parity issues for MLlib found during 2.

[jira] [Commented] (SPARK-14813) ML 2.0 QA: API: Python API coverage

2016-07-11 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371624#comment-15371624 ] holdenk commented on SPARK-14813: - Yup, auditing is done and once 2.0 is out we will go b

[jira] [Updated] (SPARK-16486) Python API parity issues from 2.0 QA

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-16486: -- Issue Type: Umbrella (was: Sub-task) Parent: (was: SPARK-14808) > Python A

[jira] [Created] (SPARK-16486) Python API parity issues from 2.0 QA

2016-07-11 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-16486: - Summary: Python API parity issues from 2.0 QA Key: SPARK-16486 URL: https://issues.apache.org/jira/browse/SPARK-16486 Project: Spark Issue Type: Su

[jira] [Updated] (SPARK-16385) NoSuchMethodException thrown by Utils.waitForProcess

2016-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16385: -- Fix Version/s: 1.6.3 > NoSuchMethodException thrown by Utils.waitForProcess > -

[jira] [Commented] (SPARK-14813) ML 2.0 QA: API: Python API coverage

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371615#comment-15371615 ] Joseph K. Bradley commented on SPARK-14813: --- Yes, this was supposed to be an au

[jira] [Commented] (SPARK-16334) [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

2016-07-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371606#comment-15371606 ] Herman van Hovell commented on SPARK-16334: --- Could you try to disable the vecto

[jira] [Updated] (SPARK-16403) Example cleanup and fix minor issues

2016-07-11 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-16403: - Description: General cleanup of examples, focused on PySpark ML, to remove unused imports, sync

[jira] [Resolved] (SPARK-16458) SessionCatalog should support `listColumns` for temporary tables

2016-07-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16458. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.0.0 > Sess

[jira] [Updated] (SPARK-16240) model loading backward compatibility for ml.clustering.LDA

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-16240: -- Assignee: Gayathri Murali > model loading backward compatibility for ml.clustering.LDA

[jira] [Commented] (SPARK-16484) Incremental Cardinality estimation operations with Hyperloglog

2016-07-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371571#comment-15371571 ] Herman van Hovell commented on SPARK-16484: --- I do think much of the machinery i

[jira] [Commented] (SPARK-16334) [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

2016-07-11 Thread Vladimir Ivanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371572#comment-15371572 ] Vladimir Ivanov commented on SPARK-16334: - I believe it relates to the following

[jira] [Commented] (SPARK-16482) If a table's schema is inferred at runtime, describe table command does not show the schema

2016-07-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371564#comment-15371564 ] Yin Huai commented on SPARK-16482: -- Yea. Thanks! Seems we can just use lookupRelation to

[jira] [Commented] (SPARK-16450) Build failes for Mesos 0.28.x

2016-07-11 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371562#comment-15371562 ] Michael Gummelt commented on SPARK-16450: - Once Mesos 1.0 is released, I'll submi

[jira] [Commented] (SPARK-16482) If a table's schema is inferred at runtime, describe table command does not show the schema

2016-07-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371554#comment-15371554 ] Xiao Li commented on SPARK-16482: - Do you want me to do it? > If a table's schema is inf

[jira] [Updated] (SPARK-16485) Additional fixes to Mllib 2.0 documentation

2016-07-11 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Hunter updated SPARK-16485: --- Description: While reviewing the documentation of MLlib, I found some additional issues. Imp

[jira] [Comment Edited] (SPARK-16485) Additional fixes to Mllib 2.0 documentation

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371518#comment-15371518 ] Joseph K. Bradley edited comment on SPARK-16485 at 7/11/16 8:11 PM: ---

[jira] [Commented] (SPARK-16485) Additional fixes to Mllib 2.0 documentation

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371518#comment-15371518 ] Joseph K. Bradley commented on SPARK-16485: --- sqlDataTypes: Yes, it is needed.

[jira] [Commented] (SPARK-16485) Additional fixes to Mllib 2.0 documentation

2016-07-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371517#comment-15371517 ] Joseph K. Bradley commented on SPARK-16485: --- MultivariateGaussian should stay I

[jira] [Updated] (SPARK-16482) If a table's schema is inferred at runtime, describe table command does not show the schema

2016-07-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16482: - Affects Version/s: 2.0.0 > If a table's schema is inferred at runtime, describe table command does not >

[jira] [Commented] (SPARK-15705) Spark won't read ORC schema from metastore for partitioned tables

2016-07-11 Thread Nic Eggert (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371513#comment-15371513 ] Nic Eggert commented on SPARK-15705: Double-checked just for fun. The problem still e

  1   2   3   >