[jira] [Created] (SPARK-13054) Always post TaskEnd event for tasks in cancelled stages

2016-01-27 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13054: - Summary: Always post TaskEnd event for tasks in cancelled stages Key: SPARK-13054 URL: https://issues.apache.org/jira/browse/SPARK-13054 Project: Spark Issue

[jira] [Created] (SPARK-13053) Rectify ignored tests in InternalAccumulatorSuite

2016-01-27 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13053: - Summary: Rectify ignored tests in InternalAccumulatorSuite Key: SPARK-13053 URL: https://issues.apache.org/jira/browse/SPARK-13053 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13047) Pyspark Params.hasParam should not throw an error

2016-01-27 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120278#comment-15120278 ] Seth Hendrickson commented on SPARK-13047: -- cc [~mengxr] who wrote the initial implementation.

[jira] [Assigned] (SPARK-13052) waitingApps metric doesn't show the number of apps currently in the WAITING state

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13052: Assignee: (was: Apache Spark) > waitingApps metric doesn't show the number of apps

[jira] [Commented] (SPARK-13047) Pyspark Params.hasParam should not throw an error

2016-01-27 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120469#comment-15120469 ] Seth Hendrickson commented on SPARK-13047: -- Since the change is small, I have submitted a PR. If

[jira] [Updated] (SPARK-13052) waitingApps metric doesn't show the number of apps currently in the WAITING state

2016-01-27 Thread Raafat Akkad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raafat Akkad updated SPARK-13052: - Attachment: Incorrect waitingApps.png > waitingApps metric doesn't show the number of apps

[jira] [Updated] (SPARK-13053) Rectify ignored tests in InternalAccumulatorSuite

2016-01-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13053: -- Target Version/s: 2.0.0 > Rectify ignored tests in InternalAccumulatorSuite >

[jira] [Commented] (SPARK-12783) Dataset map serialization error

2016-01-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120421#comment-15120421 ] Wenchen Fan commented on SPARK-12783: - ah... finally I reproduced it in 1.6.0! Luckily this bug has

[jira] [Assigned] (SPARK-13047) Pyspark Params.hasParam should not throw an error

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13047: Assignee: Apache Spark > Pyspark Params.hasParam should not throw an error >

[jira] [Assigned] (SPARK-13047) Pyspark Params.hasParam should not throw an error

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13047: Assignee: (was: Apache Spark) > Pyspark Params.hasParam should not throw an error >

[jira] [Commented] (SPARK-13047) Pyspark Params.hasParam should not throw an error

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120463#comment-15120463 ] Apache Spark commented on SPARK-13047: -- User 'sethah' has created a pull request for this issue:

[jira] [Commented] (SPARK-12683) SQL timestamp is wrong when accessed as Python datetime

2016-01-27 Thread Daniel Jalova (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120471#comment-15120471 ] Daniel Jalova commented on SPARK-12683: --- This seems to be related to the 2038 problem where some

[jira] [Updated] (SPARK-13047) Pyspark Params.hasParam should not throw an error

2016-01-27 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hendrickson updated SPARK-13047: - Description: Pyspark {{Params}} class has a method {{hasParam(paramName)}} which returns

[jira] [Created] (SPARK-13052) waitingApps metric doesn't show the number of apps currently in the WAITING state

2016-01-27 Thread Raafat Akkad (JIRA)
Raafat Akkad created SPARK-13052: Summary: waitingApps metric doesn't show the number of apps currently in the WAITING state Key: SPARK-13052 URL: https://issues.apache.org/jira/browse/SPARK-13052

[jira] [Updated] (SPARK-12913) Reimplement stddev/variance as declarative function

2016-01-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12913: --- Summary: Reimplement stddev/variance as declarative function (was: Reimplement all builtin

[jira] [Updated] (SPARK-13053) Rectify ignored tests in InternalAccumulatorSuite

2016-01-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13053: -- Description: {code} // TODO: these two tests are incorrect; they don't actually trigger stage

[jira] [Assigned] (SPARK-12963) In cluster mode,spark_local_ip will cause driver exception:Service 'Driver' failed after 16 retries!

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12963: Assignee: (was: Apache Spark) > In cluster mode,spark_local_ip will cause driver

[jira] [Commented] (SPARK-12963) In cluster mode,spark_local_ip will cause driver exception:Service 'Driver' failed after 16 retries!

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120414#comment-15120414 ] Apache Spark commented on SPARK-12963: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12963) In cluster mode,spark_local_ip will cause driver exception:Service 'Driver' failed after 16 retries!

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12963: Assignee: Apache Spark > In cluster mode,spark_local_ip will cause driver

[jira] [Assigned] (SPARK-13043) Support remaining types in ColumnarBatch

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13043: Assignee: Apache Spark > Support remaining types in ColumnarBatch >

[jira] [Assigned] (SPARK-13043) Support remaining types in ColumnarBatch

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13043: Assignee: (was: Apache Spark) > Support remaining types in ColumnarBatch >

[jira] [Commented] (SPARK-13043) Support remaining types in ColumnarBatch

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120430#comment-15120430 ] Apache Spark commented on SPARK-13043: -- User 'nongli' has created a pull request for this issue:

[jira] [Commented] (SPARK-10620) Look into whether accumulator mechanism can replace TaskMetrics

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120311#comment-15120311 ] Apache Spark commented on SPARK-10620: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Created] (SPARK-13051) Do not maintain global singleton map for accumulators

2016-01-27 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13051: - Summary: Do not maintain global singleton map for accumulators Key: SPARK-13051 URL: https://issues.apache.org/jira/browse/SPARK-13051 Project: Spark Issue Type:

[jira] [Commented] (SPARK-13054) Always post TaskEnd event for tasks in cancelled stages

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120362#comment-15120362 ] Apache Spark commented on SPARK-13054: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13054) Always post TaskEnd event for tasks in cancelled stages

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13054: Assignee: Apache Spark (was: Andrew Or) > Always post TaskEnd event for tasks in

[jira] [Assigned] (SPARK-13054) Always post TaskEnd event for tasks in cancelled stages

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13054: Assignee: Andrew Or (was: Apache Spark) > Always post TaskEnd event for tasks in

[jira] [Updated] (SPARK-13053) Rectify ignored tests in InternalAccumulatorSuite

2016-01-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13053: -- Affects Version/s: 2.0.0 > Rectify ignored tests in InternalAccumulatorSuite >

[jira] [Commented] (SPARK-4144) Support incremental model training of Naive Bayes classifier

2016-01-27 Thread Imran Younus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120306#comment-15120306 ] Imran Younus commented on SPARK-4144: - [~cfregly] [~josephkb] If there is still interest in this jira,

[jira] [Commented] (SPARK-13037) PySpark ml.recommendation support export/import

2016-01-27 Thread Evan Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120341#comment-15120341 ] Evan Chen commented on SPARK-13037: --- Hey Yanbo, I can take this one. Thank you > PySpark

[jira] [Updated] (SPARK-13052) waitingApps metric doesn't show the number of apps currently in the WAITING state

2016-01-27 Thread Raafat Akkad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raafat Akkad updated SPARK-13052: - Attachment: Correct waitingApps.png > waitingApps metric doesn't show the number of apps

[jira] [Resolved] (SPARK-13045) Clean up ColumnarBatch.Row and Column.Struct

2016-01-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13045. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10952

[jira] [Commented] (SPARK-13057) Add benchmark codes and the performance results for implemented compression schemes for InMemoryRelation

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120822#comment-15120822 ] Apache Spark commented on SPARK-13057: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13057) Add benchmark codes and the performance results for implemented compression schemes for InMemoryRelation

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13057: Assignee: Apache Spark > Add benchmark codes and the performance results for implemented

[jira] [Assigned] (SPARK-13057) Add benchmark codes and the performance results for implemented compression schemes for InMemoryRelation

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13057: Assignee: (was: Apache Spark) > Add benchmark codes and the performance results for

[jira] [Assigned] (SPARK-12989) Bad interaction between StarExpansion and ExtractWindowExpressions

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12989: Assignee: Apache Spark > Bad interaction between StarExpansion and

[jira] [Commented] (SPARK-12989) Bad interaction between StarExpansion and ExtractWindowExpressions

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120677#comment-15120677 ] Apache Spark commented on SPARK-12989: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12989) Bad interaction between StarExpansion and ExtractWindowExpressions

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12989: Assignee: (was: Apache Spark) > Bad interaction between StarExpansion and

[jira] [Created] (SPARK-13057) Add benchmark codes and the performance results for implemented compression schemes for InMemoryRelation

2016-01-27 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-13057: Summary: Add benchmark codes and the performance results for implemented compression schemes for InMemoryRelation Key: SPARK-13057 URL:

[jira] [Created] (SPARK-13058) spark-shell start failure

2016-01-27 Thread geting (JIRA)
geting created SPARK-13058: -- Summary: spark-shell start failure Key: SPARK-13058 URL: https://issues.apache.org/jira/browse/SPARK-13058 Project: Spark Issue Type: Bug Environment: sh

[jira] [Assigned] (SPARK-12977) Factoring out StreamingListener and UI to support history UI

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12977: Assignee: (was: Apache Spark) > Factoring out StreamingListener and UI to support

[jira] [Assigned] (SPARK-12977) Factoring out StreamingListener and UI to support history UI

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12977: Assignee: Apache Spark > Factoring out StreamingListener and UI to support history UI >

[jira] [Commented] (SPARK-12977) Factoring out StreamingListener and UI to support history UI

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120890#comment-15120890 ] Apache Spark commented on SPARK-12977: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Created] (SPARK-13055) SQLHistoryListener throws ClassCastException

2016-01-27 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13055: - Summary: SQLHistoryListener throws ClassCastException Key: SPARK-13055 URL: https://issues.apache.org/jira/browse/SPARK-13055 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13056) Map column would throw NPE if value is null

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120816#comment-15120816 ] Apache Spark commented on SPARK-13056: -- User 'adrian-wang' has created a pull request for this

[jira] [Assigned] (SPARK-13056) Map column would throw NPE if value is null

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13056: Assignee: (was: Apache Spark) > Map column would throw NPE if value is null >

[jira] [Assigned] (SPARK-13056) Map column would throw NPE if value is null

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13056: Assignee: Apache Spark > Map column would throw NPE if value is null >

[jira] [Assigned] (SPARK-13052) waitingApps metric doesn't show the number of apps currently in the WAITING state

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13052: Assignee: Apache Spark > waitingApps metric doesn't show the number of apps currently in

[jira] [Commented] (SPARK-13052) waitingApps metric doesn't show the number of apps currently in the WAITING state

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120357#comment-15120357 ] Apache Spark commented on SPARK-13052: -- User 'RaafatAkkad' has created a pull request for this

[jira] [Created] (SPARK-13032) Basic ML Pipeline export/import functions for PySpark

2016-01-27 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-13032: --- Summary: Basic ML Pipeline export/import functions for PySpark Key: SPARK-13032 URL: https://issues.apache.org/jira/browse/SPARK-13032 Project: Spark Issue

[jira] [Commented] (SPARK-13031) Improve test coverage for whole stage codegen

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118838#comment-15118838 ] Apache Spark commented on SPARK-13031: -- User 'davies' has created a pull request for this issue:

[jira] [Commented] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2016-01-27 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118840#comment-15118840 ] Hyukjin Kwon commented on SPARK-12890: -- [~rxin] Could you confirm if this is an issue? > Spark SQL

[jira] [Assigned] (SPARK-13032) Basic ML Pipeline export/import functions for PySpark

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13032: Assignee: Apache Spark > Basic ML Pipeline export/import functions for PySpark >

[jira] [Created] (SPARK-13031) Improve test coverage for whole stage codegen

2016-01-27 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13031: -- Summary: Improve test coverage for whole stage codegen Key: SPARK-13031 URL: https://issues.apache.org/jira/browse/SPARK-13031 Project: Spark Issue Type:

[jira] [Commented] (SPARK-13032) Basic ML Pipeline export/import functions for PySpark

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118839#comment-15118839 ] Apache Spark commented on SPARK-13032: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13031) Improve test coverage for whole stage codegen

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13031: Assignee: Davies Liu (was: Apache Spark) > Improve test coverage for whole stage codegen

[jira] [Assigned] (SPARK-13032) Basic ML Pipeline export/import functions for PySpark

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13032: Assignee: (was: Apache Spark) > Basic ML Pipeline export/import functions for PySpark

[jira] [Assigned] (SPARK-13031) Improve test coverage for whole stage codegen

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13031: Assignee: Apache Spark (was: Davies Liu) > Improve test coverage for whole stage codegen

[jira] [Created] (SPARK-13033) PySpark regression support export/import

2016-01-27 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-13033: --- Summary: PySpark regression support export/import Key: SPARK-13033 URL: https://issues.apache.org/jira/browse/SPARK-13033 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-13042) Adding Specificity metric (= true negative rate) to MulticlassMetrics.scala

2016-01-27 Thread Daniel Marcous (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119706#comment-15119706 ] Daniel Marcous commented on SPARK-13042: [~srowen] Forgive me for my ignorance, read the

[jira] [Commented] (SPARK-9740) first/last aggregate NULL behavior

2016-01-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119673#comment-15119673 ] Yin Huai commented on SPARK-9740: - Thank you! > first/last aggregate NULL behavior >

[jira] [Resolved] (SPARK-13023) Check for presence of 'root' module after computing test_modules, not changed_modules

2016-01-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-13023. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10933

[jira] [Assigned] (SPARK-13023) Check for presence of 'root' module after computing test_modules, not changed_modules

2016-01-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-13023: -- Assignee: Josh Rosen > Check for presence of 'root' module after computing test_modules, not

[jira] [Updated] (SPARK-13058) spark-shell start fail

2016-01-27 Thread geting (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] geting updated SPARK-13058: --- Summary: spark-shell start fail (was: spark-shell start failure) > spark-shell start fail >

[jira] [Updated] (SPARK-13058) spark-shell start failure

2016-01-27 Thread geting (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] geting updated SPARK-13058: --- Affects Version/s: 1.6.0 > spark-shell start failure > - > > Key:

[jira] [Updated] (SPARK-11012) Canonicalize view definitions

2016-01-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-11012: Issue Type: New Feature (was: Bug) > Canonicalize view definitions >

[jira] [Commented] (SPARK-12818) Implement Bloom filter and count-min sketch in DataFrames

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120975#comment-15120975 ] Apache Spark commented on SPARK-12818: -- User 'liancheng' has created a pull request for this issue:

[jira] [Commented] (SPARK-12987) Drop fails when columns contain dots

2016-01-27 Thread Jayadevan M (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120922#comment-15120922 ] Jayadevan M commented on SPARK-12987: - I think the exception is occurred during select method

[jira] [Commented] (SPARK-10777) order by fails when column is aliased and projection includes windowed aggregate

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120961#comment-15120961 ] Apache Spark commented on SPARK-10777: -- User 'kevinyu98' has created a pull request for this issue:

[jira] [Commented] (SPARK-13034) PySpark ml.classification support export/import

2016-01-27 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120925#comment-15120925 ] Miao Wang commented on SPARK-13034: --- Yanbo, Thanks for your reply! I will learn the background of this

[jira] [Closed] (SPARK-13004) Support Non-Volatile Data and Operations

2016-01-27 Thread Wang, Gang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang, Gang closed SPARK-13004. -- Resolution: Later Preparing actionable items so close it temporarily as Sean Owen suggested. >

[jira] [Issue Comment Deleted] (SPARK-13004) Support Non-Volatile Data and Operations

2016-01-27 Thread Yanping Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanping Wang updated SPARK-13004: - Comment: was deleted (was: Hi, Sean, in order for this model to work, we also developed a memory

[jira] [Created] (SPARK-13043) Support remaining types in ColumnarBatch

2016-01-27 Thread Nong Li (JIRA)
Nong Li created SPARK-13043: --- Summary: Support remaining types in ColumnarBatch Key: SPARK-13043 URL: https://issues.apache.org/jira/browse/SPARK-13043 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-12780) Inconsistency returning value of ML python models' properties

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119912#comment-15119912 ] Apache Spark commented on SPARK-12780: -- User 'jkbradley' has created a pull request for this issue:

[jira] [Resolved] (SPARK-10847) Pyspark - DataFrame - Optional Metadata with `None` triggers cryptic failure

2016-01-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10847. -- Resolution: Fixed Fix Version/s: 1.6.1 1.5.3 2.0.0 Issue

[jira] [Issue Comment Deleted] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-27 Thread Lin Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Zhao updated SPARK-12591: - Comment: was deleted (was: Failed batches with mapWithState) > NullPointerException using checkpointed

[jira] [Updated] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-27 Thread Lin Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Zhao updated SPARK-12591: - Attachment: Screen Shot 2016-01-27 at 10.09.18 AM.png Failed batches with mapWithState >

[jira] [Commented] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-27 Thread Lin Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119907#comment-15119907 ] Lin Zhao commented on SPARK-12591: -- I'm not sure of the timing of the checkpointing. The error seems to

[jira] [Comment Edited] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-27 Thread Lin Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119907#comment-15119907 ] Lin Zhao edited comment on SPARK-12591 at 1/27/16 6:11 PM: --- I'm not sure of the

[jira] [Commented] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119888#comment-15119888 ] Shixiong Zhu commented on SPARK-12591: -- By the way, just want to confirm that: the error doesn't

[jira] [Updated] (SPARK-10847) Pyspark - DataFrame - Optional Metadata with `None` triggers cryptic failure

2016-01-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10847: - Assignee: Jason C Lee > Pyspark - DataFrame - Optional Metadata with `None` triggers cryptic failure >

[jira] [Created] (SPARK-13044) saveAsTextFile() failed to save file to AWS S3 EU-Frankfort

2016-01-27 Thread Xin Ren (JIRA)
Xin Ren created SPARK-13044: --- Summary: saveAsTextFile() failed to save file to AWS S3 EU-Frankfort Key: SPARK-13044 URL: https://issues.apache.org/jira/browse/SPARK-13044 Project: Spark Issue

[jira] [Commented] (SPARK-13004) Support Non-Volatile Data and Operations

2016-01-27 Thread Wang, Gang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119970#comment-15119970 ] Wang, Gang commented on SPARK-13004: I have closed it according to your advice, Thanks. > Support

[jira] [Resolved] (SPARK-13022) Shade jackson core

2016-01-27 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-13022. Resolution: Won't Fix > Shade jackson core > -- > > Key: SPARK-13022 >

[jira] [Created] (SPARK-13046) Partitioning looks broken in 1.6

2016-01-27 Thread Julien Baley (JIRA)
Julien Baley created SPARK-13046: Summary: Partitioning looks broken in 1.6 Key: SPARK-13046 URL: https://issues.apache.org/jira/browse/SPARK-13046 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120023#comment-15120023 ] Shixiong Zhu commented on SPARK-12591: -- Where is LiveSequenceMergeTracker, in a separate jar or the

[jira] [Updated] (SPARK-13046) Partitioning looks broken in 1.6

2016-01-27 Thread Julien Baley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Baley updated SPARK-13046: - Description: Hello, I have a list of files in s3:

[jira] [Updated] (SPARK-13044) saveAsTextFile() failed to save file to AWS S3 EU-Frankfort

2016-01-27 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-13044: Description: I have two clusters deployed: US and EU-Frankfort with the same configs on AWS. And the

[jira] [Commented] (SPARK-12944) CrossValidator doesn't accept a Pipeline as an estimator

2016-01-27 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119976#comment-15119976 ] Seth Hendrickson commented on SPARK-12944: -- Based on what I have seen, I think the fix is more

[jira] [Created] (SPARK-13045) Clean up ColumnarBatch.Row and Column.Struct

2016-01-27 Thread Nong Li (JIRA)
Nong Li created SPARK-13045: --- Summary: Clean up ColumnarBatch.Row and Column.Struct Key: SPARK-13045 URL: https://issues.apache.org/jira/browse/SPARK-13045 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-12895) Implement TaskMetrics using accumulators

2016-01-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-12895. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10835

[jira] [Commented] (SPARK-12591) NullPointerException using checkpointed mapWithState with KryoSerializer

2016-01-27 Thread Lin Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119987#comment-15119987 ] Lin Zhao commented on SPARK-12591: -- I patched the server with branch-1.6, reverted the workaround and

[jira] [Commented] (SPARK-13034) PySpark ml.classification support export/import

2016-01-27 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119959#comment-15119959 ] Miao Wang commented on SPARK-13034: --- Hi Yanbo, I saw you opened several of these sub-tasks. Can I try

[jira] [Assigned] (SPARK-12950) Improve performance of BytesToBytesMap

2016-01-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12950: -- Assignee: Davies Liu > Improve performance of BytesToBytesMap >

[jira] [Updated] (SPARK-12950) Improve performance of BytesToBytesMap

2016-01-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12950: --- Assignee: (was: Davies Liu) > Improve performance of BytesToBytesMap >

[jira] [Created] (SPARK-13047) Pyspark Params.hasParam should not throw an error

2016-01-27 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-13047: Summary: Pyspark Params.hasParam should not throw an error Key: SPARK-13047 URL: https://issues.apache.org/jira/browse/SPARK-13047 Project: Spark

[jira] [Created] (SPARK-13048) EMLDAOptimizer deletes dependent checkpoint of DistributedLDAModel

2016-01-27 Thread Jeff Stein (JIRA)
Jeff Stein created SPARK-13048: -- Summary: EMLDAOptimizer deletes dependent checkpoint of DistributedLDAModel Key: SPARK-13048 URL: https://issues.apache.org/jira/browse/SPARK-13048 Project: Spark

[jira] [Assigned] (SPARK-13045) Clean up ColumnarBatch.Row and Column.Struct

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13045: Assignee: (was: Apache Spark) > Clean up ColumnarBatch.Row and Column.Struct >

[jira] [Commented] (SPARK-13045) Clean up ColumnarBatch.Row and Column.Struct

2016-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120094#comment-15120094 ] Apache Spark commented on SPARK-13045: -- User 'nongli' has created a pull request for this issue:

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-27 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120113#comment-15120113 ] Mark Grover commented on SPARK-12177: - I have issued a new PR

  1   2   >