[jira] [Created] (SPARK-20121) simplify NullPropagation with NullIntolerant

2017-03-27 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-20121: --- Summary: simplify NullPropagation with NullIntolerant Key: SPARK-20121 URL: https://issues.apache.org/jira/browse/SPARK-20121 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-20120) spark-sql CLI support silent mode

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20120: Assignee: (was: Apache Spark) > spark-sql CLI support silent mode >

[jira] [Commented] (SPARK-20120) spark-sql CLI support silent mode

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944549#comment-15944549 ] Apache Spark commented on SPARK-20120: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20120) spark-sql CLI support silent mode

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20120: Assignee: Apache Spark > spark-sql CLI support silent mode >

[jira] [Created] (SPARK-20120) spark-sql CLI support silent mode

2017-03-27 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-20120: --- Summary: spark-sql CLI support silent mode Key: SPARK-20120 URL: https://issues.apache.org/jira/browse/SPARK-20120 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-20119) Flaky Test: org.apache.spark.sql.execution. DataSourceScanExecRedactionSuite

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20119: Assignee: Apache Spark (was: Xiao Li) > Flaky Test: org.apache.spark.sql.execution.

[jira] [Commented] (SPARK-20119) Flaky Test: org.apache.spark.sql.execution. DataSourceScanExecRedactionSuite

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944535#comment-15944535 ] Apache Spark commented on SPARK-20119: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20119) Flaky Test: org.apache.spark.sql.execution. DataSourceScanExecRedactionSuite

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20119: Assignee: Xiao Li (was: Apache Spark) > Flaky Test: org.apache.spark.sql.execution.

[jira] [Created] (SPARK-20119) Flaky Test: org.apache.spark.sql.execution. DataSourceScanExecRedactionSuite

2017-03-27 Thread Xiao Li (JIRA)
Xiao Li created SPARK-20119: --- Summary: Flaky Test: org.apache.spark.sql.execution. DataSourceScanExecRedactionSuite Key: SPARK-20119 URL: https://issues.apache.org/jira/browse/SPARK-20119 Project: Spark

[jira] [Commented] (SPARK-19476) Running threads in Spark DataFrame foreachPartition() causes NullPointerException

2017-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944506#comment-15944506 ] Sean Owen commented on SPARK-19476: --- Why do that in a thread instead of doing that same work

[jira] [Closed] (SPARK-20118) spark2.1 support numeric datatype

2017-03-27 Thread QQShu1 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QQShu1 closed SPARK-20118. -- > spark2.1 support numeric datatype > - > > Key: SPARK-20118 >

[jira] [Resolved] (SPARK-20118) spark2.1 support numeric datatype

2017-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20118. --- Resolution: Invalid > spark2.1 support numeric datatype > - > >

[jira] [Commented] (SPARK-19963) create view from select Fails when nullif() is used

2017-03-27 Thread dharani_sugumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944461#comment-15944461 ] dharani_sugumar commented on SPARK-19963: - @Jay Danielsen: I'm looking into this. > create view

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Description: I'm seeing a very weird crash in {{GeneratedIterator.sort_addToSorter}}. The hs_err_pid and

[jira] [Created] (SPARK-20118) spark2.1 support numeric datatype

2017-03-27 Thread QQShu1 (JIRA)
QQShu1 created SPARK-20118: -- Summary: spark2.1 support numeric datatype Key: SPARK-20118 URL: https://issues.apache.org/jira/browse/SPARK-20118 Project: Spark Issue Type: New Feature

[jira] [Closed] (SPARK-20117) TaskSetManager checkSpeculatableTasks variables immutability and Use string interpolation

2017-03-27 Thread jianran.tfh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianran.tfh closed SPARK-20117. --- Resolution: Invalid > TaskSetManager checkSpeculatableTasks variables immutability and Use string >

[jira] [Updated] (SPARK-19757) Executor with task scheduled could be killed due to idleness

2017-03-27 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-19757: - Component/s: Scheduler > Executor with task scheduled could be killed due to idleness >

[jira] [Commented] (SPARK-20070) Redact datasource explain output

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944417#comment-15944417 ] Apache Spark commented on SPARK-20070: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20117) TaskSetManager checkSpeculatableTasks variables immutability and Use string interpolation

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20117: Assignee: Apache Spark > TaskSetManager checkSpeculatableTasks variables immutability and

[jira] [Assigned] (SPARK-20117) TaskSetManager checkSpeculatableTasks variables immutability and Use string interpolation

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20117: Assignee: (was: Apache Spark) > TaskSetManager checkSpeculatableTasks variables

[jira] [Commented] (SPARK-20117) TaskSetManager checkSpeculatableTasks variables immutability and Use string interpolation

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944410#comment-15944410 ] Apache Spark commented on SPARK-20117: -- User 'jianran' has created a pull request for this issue:

[jira] [Created] (SPARK-20117) TaskSetManager checkSpeculatableTasks variables immutability and Use string interpolation

2017-03-27 Thread jianran.tfh (JIRA)
jianran.tfh created SPARK-20117: --- Summary: TaskSetManager checkSpeculatableTasks variables immutability and Use string interpolation Key: SPARK-20117 URL: https://issues.apache.org/jira/browse/SPARK-20117

[jira] [Resolved] (SPARK-19088) Optimize sequence type deserialization codegen

2017-03-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19088. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16541

[jira] [Assigned] (SPARK-19088) Optimize sequence type deserialization codegen

2017-03-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19088: --- Assignee: Michal Šenkýř > Optimize sequence type deserialization codegen >

[jira] [Resolved] (SPARK-20100) Consolidate SessionState construction

2017-03-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20100. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17433

[jira] [Created] (SPARK-20116) Remove task-level functionality from the DAGScheduler

2017-03-27 Thread Kay Ousterhout (JIRA)
Kay Ousterhout created SPARK-20116: -- Summary: Remove task-level functionality from the DAGScheduler Key: SPARK-20116 URL: https://issues.apache.org/jira/browse/SPARK-20116 Project: Spark

[jira] [Resolved] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

2017-03-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19803. - Resolution: Fixed Issue resolved by pull request 17325

[jira] [Commented] (SPARK-17075) Cardinality Estimation of Predicate Expressions

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944325#comment-15944325 ] Apache Spark commented on SPARK-17075: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20115) Fix DAGScheduler to recompute all the lost shuffle blocks when external shuffle service is unavailable

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20115: Assignee: Apache Spark > Fix DAGScheduler to recompute all the lost shuffle blocks when

[jira] [Commented] (SPARK-20115) Fix DAGScheduler to recompute all the lost shuffle blocks when external shuffle service is unavailable

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944312#comment-15944312 ] Apache Spark commented on SPARK-20115: -- User 'umehrot2' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20115) Fix DAGScheduler to recompute all the lost shuffle blocks when external shuffle service is unavailable

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20115: Assignee: (was: Apache Spark) > Fix DAGScheduler to recompute all the lost shuffle

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Description: I'm seeing a very weird crash in {{GeneratedIterator.sort_addToSorter}}. The hs_err_pid and

[jira] [Issue Comment Deleted] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Comment: was deleted (was: This kind of looks like https://issues.apache.org/jira/browse/SPARK-15822, but

[jira] [Created] (SPARK-20115) Fix DAGScheduler to recompute all the lost shuffle blocks when external shuffle service is unavailable

2017-03-27 Thread Udit Mehrotra (JIRA)
Udit Mehrotra created SPARK-20115: - Summary: Fix DAGScheduler to recompute all the lost shuffle blocks when external shuffle service is unavailable Key: SPARK-20115 URL:

[jira] [Comment Edited] (SPARK-20114) spark.ml parity for sequential pattern mining - PrefixSpan

2017-03-27 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944239#comment-15944239 ] yuhao yang edited comment on SPARK-20114 at 3/27/17 11:42 PM: -- Currently I

[jira] [Commented] (SPARK-20114) spark.ml parity for sequential pattern mining - PrefixSpan

2017-03-27 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944239#comment-15944239 ] yuhao yang commented on SPARK-20114: Currently I prefer to implement the dummy PrefixSpanModel as the

[jira] [Updated] (SPARK-20114) spark.ml parity for sequential pattern mining - PrefixSpan

2017-03-27 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-20114: --- Description: Creating this jira to track the feature parity for PrefixSpan and sequential pattern

[jira] [Updated] (SPARK-20114) spark.ml parity for sequential pattern mining - PrefixSpan

2017-03-27 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-20114: --- Description: Creating this jira to track the feature parity for PrefixSpan and sequential pattern

[jira] [Created] (SPARK-20114) spark.ml parity for sequential pattern mining - PrefixSpan

2017-03-27 Thread yuhao yang (JIRA)
yuhao yang created SPARK-20114: -- Summary: spark.ml parity for sequential pattern mining - PrefixSpan Key: SPARK-20114 URL: https://issues.apache.org/jira/browse/SPARK-20114 Project: Spark Issue

[jira] [Commented] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-03-27 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944215#comment-15944215 ] Timothy Hunter commented on SPARK-19634: [~sethah], yes, thanks for bringing up these concerns.

[jira] [Commented] (SPARK-19612) Tests failing with timeout

2017-03-27 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944181#comment-15944181 ] Kay Ousterhout commented on SPARK-19612: And another:

[jira] [Commented] (SPARK-19476) Running threads in Spark DataFrame foreachPartition() causes NullPointerException

2017-03-27 Thread Lucy Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944163#comment-15944163 ] Lucy Yu commented on SPARK-19476: - bq. I don't think in general you're expected to be able to do this

[jira] [Comment Edited] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-03-27 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944030#comment-15944030 ] Seth Hendrickson edited comment on SPARK-19634 at 3/27/17 10:23 PM:

[jira] [Resolved] (SPARK-20111) codegen bug surfaced by GraphFrames issue 165

2017-03-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-20111. --- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > codegen bug

[jira] [Comment Edited] (SPARK-20111) codegen bug surfaced by GraphFrames issue 165

2017-03-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944113#comment-15944113 ] Joseph K. Bradley edited comment on SPARK-20111 at 3/27/17 9:59 PM:

[jira] [Commented] (SPARK-20111) codegen bug surfaced by GraphFrames issue 165

2017-03-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944113#comment-15944113 ] Joseph K. Bradley commented on SPARK-20111: --- Yep, you're right. Is the fix worth backporting,

[jira] [Commented] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944100#comment-15944100 ] Mitesh commented on SPARK-20112: This kind of looks like

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: codegen_sorter_crash.log > SIGSEGV in GeneratedIterator.sort_addToSorter >

[jira] [Commented] (SPARK-14560) Cooperative Memory Management for Spillables

2017-03-27 Thread Morten Hornbech (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944094#comment-15944094 ] Morten Hornbech commented on SPARK-14560: - No. We worried it might just trigger some other bad

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: (was: codegen_sorter_crash.log) > SIGSEGV in GeneratedIterator.sort_addToSorter >

[jira] [Commented] (SPARK-14560) Cooperative Memory Management for Spillables

2017-03-27 Thread Darshan Mehta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944090#comment-15944090 ] Darshan Mehta commented on SPARK-14560: --- [~mhornbech] thanks for the prompt response. Before

[jira] [Commented] (SPARK-20111) codegen bug surfaced by GraphFrames issue 165

2017-03-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944089#comment-15944089 ] Herman van Hovell commented on SPARK-20111: --- [~josephkb] this might be already fixed in the

[jira] [Commented] (SPARK-20113) overwrite mode appends data on MySQL table that does not have a primary key

2017-03-27 Thread Bhanu Akaveeti (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944088#comment-15944088 ] Bhanu Akaveeti commented on SPARK-20113: I was expecting entire row comparison, but is there a

[jira] [Commented] (SPARK-14560) Cooperative Memory Management for Spillables

2017-03-27 Thread Morten Hornbech (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944080#comment-15944080 ] Morten Hornbech commented on SPARK-14560: - No, not really. We were able to work around it by

[jira] [Commented] (SPARK-20113) overwrite mode appends data on MySQL table that does not have a primary key

2017-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944079#comment-15944079 ] Sean Owen commented on SPARK-20113: --- If there is no primary key, how would anything know that the data

[jira] [Created] (SPARK-20113) overwrite mode appends data on MySQL table that does not have a primary key

2017-03-27 Thread Bhanu Akaveeti (JIRA)
Bhanu Akaveeti created SPARK-20113: -- Summary: overwrite mode appends data on MySQL table that does not have a primary key Key: SPARK-20113 URL: https://issues.apache.org/jira/browse/SPARK-20113

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Description: I'm seeing a very weird crash in {{GeneratedIterator.sort_addToSorter}}. The hs_err_pid and

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Description: I'm seeing a very weird crash in {{GeneratedIterator.sort_addToSorter}}. The hs_err_pid and

[jira] [Commented] (SPARK-14560) Cooperative Memory Management for Spillables

2017-03-27 Thread Darshan Mehta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944077#comment-15944077 ] Darshan Mehta commented on SPARK-14560: --- [~lovemylover] [~mhornbech] Were you able to figure

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: (was: codegen_sorter_crash) > SIGSEGV in GeneratedIterator.sort_addToSorter >

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: codegen_sorter_crash.log > SIGSEGV in GeneratedIterator.sort_addToSorter >

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: hs_err_pid19271.log codegen_sorter_crash > SIGSEGV in

[jira] [Created] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
Mitesh created SPARK-20112: -- Summary: SIGSEGV in GeneratedIterator.sort_addToSorter Key: SPARK-20112 URL: https://issues.apache.org/jira/browse/SPARK-20112 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-19876) Add OneTime trigger executor

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944073#comment-15944073 ] Apache Spark commented on SPARK-19876: -- User 'tdas' has created a pull request for this issue:

[jira] [Resolved] (SPARK-19904) SPIP Add Spark Project Improvement Proposal doc to website

2017-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19904. --- Resolution: Fixed Fix Version/s: 2.2.0 There's not a bright line between what goes in the

[jira] [Updated] (SPARK-18127) Add hooks and extension points to Spark

2017-03-27 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-18127: Component/s: (was: Spark Core) SQL > Add hooks and extension points to Spark >

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

2017-03-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944046#comment-15944046 ] Thomas Graves commented on SPARK-19143: --- Yes I can be Shephard. > API in Spark for distributing

[jira] [Commented] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-03-27 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944030#comment-15944030 ] Seth Hendrickson commented on SPARK-19634: -- I'm coming to this a bit late, but I'm finding

[jira] [Commented] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-03-27 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944019#comment-15944019 ] Timothy Hunter commented on SPARK-19634: [~dongjin] [~wm624] sorry it looks like I missed your

[jira] [Commented] (SPARK-20111) codegen bug surfaced by GraphFrames issue 165

2017-03-27 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944004#comment-15944004 ] Timothy Hunter commented on SPARK-20111: As Spark SQL is making more and more forays into code

[jira] [Updated] (SPARK-20111) codegen bug surfaced by GraphFrames issue 165

2017-03-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20111: -- Description: In GraphFrames, test {{test("named edges")}} in {{PatternMatchSuite}}

[jira] [Created] (SPARK-20111) codegen bug surfaced by GraphFrames issue 165

2017-03-27 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-20111: - Summary: codegen bug surfaced by GraphFrames issue 165 Key: SPARK-20111 URL: https://issues.apache.org/jira/browse/SPARK-20111 Project: Spark

[jira] [Created] (SPARK-20110) Windowed aggregation do not work when the timestamp is a nested field

2017-03-27 Thread Alexis Seigneurin (JIRA)
Alexis Seigneurin created SPARK-20110: - Summary: Windowed aggregation do not work when the timestamp is a nested field Key: SPARK-20110 URL: https://issues.apache.org/jira/browse/SPARK-20110

[jira] [Commented] (SPARK-20109) Need a way to convert from IndexedRowMatrix to Block

2017-03-27 Thread John Compitello (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943966#comment-15943966 ] John Compitello commented on SPARK-20109: - I have a PR for this issue in the works, I'd like to

[jira] [Created] (SPARK-20109) Need a way to convert from IndexedRowMatrix to Block

2017-03-27 Thread John Compitello (JIRA)
John Compitello created SPARK-20109: --- Summary: Need a way to convert from IndexedRowMatrix to Block Key: SPARK-20109 URL: https://issues.apache.org/jira/browse/SPARK-20109 Project: Spark

[jira] [Commented] (SPARK-20083) Change matrix toArray to not create a new array when matrix is already column major

2017-03-27 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943873#comment-15943873 ] Seth Hendrickson commented on SPARK-20083: -- Yes, that would be the intention. We have to take

[jira] [Commented] (SPARK-20083) Change matrix toArray to not create a new array when matrix is already column major

2017-03-27 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943857#comment-15943857 ] yuhao yang commented on SPARK-20083: So the result array will allow users to manipulate the matrix

[jira] [Commented] (SPARK-19904) SPIP Add Spark Project Improvement Proposal doc to website

2017-03-27 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943827#comment-15943827 ] Cody Koeninger commented on SPARK-19904: It has been added to apache/spark-website git repo

[jira] [Commented] (SPARK-19904) SPIP Add Spark Project Improvement Proposal doc to website

2017-03-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943799#comment-15943799 ] Thomas Graves commented on SPARK-19904: --- Is this done or what is this waiting on? > SPIP Add

[jira] [Assigned] (SPARK-20087) Include accumulators / taskMetrics when sending TaskKilled to onTaskEnd listeners

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20087: Assignee: Apache Spark > Include accumulators / taskMetrics when sending TaskKilled to

[jira] [Commented] (SPARK-20087) Include accumulators / taskMetrics when sending TaskKilled to onTaskEnd listeners

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943755#comment-15943755 ] Apache Spark commented on SPARK-20087: -- User 'noodle-fb' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20087) Include accumulators / taskMetrics when sending TaskKilled to onTaskEnd listeners

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20087: Assignee: (was: Apache Spark) > Include accumulators / taskMetrics when sending

[jira] [Resolved] (SPARK-20105) Add tests for checkType and type string in structField in R

2017-03-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20105. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.2.0

[jira] [Resolved] (SPARK-20102) Fix two minor build script issues blocking 2.1.1 RC + master snapshot builds

2017-03-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-20102. Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 Fixed for 2.1.1 and master.

[jira] [Commented] (SPARK-20037) impossible to set kafka offsets using kafka 0.10 and spark 2.0.0

2017-03-27 Thread Daniel Nuriyev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943654#comment-15943654 ] Daniel Nuriyev commented on SPARK-20037: Thank you for your feedback, This problem started when I

[jira] [Commented] (SPARK-20037) impossible to set kafka offsets using kafka 0.10 and spark 2.0.0

2017-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943652#comment-15943652 ] Sean Owen commented on SPARK-20037: --- I cannot reproduce this in my application. There is more to your

[jira] [Updated] (SPARK-20037) impossible to set kafka offsets using kafka 0.10 and spark 2.0.0

2017-03-27 Thread Daniel Nuriyev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Nuriyev updated SPARK-20037: --- Attachment: Main.java > impossible to set kafka offsets using kafka 0.10 and spark 2.0.0 >

[jira] [Commented] (SPARK-20037) impossible to set kafka offsets using kafka 0.10 and spark 2.0.0

2017-03-27 Thread Daniel Nuriyev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943642#comment-15943642 ] Daniel Nuriyev commented on SPARK-20037: My system is absolutely simple: a topic whose offset

[jira] [Assigned] (SPARK-20088) Do not create new SparkContext in SparkR createSparkContext

2017-03-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-20088: - Assignee: Hossein Falaki > Do not create new SparkContext in SparkR createSparkContext

[jira] [Resolved] (SPARK-20088) Do not create new SparkContext in SparkR createSparkContext

2017-03-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-20088. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17423

[jira] [Resolved] (SPARK-20104) Don't estimate IsNull or IsNotNull predicates for non-leaf node

2017-03-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20104. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17438

[jira] [Assigned] (SPARK-20104) Don't estimate IsNull or IsNotNull predicates for non-leaf node

2017-03-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20104: --- Assignee: Zhenhua Wang > Don't estimate IsNull or IsNotNull predicates for non-leaf node >

[jira] [Commented] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

2017-03-27 Thread Shubham Chopra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943460#comment-15943460 ] Shubham Chopra commented on SPARK-19803: The PR enforces a refresh of the peer list cached at the

[jira] [Commented] (SPARK-20107) Speed up HadoopMapReduceCommitProtocol#commitJob for many output files

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943398#comment-15943398 ] Apache Spark commented on SPARK-20107: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20107) Speed up HadoopMapReduceCommitProtocol#commitJob for many output files

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20107: Assignee: (was: Apache Spark) > Speed up HadoopMapReduceCommitProtocol#commitJob for

[jira] [Assigned] (SPARK-20107) Speed up HadoopMapReduceCommitProtocol#commitJob for many output files

2017-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20107: Assignee: Apache Spark > Speed up HadoopMapReduceCommitProtocol#commitJob for many output

[jira] [Updated] (SPARK-20107) Speed up HadoopMapReduceCommitProtocol#commitJob for many output files

2017-03-27 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-20107: Summary: Speed up HadoopMapReduceCommitProtocol#commitJob for many output files (was: Speed up

[jira] [Updated] (SPARK-20107) Speed up FileOutputCommitter#commitJob for many output files

2017-03-27 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-20107: Description: Set {{mapreduce.fileoutputcommitter.algorithm.version=2}} to speed up

[jira] [Commented] (SPARK-16938) Cannot resolve column name after a join

2017-03-27 Thread Michel Lemay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943382#comment-15943382 ] Michel Lemay commented on SPARK-16938: -- I just stumbled upon a similar issue with 'union' on two

[jira] [Created] (SPARK-20108) Spark query is getting failed with exception

2017-03-27 Thread ZS EDGE (JIRA)
ZS EDGE created SPARK-20108: --- Summary: Spark query is getting failed with exception Key: SPARK-20108 URL: https://issues.apache.org/jira/browse/SPARK-20108 Project: Spark Issue Type: Bug

  1   2   >