[jira] [Commented] (SPARK-22165) Type conflicts between dates, timestamps and date in partition column

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185388#comment-16185388 ] Apache Spark commented on SPARK-22165: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-22165) Type conflicts between dates, timestamps and date in partition column

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22165: Assignee: Apache Spark > Type conflicts between dates, timestamps and date in partition

[jira] [Assigned] (SPARK-22165) Type conflicts between dates, timestamps and date in partition column

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22165: Assignee: (was: Apache Spark) > Type conflicts between dates, timestamps and date in

[jira] [Created] (SPARK-22165) Type conflicts between dates, timestamps and date in partition column

2017-09-28 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-22165: Summary: Type conflicts between dates, timestamps and date in partition column Key: SPARK-22165 URL: https://issues.apache.org/jira/browse/SPARK-22165 Project: Spark

[jira] [Resolved] (SPARK-22160) Allow changing sample points per partition in range shuffle exchange

2017-09-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-22160. - Resolution: Fixed Fix Version/s: 2.3.0 > Allow changing sample points per partition in

[jira] [Closed] (SPARK-15687) Columnar execution engine

2017-09-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-15687. --- Resolution: Later Actually closing this, since with whole-stage code generation, it's unclear what

[jira] [Commented] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread jackyoh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185307#comment-16185307 ] jackyoh commented on SPARK-18935: - It's clear to me now. Thank you for your kind assistance. > Use

[jira] [Closed] (SPARK-21971) Too many open files in Spark due to concurrent files being opened

2017-09-28 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro closed SPARK-21971. Resolution: Not A Problem > Too many open files in Spark due to concurrent files being

[jira] [Resolved] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-09-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21999. --- Resolution: Not A Problem > ConcurrentModificationException - Spark Streaming >

[jira] [Commented] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-09-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185177#comment-16185177 ] Sean Owen commented on SPARK-21999: --- How would Spark do this synchronously w.r.t your app -- how can it

[jira] [Resolved] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-09-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22163. --- Resolution: Duplicate Please don't fork the issue. This is not a bug. > Design Issue of Spark

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-09-28 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185169#comment-16185169 ] Hyukjin Kwon commented on SPARK-21190: -- Will keep in mind and suggest to fix it or fix it by myself

[jira] [Created] (SPARK-22164) support histogram in estimating the cardinality of aggregate (or group-by) operator

2017-09-28 Thread Ron Hu (JIRA)
Ron Hu created SPARK-22164: -- Summary: support histogram in estimating the cardinality of aggregate (or group-by) operator Key: SPARK-22164 URL: https://issues.apache.org/jira/browse/SPARK-22164 Project:

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-09-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185166#comment-16185166 ] Reynold Xin commented on SPARK-21190: - OK it would be great to have a better error message, e.g.

[jira] [Comment Edited] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-09-28 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185164#comment-16185164 ] Hyukjin Kwon edited comment on SPARK-21190 at 9/29/17 12:33 AM: [~rxin],

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-09-28 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185164#comment-16185164 ] Hyukjin Kwon commented on SPARK-21190: -- [~rxin], I suggested to disallow it

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-09-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185155#comment-16185155 ] Reynold Xin commented on SPARK-21190: - Where did we settle on 0-arg UDFs? I think we should just

[jira] [Commented] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-09-28 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185130#comment-16185130 ] Michael N commented on SPARK-21999: --- Thanks Shixiong Zhu for tracking this issue to

[jira] [Created] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-09-28 Thread Michael N (JIRA)
Michael N created SPARK-22163: - Summary: Design Issue of Spark Streaming that Causes Random Run-time Exception Key: SPARK-22163 URL: https://issues.apache.org/jira/browse/SPARK-22163 Project: Spark

[jira] [Assigned] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22162: Assignee: Apache Spark > Executors and the driver use inconsistent Job IDs during the new

[jira] [Assigned] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22162: Assignee: (was: Apache Spark) > Executors and the driver use inconsistent Job IDs

[jira] [Commented] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185096#comment-16185096 ] Apache Spark commented on SPARK-22162: -- User 'rezasafi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22160) Allow changing sample points per partition in range shuffle exchange

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22160: Assignee: Reynold Xin (was: Apache Spark) > Allow changing sample points per partition

[jira] [Assigned] (SPARK-22160) Allow changing sample points per partition in range shuffle exchange

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22160: Assignee: Apache Spark (was: Reynold Xin) > Allow changing sample points per partition

[jira] [Updated] (SPARK-22162) Executors and the driver should use inconsistent Job IDs during the new RDD commit protocol

2017-09-28 Thread Reza Safi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-22162: -- Summary: Executors and the driver should use inconsistent Job IDs during the new RDD commit protocol

[jira] [Updated] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

2017-09-28 Thread Reza Safi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-22162: -- Summary: Executors and the driver use inconsistent Job IDs during the new RDD commit protocol (was:

[jira] [Commented] (SPARK-22160) Allow changing sample points per partition in range shuffle exchange

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185051#comment-16185051 ] Apache Spark commented on SPARK-22160: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

2017-09-28 Thread Reza Safi (JIRA)
Reza Safi created SPARK-22162: - Summary: Executors and the driver use inconsistent Job IDs during the new RDD commit protocol Key: SPARK-22162 URL: https://issues.apache.org/jira/browse/SPARK-22162

[jira] [Commented] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

2017-09-28 Thread Reza Safi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185036#comment-16185036 ] Reza Safi commented on SPARK-22162: --- I will send a pull request shortly for this issue. > Executors

[jira] [Commented] (SPARK-18660) Parquet complains "Can not initialize counter due to context is not a instance of TaskInputOutputContext, but is org.apache.hadoop.mapreduce.task.TaskAttemptContextImp

2017-09-28 Thread Randy Tidd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185024#comment-16185024 ] Randy Tidd commented on SPARK-18660: I am experiencing this problem now. However, it occurs at a

[jira] [Commented] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184979#comment-16184979 ] Stavros Kontopoulos commented on SPARK-18935: - Using a principal does not make any

[jira] [Assigned] (SPARK-22161) Add Impala-modified TPC-DS queries

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22161: Assignee: Apache Spark (was: Xiao Li) > Add Impala-modified TPC-DS queries >

[jira] [Assigned] (SPARK-22161) Add Impala-modified TPC-DS queries

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22161: Assignee: Xiao Li (was: Apache Spark) > Add Impala-modified TPC-DS queries >

[jira] [Commented] (SPARK-22161) Add Impala-modified TPC-DS queries

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184987#comment-16184987 ] Apache Spark commented on SPARK-22161: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Created] (SPARK-22161) Add Impala-modified TPC-DS queries

2017-09-28 Thread Xiao Li (JIRA)
Xiao Li created SPARK-22161: --- Summary: Add Impala-modified TPC-DS queries Key: SPARK-22161 URL: https://issues.apache.org/jira/browse/SPARK-22161 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-11034) Launcher: add support for monitoring Mesos apps

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11034: Assignee: (was: Apache Spark) > Launcher: add support for monitoring Mesos apps >

[jira] [Commented] (SPARK-11034) Launcher: add support for monitoring Mesos apps

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184972#comment-16184972 ] Apache Spark commented on SPARK-11034: -- User 'devaraj-kavali' has created a pull request for this

[jira] [Assigned] (SPARK-11034) Launcher: add support for monitoring Mesos apps

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11034: Assignee: Apache Spark > Launcher: add support for monitoring Mesos apps >

[jira] [Created] (SPARK-22160) Allow changing sample points per partition in range shuffle exchange

2017-09-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-22160: --- Summary: Allow changing sample points per partition in range shuffle exchange Key: SPARK-22160 URL: https://issues.apache.org/jira/browse/SPARK-22160 Project: Spark

[jira] [Assigned] (SPARK-22159) spark.sql.execution.arrow.enable and spark.sql.codegen.aggregate.map.twolevel.enable -> enabled

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22159: Assignee: Apache Spark (was: Reynold Xin) > spark.sql.execution.arrow.enable and >

[jira] [Assigned] (SPARK-22159) spark.sql.execution.arrow.enable and spark.sql.codegen.aggregate.map.twolevel.enable -> enabled

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22159: Assignee: Reynold Xin (was: Apache Spark) > spark.sql.execution.arrow.enable and >

[jira] [Commented] (SPARK-22159) spark.sql.execution.arrow.enable and spark.sql.codegen.aggregate.map.twolevel.enable -> enabled

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184634#comment-16184634 ] Apache Spark commented on SPARK-22159: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-22159) spark.sql.execution.arrow.enable and spark.sql.codegen.aggregate.map.twolevel.enable -> enabled

2017-09-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-22159: --- Summary: spark.sql.execution.arrow.enable and spark.sql.codegen.aggregate.map.twolevel.enable -> enabled Key: SPARK-22159 URL: https://issues.apache.org/jira/browse/SPARK-22159

[jira] [Updated] (SPARK-11412) Support merge schema for ORC

2017-09-28 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-11412: -- Affects Version/s: 1.6.3 2.0.0 2.2.0 > Support

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184588#comment-16184588 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 6:20 PM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184588#comment-16184588 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 6:20 PM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184588#comment-16184588 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 6:17 PM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184588#comment-16184588 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 6:16 PM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184588#comment-16184588 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 6:16 PM:

[jira] [Commented] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184588#comment-16184588 ] Stavros Kontopoulos commented on SPARK-18935: - I verified the example and error is the same

[jira] [Assigned] (SPARK-20643) Implement listener for saving application status data in key-value store

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20643: Assignee: (was: Apache Spark) > Implement listener for saving application status data

[jira] [Assigned] (SPARK-20643) Implement listener for saving application status data in key-value store

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20643: Assignee: Apache Spark > Implement listener for saving application status data in

[jira] [Commented] (SPARK-20643) Implement listener for saving application status data in key-value store

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184548#comment-16184548 ] Apache Spark commented on SPARK-20643: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22158) convertMetastoreOrc should not ignore table properties

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22158: Assignee: (was: Apache Spark) > convertMetastoreOrc should not ignore table

[jira] [Commented] (SPARK-22158) convertMetastoreOrc should not ignore table properties

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184521#comment-16184521 ] Apache Spark commented on SPARK-22158: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-22158) convertMetastoreOrc should not ignore table properties

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22158: Assignee: Apache Spark > convertMetastoreOrc should not ignore table properties >

[jira] [Commented] (SPARK-22152) Add Dataset flatten function

2017-09-28 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184511#comment-16184511 ] Wenchen Fan commented on SPARK-22152: - I don't have a strong opinion, if someone already has a PR, I

[jira] [Created] (SPARK-22158) convertMetastoreOrc should not ignore table properties

2017-09-28 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-22158: - Summary: convertMetastoreOrc should not ignore table properties Key: SPARK-22158 URL: https://issues.apache.org/jira/browse/SPARK-22158 Project: Spark

[jira] [Updated] (SPARK-22143) OffHeapColumnVector may leak memory

2017-09-28 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-22143: -- Fix Version/s: 2.2.1 > OffHeapColumnVector may leak memory >

[jira] [Commented] (SPARK-22152) Add Dataset flatten function

2017-09-28 Thread Drew Robb (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184296#comment-16184296 ] Drew Robb commented on SPARK-22152: --- There is also a ticket for adding it to RDD:

[jira] [Updated] (SPARK-22074) Task killed by other attempt task should not be resubmitted

2017-09-28 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-22074: - Component/s: Scheduler > Task killed by other attempt task should not be resubmitted >

[jira] [Updated] (SPARK-22074) Task killed by other attempt task should not be resubmitted

2017-09-28 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-22074: - Labels: speculation (was: ) > Task killed by other attempt task should not be resubmitted >

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 2:12 PM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 2:11 PM:

[jira] [Assigned] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18016: Assignee: Apache Spark (was: Aleksander Eskilson) > Code Generation: Constant Pool Past

[jira] [Assigned] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18016: Assignee: Aleksander Eskilson (was: Apache Spark) > Code Generation: Constant Pool Past

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2017-09-28 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184206#comment-16184206 ] Kazuaki Ishizaki commented on SPARK-18016: -- Thank you for reporting this again. While I pinged

[jira] [Reopened] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2017-09-28 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash reopened SPARK-18016: // reopening issue One PR addressing this bug has been merged --

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 1:13 PM:

[jira] [Commented] (SPARK-10884) Support prediction on single instance for regression and classification related models

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184126#comment-16184126 ] Apache Spark commented on SPARK-10884: -- User 'WeichenXu123' has created a pull request for this

[jira] [Commented] (SPARK-21737) Create communication channel between arbitrary clients and the Spark AM in YARN mode

2017-09-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184120#comment-16184120 ] Thomas Graves commented on SPARK-21737: --- We definitely still want to do this. I don't think

[jira] [Updated] (SPARK-22151) PYTHONPATH not picked up from the spark.yarn.appMasterEnv properly

2017-09-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-22151: -- Description: Running in yarn cluster mode and trying to set pythonpath via

[jira] [Commented] (SPARK-22157) The uniux_timestamp method handles the time field that is lost in mill

2017-09-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184088#comment-16184088 ] Sean Owen commented on SPARK-22157: --- This is invalid. A UNIX timestamp is in whole seconds. > The

[jira] [Assigned] (SPARK-22157) The uniux_timestamp method handles the time field that is lost in mill

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22157: Assignee: (was: Apache Spark) > The uniux_timestamp method handles the time field

[jira] [Commented] (SPARK-22157) The uniux_timestamp method handles the time field that is lost in mill

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184086#comment-16184086 ] Apache Spark commented on SPARK-22157: -- User 'httfighter' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22157) The uniux_timestamp method handles the time field that is lost in mill

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22157: Assignee: Apache Spark > The uniux_timestamp method handles the time field that is lost

[jira] [Created] (SPARK-22157) The uniux_timestamp method handles the time field that is lost in mill

2017-09-28 Thread hantiantian (JIRA)
hantiantian created SPARK-22157: --- Summary: The uniux_timestamp method handles the time field that is lost in mill Key: SPARK-22157 URL: https://issues.apache.org/jira/browse/SPARK-22157 Project: Spark

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 12:19 PM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 12:13 PM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 12:00 PM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:58 AM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:58 AM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:58 AM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:47 AM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:46 AM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:26 AM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:25 AM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:24 AM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:24 AM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:23 AM:

[jira] [Comment Edited] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos edited comment on SPARK-18935 at 9/28/17 11:23 AM:

[jira] [Commented] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-09-28 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184025#comment-16184025 ] Stavros Kontopoulos commented on SPARK-18935: - I tested this a while ago and stuck here:

[jira] [Commented] (SPARK-22152) Add Dataset flatten function

2017-09-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183996#comment-16183996 ] Sean Owen commented on SPARK-22152: --- (It does not have a flatten method. I was suggesting it makes

[jira] [Commented] (SPARK-22152) Add Dataset flatten function

2017-09-28 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183989#comment-16183989 ] Wenchen Fan commented on SPARK-22152: - if RDD has flatten, I'm ok to add it to Dataset. > Add

[jira] [Assigned] (SPARK-22156) incorrect learning rate update equation when numIterations > 1

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22156: Assignee: (was: Apache Spark) > incorrect learning rate update equation when

[jira] [Commented] (SPARK-22156) incorrect learning rate update equation when numIterations > 1

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183981#comment-16183981 ] Apache Spark commented on SPARK-22156: -- User 'nzw0301' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22156) incorrect learning rate update equation when numIterations > 1

2017-09-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22156: Assignee: Apache Spark > incorrect learning rate update equation when numIterations > 1 >

[jira] [Updated] (SPARK-22156) incorrect learning rate update equation when numIterations > 1

2017-09-28 Thread kento nozawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kento nozawa updated SPARK-22156: - Description: Current equation of learning rate is incorrect when numIterations > 1. Original

[jira] [Created] (SPARK-22156) incorrect learning rate update equation when numIterations > 1

2017-09-28 Thread kento nozawa (JIRA)
kento nozawa created SPARK-22156: Summary: incorrect learning rate update equation when numIterations > 1 Key: SPARK-22156 URL: https://issues.apache.org/jira/browse/SPARK-22156 Project: Spark

[jira] [Created] (SPARK-22155) Generates a simple default alias for the function in select

2017-09-28 Thread Lijia Liu (JIRA)
Lijia Liu created SPARK-22155: - Summary: Generates a simple default alias for the function in select Key: SPARK-22155 URL: https://issues.apache.org/jira/browse/SPARK-22155 Project: Spark Issue

  1   2   >