[jira] [Assigned] (SPARK-22196) Combine multiple input splits into a HadoopPartition

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22196: Assignee: Apache Spark > Combine multiple input splits into a HadoopPartition > --

[jira] [Assigned] (SPARK-22196) Combine multiple input splits into a HadoopPartition

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22196: Assignee: (was: Apache Spark) > Combine multiple input splits into a HadoopPartition >

[jira] [Commented] (SPARK-22196) Combine multiple input splits into a HadoopPartition

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16190891#comment-16190891 ] Apache Spark commented on SPARK-22196: -- User 'vgankidi' has created a pull request f

[jira] [Resolved] (SPARK-20783) Enhance ColumnVector to support compressed representation

2017-10-04 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20783. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18704 [https://githu

[jira] [Assigned] (SPARK-20783) Enhance ColumnVector to support compressed representation

2017-10-04 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20783: --- Assignee: Kazuaki Ishizaki > Enhance ColumnVector to support compressed representation > ---

[jira] [Commented] (SPARK-12606) Scala/Java compatibility issue Re: how to extend java transformer from Scala UnaryTransformer ?

2017-10-04 Thread Akos Tomasits (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16190904#comment-16190904 ] Akos Tomasits commented on SPARK-12606: --- We have run into the same issue. We cannot

[jira] [Assigned] (SPARK-22190) Add Spark executor task metrics to Dropwizard metrics

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22190: Assignee: (was: Apache Spark) > Add Spark executor task metrics to Dropwizard metrics

[jira] [Assigned] (SPARK-22190) Add Spark executor task metrics to Dropwizard metrics

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22190: Assignee: Apache Spark > Add Spark executor task metrics to Dropwizard metrics > -

[jira] [Commented] (SPARK-22190) Add Spark executor task metrics to Dropwizard metrics

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16190920#comment-16190920 ] Apache Spark commented on SPARK-22190: -- User 'LucaCanali' has created a pull request

[jira] [Created] (SPARK-22198) Java incompatibility when extending UnaryTransformer

2017-10-04 Thread Akos Tomasits (JIRA)
Akos Tomasits created SPARK-22198: - Summary: Java incompatibility when extending UnaryTransformer Key: SPARK-22198 URL: https://issues.apache.org/jira/browse/SPARK-22198 Project: Spark Issue

[jira] [Updated] (SPARK-22198) Java incompatibility when extending UnaryTransformer

2017-10-04 Thread Akos Tomasits (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akos Tomasits updated SPARK-22198: -- Description: It is not possible to create proper Java custom Transformer by extending UnaryTra

[jira] [Updated] (SPARK-22198) Java incompatibility when extending UnaryTransformer

2017-10-04 Thread Akos Tomasits (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akos Tomasits updated SPARK-22198: -- Description: It is not possible to create proper Java custom Transformer by extending UnaryTra

[jira] [Updated] (SPARK-22198) Java incompatibility when extending UnaryTransformer

2017-10-04 Thread Akos Tomasits (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akos Tomasits updated SPARK-22198: -- Description: It is not possible to create proper Java custom Transformer by extending UnaryTra

[jira] [Updated] (SPARK-22198) Java incompatibility when extending UnaryTransformer

2017-10-04 Thread Akos Tomasits (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akos Tomasits updated SPARK-22198: -- Component/s: Java API > Java incompatibility when extending UnaryTransformer >

[jira] [Updated] (SPARK-22198) Java incompatibility when extending UnaryTransformer

2017-10-04 Thread Akos Tomasits (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akos Tomasits updated SPARK-22198: -- Description: It is not possible to create proper Java custom Transformer by extending UnaryTra

[jira] [Updated] (SPARK-22198) Java incompatibility when extending UnaryTransformer

2017-10-04 Thread Akos Tomasits (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akos Tomasits updated SPARK-22198: -- Description: It is not possible to create proper Java custom Transformer by extending UnaryTra

[jira] [Comment Edited] (SPARK-12606) Scala/Java compatibility issue Re: how to extend java transformer from Scala UnaryTransformer ?

2017-10-04 Thread Akos Tomasits (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16190904#comment-16190904 ] Akos Tomasits edited comment on SPARK-12606 at 10/4/17 9:02 AM: ---

[jira] [Comment Edited] (SPARK-14172) Hive table partition predicate not passed down correctly

2017-10-04 Thread Saktheesh Balaraj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189481#comment-16189481 ] Saktheesh Balaraj edited comment on SPARK-14172 at 10/4/17 9:21 AM: ---

[jira] [Commented] (SPARK-22195) Add cosine similarity to org.apache.spark.ml.linalg.Vectors

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191037#comment-16191037 ] Sean Owen commented on SPARK-22195: --- When I've heard this, it's been in the context of

[jira] [Commented] (SPARK-22131) Add Mesos Secrets Support to the Mesos Driver

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191155#comment-16191155 ] Apache Spark commented on SPARK-22131: -- User 'susanxhuynh' has created a pull reques

[jira] [Assigned] (SPARK-22131) Add Mesos Secrets Support to the Mesos Driver

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22131: Assignee: (was: Apache Spark) > Add Mesos Secrets Support to the Mesos Driver > --

[jira] [Assigned] (SPARK-22131) Add Mesos Secrets Support to the Mesos Driver

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22131: Assignee: Apache Spark > Add Mesos Secrets Support to the Mesos Driver > -

[jira] [Created] (SPARK-22199) Spark Job on YARN fails with executors "Slave registration failed"

2017-10-04 Thread Prabhu Joseph (JIRA)
Prabhu Joseph created SPARK-22199: - Summary: Spark Job on YARN fails with executors "Slave registration failed" Key: SPARK-22199 URL: https://issues.apache.org/jira/browse/SPARK-22199 Project: Spark

[jira] [Updated] (SPARK-22199) Spark Job on YARN fails with executors "Slave registration failed"

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22199: -- Priority: Minor (was: Major) > Spark Job on YARN fails with executors "Slave registration failed" > --

[jira] [Assigned] (SPARK-20055) Documentation for CSV datasets in SQL programming guide

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20055: Assignee: Apache Spark > Documentation for CSV datasets in SQL programming guide > ---

[jira] [Assigned] (SPARK-20055) Documentation for CSV datasets in SQL programming guide

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20055: Assignee: (was: Apache Spark) > Documentation for CSV datasets in SQL programming guid

[jira] [Commented] (SPARK-20055) Documentation for CSV datasets in SQL programming guide

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191230#comment-16191230 ] Apache Spark commented on SPARK-20055: -- User 'jomach' has created a pull request for

[jira] [Commented] (SPARK-20055) Documentation for CSV datasets in SQL programming guide

2017-10-04 Thread Jorge Machado (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191232#comment-16191232 ] Jorge Machado commented on SPARK-20055: --- [~hyukjin.kwon] just added some docs. >

[jira] [Commented] (SPARK-20557) JdbcUtils doesn't support java.sql.Types.TIMESTAMP_WITH_TIMEZONE

2017-10-04 Thread Dan Stine (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191255#comment-16191255 ] Dan Stine commented on SPARK-20557: --- Got it. Thank you, Sean. > JdbcUtils doesn't supp

[jira] [Created] (SPARK-22200) Kinesis Receivers stops if Kinesis stream was re-sharded

2017-10-04 Thread Alex Mikhailau (JIRA)
Alex Mikhailau created SPARK-22200: -- Summary: Kinesis Receivers stops if Kinesis stream was re-sharded Key: SPARK-22200 URL: https://issues.apache.org/jira/browse/SPARK-22200 Project: Spark

[jira] [Commented] (SPARK-10884) Support prediction on single instance for regression and classification related models

2017-10-04 Thread Derek Kaknes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191424#comment-16191424 ] Derek Kaknes commented on SPARK-10884: -- [~WeichenXu123] Thanks for picking this up,

[jira] [Created] (SPARK-22201) Dataframe describe includes string columns

2017-10-04 Thread cold gin (JIRA)
cold gin created SPARK-22201: Summary: Dataframe describe includes string columns Key: SPARK-22201 URL: https://issues.apache.org/jira/browse/SPARK-22201 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-22201) Dataframe describe includes string columns

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191461#comment-16191461 ] Sean Owen commented on SPARK-22201: --- This is just a duplicate of SPARK-16468 What API a

[jira] [Updated] (SPARK-22201) Dataframe describe includes string columns

2017-10-04 Thread cold gin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cold gin updated SPARK-22201: - Description: As per the api documentation, the default no-arg Dataframe describe() function should only

[jira] [Updated] (SPARK-22169) support byte length literal as identifier

2017-10-04 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-22169: Summary: support byte length literal as identifier (was: table name with numbers and characters sh

[jira] [Commented] (SPARK-22192) An RDD of nested POJO objects cannot be converted into a DataFrame using SQLContext.createDataFrame API

2017-10-04 Thread Asif Hussain Shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191526#comment-16191526 ] Asif Hussain Shahid commented on SPARK-22192: - The PR opened for this bug is

[jira] [Assigned] (SPARK-21871) Check actual bytecode size when compiling generated code

2017-10-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-21871: --- Assignee: Takeshi Yamamuro > Check actual bytecode size when compiling generated code >

[jira] [Resolved] (SPARK-21871) Check actual bytecode size when compiling generated code

2017-10-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21871. - Resolution: Fixed Fix Version/s: 2.3.0 > Check actual bytecode size when compiling generated code

[jira] [Updated] (SPARK-21871) Check actual bytecode size when compiling generated code

2017-10-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21871: Priority: Critical (was: Minor) > Check actual bytecode size when compiling generated code > -

[jira] [Updated] (SPARK-22187) Update unsaferow format for saved state such that we can set timeouts when state is null

2017-10-04 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-22187: - Labels: release-notes (was: ) > Update unsaferow format for saved state such that we can set tim

[jira] [Updated] (SPARK-22187) Update unsaferow format for saved state such that we can set timeouts when state is null

2017-10-04 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-22187: - Labels: release-notes releasenotes (was: release-notes) > Update unsaferow format for saved stat

[jira] [Created] (SPARK-22202) Release tgz content differences for python and R

2017-10-04 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-22202: Summary: Release tgz content differences for python and R Key: SPARK-22202 URL: https://issues.apache.org/jira/browse/SPARK-22202 Project: Spark Issue Type:

[jira] [Updated] (SPARK-22202) Release tgz content differences for python and R

2017-10-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22202: - Description: As a follow up to SPARK-22167, currently we are running different profiles/steps in

[jira] [Commented] (SPARK-18580) Use spark.streaming.backpressure.initialRate in DirectKafkaInputDStream

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191767#comment-16191767 ] Apache Spark commented on SPARK-18580: -- User 'akonopko' has created a pull request f

[jira] [Commented] (SPARK-18580) Use spark.streaming.backpressure.initialRate in DirectKafkaInputDStream

2017-10-04 Thread Oleksandr Konopko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191776#comment-16191776 ] Oleksandr Konopko commented on SPARK-18580: --- https://github.com/apache/spark/pu

[jira] [Assigned] (SPARK-15689) Data source API v2

2017-10-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-15689: --- Assignee: Wenchen Fan > Data source API v2 > -- > > Key: SPA

[jira] [Commented] (SPARK-22201) Dataframe describe includes string columns

2017-10-04 Thread cold gin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191973#comment-16191973 ] cold gin commented on SPARK-22201: -- Ok I see what you mean - it is Dataset, not Datafram

[jira] [Created] (SPARK-22203) Add job description for file listing Spark jobs

2017-10-04 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-22203: Summary: Add job description for file listing Spark jobs Key: SPARK-22203 URL: https://issues.apache.org/jira/browse/SPARK-22203 Project: Spark Issue Type: I

[jira] [Reopened] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael N reopened SPARK-22163: --- Please distinguish between code bug vs design flaws. That is why this ticket is separate from the other

[jira] [Comment Edited] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192110#comment-16192110 ] Michael N edited comment on SPARK-22163 at 10/4/17 10:02 PM: -

[jira] [Reopened] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael N reopened SPARK-21999: --- The posted questions were intended to show why this is a design issue. You have not been able to provide

[jira] [Commented] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192131#comment-16192131 ] Michael N commented on SPARK-21999: --- The posted questions were intended to show why thi

[jira] [Issue Comment Deleted] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael N updated SPARK-21999: -- Comment: was deleted (was: The posted questions were intended to show why this is a design issue. You

[jira] [Comment Edited] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192135#comment-16192135 ] Michael N edited comment on SPARK-21999 at 10/4/17 10:13 PM: -

[jira] [Comment Edited] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192110#comment-16192110 ] Michael N edited comment on SPARK-22163 at 10/4/17 10:12 PM: -

[jira] [Assigned] (SPARK-22203) Add job description for file listing Spark jobs

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22203: Assignee: Shixiong Zhu (was: Apache Spark) > Add job description for file listing Spark j

[jira] [Commented] (SPARK-22203) Add job description for file listing Spark jobs

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192162#comment-16192162 ] Apache Spark commented on SPARK-22203: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-22203) Add job description for file listing Spark jobs

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22203: Assignee: Apache Spark (was: Shixiong Zhu) > Add job description for file listing Spark j

[jira] [Comment Edited] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192135#comment-16192135 ] Michael N edited comment on SPARK-21999 at 10/4/17 10:31 PM: -

[jira] [Commented] (SPARK-3162) Train DecisionTree locally when possible

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192166#comment-16192166 ] Apache Spark commented on SPARK-3162: - User 'smurching' has created a pull request for

[jira] [Comment Edited] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192135#comment-16192135 ] Michael N edited comment on SPARK-21999 at 10/4/17 10:32 PM: -

[jira] [Comment Edited] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192110#comment-16192110 ] Michael N edited comment on SPARK-22163 at 10/4/17 10:33 PM: -

[jira] [Commented] (SPARK-3162) Train DecisionTree locally when possible

2017-10-04 Thread Siddharth Murching (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192201#comment-16192201 ] Siddharth Murching commented on SPARK-3162: --- Commenting here to note that I'm re

[jira] [Comment Edited] (SPARK-3162) Train DecisionTree locally when possible

2017-10-04 Thread Siddharth Murching (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192201#comment-16192201 ] Siddharth Murching edited comment on SPARK-3162 at 10/4/17 11:35 PM: ---

[jira] [Commented] (SPARK-3181) Add Robust Regression Algorithm with Huber Estimator

2017-10-04 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192217#comment-16192217 ] yuhao yang commented on SPARK-3181: --- Regarding to whether to separate Huber loss an an i

[jira] [Commented] (SPARK-22201) Dataframe describe includes string columns

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192231#comment-16192231 ] Sean Owen commented on SPARK-22201: --- If you're saying that string-valued columns should

[jira] [Resolved] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22163. --- Resolution: Duplicate If one accepts your premise, then the bug is simply a manifestation of the 'de

[jira] [Closed] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-22163. - > Design Issue of Spark Streaming that Causes Random Run-time Exception > ---

[jira] [Closed] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-21999. - > ConcurrentModificationException - Spark Streaming > - > >

[jira] [Resolved] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21999. --- Resolution: Not A Problem I am not sure what you mean by bold-faced asynchronously. Your app is modif

[jira] [Assigned] (SPARK-21785) Support create table from a file schema

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21785: Assignee: Apache Spark > Support create table from a file schema > ---

[jira] [Commented] (SPARK-21785) Support create table from a file schema

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192244#comment-16192244 ] Apache Spark commented on SPARK-21785: -- User 'CrazyJacky' has created a pull request

[jira] [Assigned] (SPARK-21785) Support create table from a file schema

2017-10-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21785: Assignee: (was: Apache Spark) > Support create table from a file schema >

[jira] [Created] (SPARK-22204) Explain output for SQL with commands shows no optimization

2017-10-04 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-22204: -- Summary: Explain output for SQL with commands shows no optimization Key: SPARK-22204 URL: https://issues.apache.org/jira/browse/SPARK-22204 Project: Spark Issue

[jira] [Reopened] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael N reopened SPARK-21999: --- No, my app is *not* modifying a collection asynchronously w.r.t. Spark. You misunderstood the issue co

[jira] [Reopened] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael N reopened SPARK-22163: --- It is obvious that you don't understand the differences between design flaws vs coding bugs, particularl

[jira] [Created] (SPARK-22205) Incorrect result with user defined agg function followed by a non deterministic function

2017-10-04 Thread kanika dhuria (JIRA)
kanika dhuria created SPARK-22205: - Summary: Incorrect result with user defined agg function followed by a non deterministic function Key: SPARK-22205 URL: https://issues.apache.org/jira/browse/SPARK-22205

[jira] [Comment Edited] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192348#comment-16192348 ] Michael N edited comment on SPARK-21999 at 10/5/17 1:39 AM: N

[jira] [Comment Edited] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192352#comment-16192352 ] Michael N edited comment on SPARK-22163 at 10/5/17 1:42 AM: I

[jira] [Comment Edited] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192348#comment-16192348 ] Michael N edited comment on SPARK-21999 at 10/5/17 1:43 AM: N

[jira] [Comment Edited] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192348#comment-16192348 ] Michael N edited comment on SPARK-21999 at 10/5/17 1:46 AM: N

[jira] [Commented] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192364#comment-16192364 ] Sean Owen commented on SPARK-21999: --- [~michaeln_apache] I'll have to see if we can bloc

[jira] [Commented] (SPARK-21785) Support create table from a file schema

2017-10-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192362#comment-16192362 ] Takeshi Yamamuro commented on SPARK-21785: -- `CREATE TABLE xxx USING parquet OPTI

[jira] [Closed] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-21999. - Resolution: Invalid Discussion may continue here but should not be reopened except by committers, when t

[jira] [Reopened] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael N reopened SPARK-21999: --- Sean, this issue occurs in both the slave nodes and the driver. No, my app is not modifying a collectio

[jira] [Commented] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192374#comment-16192374 ] Sean Owen commented on SPARK-21999: --- Committers: rather than continue the close battle

[jira] [Commented] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192373#comment-16192373 ] Sean Owen commented on SPARK-22163: --- Committers: rather than continue the close battle

[jira] [Comment Edited] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192352#comment-16192352 ] Michael N edited comment on SPARK-22163 at 10/5/17 2:11 AM: I

[jira] [Resolved] (SPARK-22187) Update unsaferow format for saved state such that we can set timeouts when state is null

2017-10-04 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-22187. --- Resolution: Fixed Fix Version/s: (was: 2.2.0) 3.0.0 Issue resol

[jira] [Commented] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192394#comment-16192394 ] Michael N commented on SPARK-21999: --- The actual issues are with Seam Owen himself where

[jira] [Comment Edited] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192394#comment-16192394 ] Michael N edited comment on SPARK-21999 at 10/5/17 2:38 AM: C

[jira] [Commented] (SPARK-22163) Design Issue of Spark Streaming that Causes Random Run-time Exception

2017-10-04 Thread Michael N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192395#comment-16192395 ] Michael N commented on SPARK-22163: --- Committers: The actual issues are with Seam Owen h

[jira] [Commented] (SPARK-22187) Update unsaferow format for saved state such that we can set timeouts when state is null

2017-10-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192406#comment-16192406 ] Sean Owen commented on SPARK-22187: --- [~tdas] fixed for 2.3.0 right, if it's in master?

[jira] [Commented] (SPARK-21999) ConcurrentModificationException - Spark Streaming

2017-10-04 Thread Vadim Semenov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192407#comment-16192407 ] Vadim Semenov commented on SPARK-21999: --- Could you somehow reproduce the issue? If

[jira] [Reopened] (SPARK-21783) Turn on ORC filter push-down by default

2017-10-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-21783: --- > Turn on ORC filter push-down by default > --- > >

[jira] [Commented] (SPARK-22137) Failed to insert VectorUDT to hive table with DataFrameWriter.insertInto(tableName: String)

2017-10-04 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192414#comment-16192414 ] Liang-Chi Hsieh commented on SPARK-22137: - Ideally I think a UDT should be able c

[jira] [Commented] (SPARK-21785) Support create table from a file schema

2017-10-04 Thread Jacky Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192430#comment-16192430 ] Jacky Shen commented on SPARK-21785: from my understanding, "CREATE TABLE xxx USING p

[jira] [Commented] (SPARK-21785) Support create table from a file schema

2017-10-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192433#comment-16192433 ] Takeshi Yamamuro commented on SPARK-21785: -- {code} scala> sql("""CREATE TABLE cu

[jira] [Resolved] (SPARK-22203) Add job description for file listing Spark jobs

2017-10-04 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-22203. -- Resolution: Fixed Fix Version/s: 2.3.0 > Add job description for file listing Spark jobs

[jira] [Commented] (SPARK-22204) Explain output for SQL with commands shows no optimization

2017-10-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192454#comment-16192454 ] Takeshi Yamamuro commented on SPARK-22204: -- good catch. The optimization is inte

  1   2   >