[jira] [Resolved] (SPARK-18232) Support Mesos CNI

2016-11-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18232. - Resolution: Fixed Assignee: Michael Gummelt Fix Version/s: 2.2.0 > Support Mesos

[jira] [Assigned] (SPARK-18442) Fix nullability of WrapOption.

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18442: Assignee: Apache Spark > Fix nullability of WrapOption. > -- >

[jira] [Commented] (SPARK-18442) Fix nullability of WrapOption.

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666206#comment-15666206 ] Apache Spark commented on SPARK-18442: -- User 'ueshin' has created a pull request for

[jira] [Assigned] (SPARK-18442) Fix nullability of WrapOption.

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18442: Assignee: (was: Apache Spark) > Fix nullability of WrapOption. > -

[jira] [Created] (SPARK-18442) Fix nullability of WrapOption.

2016-11-14 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-18442: - Summary: Fix nullability of WrapOption. Key: SPARK-18442 URL: https://issues.apache.org/jira/browse/SPARK-18442 Project: Spark Issue Type: Bug Co

[jira] [Resolved] (SPARK-18430) Returned Message Null when Hitting an Invocation Exception of Function Lookup.

2016-11-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18430. - Resolution: Fixed Fix Version/s: 2.1.0 > Returned Message Null when Hitting an Invocation

[jira] [Resolved] (SPARK-18428) Update docs for GraphX

2016-11-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18428. - Resolution: Fixed Assignee: zhengruifeng Fix Version/s: 2.1.0 > Update docs for G

[jira] [Commented] (SPARK-14560) Cooperative Memory Management for Spillables

2016-11-14 Thread KaiXu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666005#comment-15666005 ] KaiXu commented on SPARK-14560: --- spark1.6.2 has the same issue: 16/11/15 10:11:15 INFO sch

[jira] [Updated] (SPARK-18428) Update docs for GraphX

2016-11-14 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-18428: - Description: 1, notify that not all methods are listed in {{Vertex and Edge RDDs}} 2, add link of

[jira] [Updated] (SPARK-18428) Update docs for GraphX

2016-11-14 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-18428: - Summary: Update docs for GraphX (was: Update docs for Graph.op) > Update docs for GraphX > -

[jira] [Created] (SPARK-18441) Add Smote in spark mlib and ml

2016-11-14 Thread lichenglin (JIRA)
lichenglin created SPARK-18441: -- Summary: Add Smote in spark mlib and ml Key: SPARK-18441 URL: https://issues.apache.org/jira/browse/SPARK-18441 Project: Spark Issue Type: Wish Compone

[jira] [Commented] (SPARK-17076) Cardinality estimation of join operator

2016-11-14 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665664#comment-15665664 ] Ron Hu commented on SPARK-17076: Hi, I am out of office 11 /15 through 11/18 with very l

[jira] [Commented] (SPARK-18050) spark 2.0.1 enable hive throw AlreadyExistsException(message:Database default already exists)

2016-11-14 Thread todd.chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665655#comment-15665655 ] todd.chen commented on SPARK-18050: --- thanks > spark 2.0.1 enable hive throw AlreadyExi

[jira] [Comment Edited] (SPARK-17076) Cardinality estimation of join operator

2016-11-14 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665644#comment-15665644 ] Ruslan Dautkhanov edited comment on SPARK-17076 at 11/15/16 1:19 AM: --

[jira] [Commented] (SPARK-17076) Cardinality estimation of join operator

2016-11-14 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665644#comment-15665644 ] Ruslan Dautkhanov commented on SPARK-17076: --- HIVE-13076 added FK constraint to

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2016-11-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665618#comment-15665618 ] Marcelo Vanzin commented on SPARK-18085: Hey all, any more feedback here? I've m

[jira] [Commented] (SPARK-18050) spark 2.0.1 enable hive throw AlreadyExistsException(message:Database default already exists)

2016-11-14 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665599#comment-15665599 ] Wenchen Fan commented on SPARK-18050: - ah, it's not throwing an exception, but loggin

[jira] [Closed] (SPARK-18050) spark 2.0.1 enable hive throw AlreadyExistsException(message:Database default already exists)

2016-11-14 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan closed SPARK-18050. --- Resolution: Not A Problem > spark 2.0.1 enable hive throw AlreadyExistsException(message:Database def

[jira] [Commented] (SPARK-8133) sticky partitions

2016-11-14 Thread sandeep pournami (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665590#comment-15665590 ] sandeep pournami commented on SPARK-8133: - +1 as when using Spark streaming, the

[jira] [Assigned] (SPARK-18440) Fix FileStreamSink with aggregation + watermark + append mode

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18440: Assignee: (was: Apache Spark) > Fix FileStreamSink with aggregation + watermark + appe

[jira] [Commented] (SPARK-18440) Fix FileStreamSink with aggregation + watermark + append mode

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665589#comment-15665589 ] Apache Spark commented on SPARK-18440: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-18440) Fix FileStreamSink with aggregation + watermark + append mode

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18440: Assignee: Apache Spark > Fix FileStreamSink with aggregation + watermark + append mode > -

[jira] [Commented] (SPARK-18220) ClassCastException occurs when using select query on ORC file

2016-11-14 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665580#comment-15665580 ] Wenchen Fan commented on SPARK-18220: - do you have the code snippet to reproduce this

[jira] [Updated] (SPARK-18440) Fix FileStreamSink with aggregation + watermark + append mode

2016-11-14 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-18440: -- Description: SPARK-18012 refactored the file write path in FileStreamSink using FileFormatWrit

[jira] [Resolved] (SPARK-18124) Observed delay based event time watermarks

2016-11-14 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-18124. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15702 [https://g

[jira] [Reopened] (SPARK-8133) sticky partitions

2016-11-14 Thread Evan Chan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evan Chan reopened SPARK-8133: -- I think this is worth looking into again - for streaming. My team is creating spark streaming pipelines th

[jira] [Commented] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665547#comment-15665547 ] Hyukjin Kwon commented on SPARK-18437: -- Sounds good and safe. Thanks! > Inconsisten

[jira] [Comment Edited] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665541#comment-15665541 ] Felix Cheung edited comment on SPARK-18437 at 11/15/16 12:37 AM: --

[jira] [Commented] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665541#comment-15665541 ] Felix Cheung commented on SPARK-18437: -- For R, I'm a bit cautious about {code}@note{

[jira] [Created] (SPARK-18440) Fix FileStreamSink with aggregation + watermark + append mode

2016-11-14 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-18440: - Summary: Fix FileStreamSink with aggregation + watermark + append mode Key: SPARK-18440 URL: https://issues.apache.org/jira/browse/SPARK-18440 Project: Spark

[jira] [Assigned] (SPARK-18433) Improve DataSource option keys to be more case-insensitive

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18433: Assignee: Apache Spark > Improve DataSource option keys to be more case-insensitive >

[jira] [Assigned] (SPARK-18433) Improve DataSource option keys to be more case-insensitive

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18433: Assignee: (was: Apache Spark) > Improve DataSource option keys to be more case-insensi

[jira] [Commented] (SPARK-18433) Improve DataSource option keys to be more case-insensitive

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665508#comment-15665508 ] Apache Spark commented on SPARK-18433: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Updated] (SPARK-18439) spark 2.0.1 fails in windows when using file:/// scratchdir in hive-site.xml

2016-11-14 Thread Javier Luraschi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Javier Luraschi updated SPARK-18439: Summary: spark 2.0.1 fails in windows when using file:/// scratchdir in hive-site.xml (was

[jira] [Created] (SPARK-18439) spark 2.0.1 fails in windows when using file:/// scratchfir

2016-11-14 Thread Javier Luraschi (JIRA)
Javier Luraschi created SPARK-18439: --- Summary: spark 2.0.1 fails in windows when using file:/// scratchfir Key: SPARK-18439 URL: https://issues.apache.org/jira/browse/SPARK-18439 Project: Spark

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

2016-11-14 Thread Saikat Kanjilal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665370#comment-15665370 ] Saikat Kanjilal commented on SPARK-9487: I am running the tests by the following c

[jira] [Commented] (SPARK-14138) Generated SpecificColumnarIterator code can exceed JVM size limit for cached DataFrames

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665334#comment-15665334 ] Sean Owen commented on SPARK-14138: --- That does not look like the problem described here

[jira] [Commented] (SPARK-18187) CompactibleFileStreamLog should not rely on "compactInterval" to detect a compaction batch

2016-11-14 Thread Tyson Condie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665239#comment-15665239 ] Tyson Condie commented on SPARK-18187: -- Hi [~zsxwing], I have added some new logic t

[jira] [Commented] (SPARK-18392) LSH API, algorithm, and documentation follow-ups

2016-11-14 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665233#comment-15665233 ] Seth Hendrickson commented on SPARK-18392: -- Thank you for clarifying, I see it n

[jira] [Commented] (SPARK-2141) Add sc.getPersistentRDDs() to PySpark

2016-11-14 Thread Shea Parkes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665137#comment-15665137 ] Shea Parkes commented on SPARK-2141: This would have been nice to have today. We want

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

2016-11-14 Thread Harish (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665010#comment-15665010 ] Harish commented on SPARK-16845: This issue is still open - https://github.com/apache/spa

[jira] [Commented] (SPARK-18433) Improve DataSource option keys to be more case-insensitive

2016-11-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664901#comment-15664901 ] Dongjoon Hyun commented on SPARK-18433: --- Hi, [~smilegator]. In this PR, I'm trying

[jira] [Updated] (SPARK-18433) Improve DataSource option keys to be more case-insensitive

2016-11-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18433: -- Description: Currently, `DataSource.scala` partially use `CaseInsensitiveMap` in code-path. T

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

2016-11-14 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664866#comment-15664866 ] Barry Becker commented on SPARK-16845: -- I am encountering a similar exception in spa

[jira] [Resolved] (SPARK-17348) Incorrect results from subquery transformation

2016-11-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17348. --- Resolution: Fixed Assignee: Nattavut Sutyanyong Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-14138) Generated SpecificColumnarIterator code can exceed JVM size limit for cached DataFrames

2016-11-14 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664842#comment-15664842 ] Barry Becker commented on SPARK-14138: -- I am using spark 1.6.3 on a DataFrame with 2

[jira] [Commented] (SPARK-18374) Incorrect words in StopWords/english.txt

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664807#comment-15664807 ] Sean Owen commented on SPARK-18374: --- [~whisper] do you have a comment here? it does see

[jira] [Commented] (SPARK-18436) isin causing SQL syntax error with JDBC

2016-11-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664803#comment-15664803 ] Herman van Hovell commented on SPARK-18436: --- An empty {{IN}} is currently not f

[jira] [Commented] (SPARK-8443) GenerateMutableProjection Exceeds JVM Code Size Limits

2016-11-14 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664779#comment-15664779 ] Barry Becker commented on SPARK-8443: - I see the same error in spark 1.6.3. Is there a

[jira] [Commented] (SPARK-18436) isin causing SQL syntax error with JDBC

2016-11-14 Thread Dan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664768#comment-15664768 ] Dan commented on SPARK-18436: - That's right, IN () is invalid. IN ('') is the correct syntax

[jira] [Commented] (SPARK-18392) LSH API, algorithm, and documentation follow-ups

2016-11-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664764#comment-15664764 ] Joseph K. Bradley commented on SPARK-18392: --- I thought I suggested saying {{sel

[jira] [Resolved] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

2016-11-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17510. -- Resolution: Fixed Assignee: Cody Koeninger Fix Version/s: 2.1.0 > Set Streaming

[jira] [Resolved] (SPARK-18416) State Store leaks temporary files

2016-11-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-18416. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.3 > State Store leaks temp

[jira] [Comment Edited] (SPARK-18374) Incorrect words in StopWords/english.txt

2016-11-14 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664503#comment-15664503 ] yuhao yang edited comment on SPARK-18374 at 11/14/16 5:44 PM: -

[jira] [Commented] (SPARK-18374) Incorrect words in StopWords/english.txt

2016-11-14 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664503#comment-15664503 ] yuhao yang commented on SPARK-18374: Thanks for the response. By default, _Tokenizer_

[jira] [Commented] (SPARK-17436) dataframe.write sometimes does not keep sorting

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664352#comment-15664352 ] Sean Owen commented on SPARK-17436: --- See recent JIRAs about JavaAPISuite on Windows. I

[jira] [Updated] (SPARK-18436) isin causing SQL syntax error with JDBC

2016-11-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18436: -- Description: When using a JDBC data source, the "isin" function generates invalid SQL s

[jira] [Commented] (SPARK-18438) spark.mlp should support RFormula

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664239#comment-15664239 ] Apache Spark commented on SPARK-18438: -- User 'yanboliang' has created a pull request

[jira] [Assigned] (SPARK-18438) spark.mlp should support RFormula

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18438: Assignee: (was: Apache Spark) > spark.mlp should support RFormula > --

[jira] [Assigned] (SPARK-18438) spark.mlp should support RFormula

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18438: Assignee: Apache Spark > spark.mlp should support RFormula > -

[jira] [Comment Edited] (SPARK-17436) dataframe.write sometimes does not keep sorting

2016-11-14 Thread Ran Haim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664233#comment-15664233 ] Ran Haim edited comment on SPARK-17436 at 11/14/16 3:45 PM: S

[jira] [Commented] (SPARK-17436) dataframe.write sometimes does not keep sorting

2016-11-14 Thread Ran Haim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664233#comment-15664233 ] Ran Haim commented on SPARK-17436: -- So... Can you help? Or at least point me to someone

[jira] [Closed] (SPARK-18435) The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html was not found on this server.

2016-11-14 Thread xubo245 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xubo245 closed SPARK-18435. --- > The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html > was not found on this server. >

[jira] [Created] (SPARK-18438) spark.mlp should support RFormula

2016-11-14 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-18438: --- Summary: spark.mlp should support RFormula Key: SPARK-18438 URL: https://issues.apache.org/jira/browse/SPARK-18438 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-17683) Support ArrayType in Literal.apply

2016-11-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664088#comment-15664088 ] Takeshi Yamamuro commented on SPARK-17683: -- [~rxin] I revisited this issue; if i

[jira] [Commented] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664057#comment-15664057 ] Hyukjin Kwon commented on SPARK-18437: -- Ah, thank you. Let me try to do this separat

[jira] [Commented] (SPARK-18436) isin causing SQL syntax error with JDBC

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664038#comment-15664038 ] Sean Owen commented on SPARK-18436: --- You're saying that " ... IN () " is a SQL error in

[jira] [Commented] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

2016-11-14 Thread Babak Alipour (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664035#comment-15664035 ] Babak Alipour commented on SPARK-17788: --- The details were in the email thread. Here

[jira] [Commented] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664006#comment-15664006 ] Sean Owen commented on SPARK-18437: --- It seems reasonable to standardize this, if you're

[jira] [Reopened] (SPARK-18435) The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html was not found on this server.

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-18435: --- > The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html > was not found on this s

[jira] [Resolved] (SPARK-18435) The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html was not found on this server.

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18435. --- Resolution: Duplicate > The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html

[jira] [Updated] (SPARK-18435) The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html was not found on this server.

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18435: -- Priority: Trivial (was: Major) > The requested URL /docs/latest/api/scala/org/apache/spark/Accumulator

[jira] [Closed] (SPARK-18435) The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html was not found on this server.

2016-11-14 Thread xubo245 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xubo245 closed SPARK-18435. --- Resolution: Fixed > The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html > was not f

[jira] [Updated] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-18437: - Description: It seems in Scala/Java, - {{Note:}} - {{NOTE:}} - {{Note that}} - {{'''Note:'''}

[jira] [Commented] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663968#comment-15663968 ] Hyukjin Kwon commented on SPARK-18437: -- Hi [~felixcheung], what do you think about R

[jira] [Updated] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-18437: - Description: It seems in Scala/Java, - {{Note:}} - {{NOTE:}} - {{Note that}} - {{'''Note:'''}

[jira] [Updated] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-18437: - Description: It seems in Scala/Java, - {{Note:}} - {{NOTE:}} - {{Note that}} - {{'''Note:'''}

[jira] [Updated] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-18437: - Description: It seems in Scala/Java, - {{Note:}} - {{NOTE:}} - {{Note that}} - {{'''Note:'''}

[jira] [Created] (SPARK-18437) Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations

2016-11-14 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-18437: Summary: Inconsistent mark-down for `Note:` across Scala/Java/R/Python in API documentations Key: SPARK-18437 URL: https://issues.apache.org/jira/browse/SPARK-18437 P

[jira] [Commented] (SPARK-18435) The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html was not found on this server.

2016-11-14 Thread xubo245 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663957#comment-15663957 ] xubo245 commented on SPARK-18435: - It has fix in latest code :https://github.com/apache/

[jira] [Commented] (SPARK-18134) SQL: MapType in Group BY and Joins not working

2016-11-14 Thread Christian Zorneck (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663950#comment-15663950 ] Christian Zorneck commented on SPARK-18134: --- We have many use cases. Generally

[jira] [Updated] (SPARK-18436) isin causing SQL syntax error with JDBC

2016-11-14 Thread Dan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan updated SPARK-18436: Priority: Major (was: Blocker) > isin causing SQL syntax error with JDBC > ---

[jira] [Created] (SPARK-18436) isin causing SQL syntax error with JDBC

2016-11-14 Thread Dan (JIRA)
Dan created SPARK-18436: --- Summary: isin causing SQL syntax error with JDBC Key: SPARK-18436 URL: https://issues.apache.org/jira/browse/SPARK-18436 Project: Spark Issue Type: Bug Components: S

[jira] [Created] (SPARK-18435) The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html was not found on this server.

2016-11-14 Thread xubo245 (JIRA)
xubo245 created SPARK-18435: --- Summary: The requested URL /docs/latest/api/scala/org/apache/spark/AccumulatorV2.html was not found on this server. Key: SPARK-18435 URL: https://issues.apache.org/jira/browse/SPARK-18435

[jira] [Commented] (SPARK-18420) Fix the errors caused by lint check in Java

2016-11-14 Thread coneyliu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663890#comment-15663890 ] coneyliu commented on SPARK-18420: -- Thanks for the suggestion, I've updated it. > Fix t

[jira] [Updated] (SPARK-18420) Fix the compile errors caused by checkstyle

2016-11-14 Thread coneyliu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] coneyliu updated SPARK-18420: - Description: What changes were proposed in this pull request? Small fix, fix the errors caused by lint c

[jira] [Updated] (SPARK-18420) Fix the errors caused by lint check in Java

2016-11-14 Thread coneyliu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] coneyliu updated SPARK-18420: - Summary: Fix the errors caused by lint check in Java (was: Fix the compile errors caused by checkstyle)

[jira] [Comment Edited] (SPARK-18050) spark 2.0.1 enable hive throw AlreadyExistsException(message:Database default already exists)

2016-11-14 Thread todd.chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663821#comment-15663821 ] todd.chen edited comment on SPARK-18050 at 11/14/16 12:46 PM: -

[jira] [Commented] (SPARK-18050) spark 2.0.1 enable hive throw AlreadyExistsException(message:Database default already exists)

2016-11-14 Thread todd.chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663828#comment-15663828 ] todd.chen commented on SPARK-18050: --- when I init sparkSession ,throw this error > spa

[jira] [Commented] (SPARK-18050) spark 2.0.1 enable hive throw AlreadyExistsException(message:Database default already exists)

2016-11-14 Thread todd.chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663821#comment-15663821 ] todd.chen commented on SPARK-18050: --- 16/11/14 20:38:03 INFO hive.HiveUtils: Initializin

[jira] [Updated] (SPARK-18372) .Hive-staging folders created from Spark hiveContext are not getting cleaned up

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18372: -- Fix Version/s: (was: 2.0.2) > .Hive-staging folders created from Spark hiveContext are not getting

[jira] [Updated] (SPARK-18396) "Duration" column makes search result confused, maybe we should make it unsearchable

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18396: -- Assignee: Tao Wang > "Duration" column makes search result confused, maybe we should make it > unsearc

[jira] [Updated] (SPARK-18416) State Store leaks temporary files

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18416: -- Target Version/s: 2.1.0 Fix Version/s: (was: 2.1.0) > State Store leaks temporary files > --

[jira] [Updated] (SPARK-18410) Add structured kafka example

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18410: -- Target Version/s: (was: 2.0.2) > Add structured kafka example > > >

[jira] [Updated] (SPARK-18432) Fix HDFS block size in programming guide

2016-11-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18432: -- Priority: Trivial (was: Minor) > Fix HDFS block size in programming guide > --

[jira] [Resolved] (SPARK-18432) Fix HDFS block size in programming guide

2016-11-14 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-18432. Resolution: Fixed Fix Version/s: 2.1.0 2.0.3 > Fix HDFS block siz

[jira] [Updated] (SPARK-18432) Fix HDFS block size in programming guide

2016-11-14 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-18432: --- Assignee: Noritaka Sekiyama > Fix HDFS block size in programming guide >

[jira] [Commented] (SPARK-9896) Parquet Schema Assertion

2016-11-14 Thread Manish Maheshwari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663738#comment-15663738 ] Manish Maheshwari commented on SPARK-9896: -- The issue got fixed automatically. It

[jira] [Assigned] (SPARK-18400) NPE when resharding Kinesis Stream

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18400: Assignee: Apache Spark > NPE when resharding Kinesis Stream >

[jira] [Assigned] (SPARK-18400) NPE when resharding Kinesis Stream

2016-11-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18400: Assignee: (was: Apache Spark) > NPE when resharding Kinesis Stream > -

  1   2   >