[jira] [Updated] (SPARK-19035) rand() function in case when cause will failed

2016-12-29 Thread Feng Yuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Yuan updated SPARK-19035: -- Description: *In this case:* select case when a=1 then 1 else

[jira] [Updated] (SPARK-19035) rand() function in case when cause will failed

2016-12-29 Thread Feng Yuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Yuan updated SPARK-19035: -- Summary: rand() function in case when cause will failed (was: nested functions in case when statement

[jira] [Updated] (SPARK-19035) nested functions in case when statement will failed

2016-12-29 Thread Feng Yuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Yuan updated SPARK-19035: -- Description: In this case: select case when a=1 then 1 else

[jira] [Updated] (SPARK-19035) nested functions in case when statement will failed

2016-12-29 Thread Feng Yuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Yuan updated SPARK-19035: -- Description: *In this case:* select case when a=1 then 1 else

[jira] [Updated] (SPARK-19035) nested functions in case when statement will failed

2016-12-29 Thread Feng Yuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Yuan updated SPARK-19035: -- Description: In this case: select case when a=1 then 1 else

[jira] [Created] (SPARK-19035) nested functions in case when statement will failed

2016-12-29 Thread Feng Yuan (JIRA)
Feng Yuan created SPARK-19035: - Summary: nested functions in case when statement will failed Key: SPARK-19035 URL: https://issues.apache.org/jira/browse/SPARK-19035 Project: Spark Issue Type:

[jira] [Created] (SPARK-19034) Download packages on 'spark.apache.org/downloads.html' contain release 2.0.2

2016-12-29 Thread Sanjay Dasgupta (JIRA)
Sanjay Dasgupta created SPARK-19034: --- Summary: Download packages on 'spark.apache.org/downloads.html' contain release 2.0.2 Key: SPARK-19034 URL: https://issues.apache.org/jira/browse/SPARK-19034

[jira] [Commented] (SPARK-19033) HistoryServer still uses old ACLs even if ACLs are updated

2016-12-29 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15787031#comment-15787031 ] Saisai Shao commented on SPARK-19033: - Ping [~vanzin], I found that you made this change, would you

[jira] [Updated] (SPARK-19033) HistoryServer still uses old ACLs even if ACLs are updated

2016-12-29 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-19033: Summary: HistoryServer still uses old ACLs even if ACLs are updated (was: HistoryServer will

[jira] [Created] (SPARK-19033) HistoryServer will honor old ACLs even if ACLs are updated

2016-12-29 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-19033: --- Summary: HistoryServer will honor old ACLs even if ACLs are updated Key: SPARK-19033 URL: https://issues.apache.org/jira/browse/SPARK-19033 Project: Spark

[jira] [Commented] (SPARK-18986) ExternalAppendOnlyMap shouldn't fail when forced to spill before calling its iterator

2016-12-29 Thread Sameer Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786971#comment-15786971 ] Sameer Kumar commented on SPARK-18986: -- Shouldn't the priority be increased for this because I am

[jira] [Comment Edited] (SPARK-19032) Non-deterministic results using aggregation first across multiple workers

2016-12-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786883#comment-15786883 ] Liang-Chi Hsieh edited comment on SPARK-19032 at 12/30/16 4:50 AM: --- I

[jira] [Commented] (SPARK-19032) Non-deterministic results using aggregation first across multiple workers

2016-12-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786883#comment-15786883 ] Liang-Chi Hsieh commented on SPARK-19032: - I think you can not guarantee the sort order per group

[jira] [Commented] (SPARK-15359) Mesos dispatcher should handle DRIVER_ABORTED status from mesosDriver.run()

2016-12-29 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786878#comment-15786878 ] Devaraj K commented on SPARK-15359: --- Thanks [~yu2003w] for verifying this PR, I forgot to mention that

[jira] [Commented] (SPARK-19032) Non-deterministic results using aggregation first across multiple workers

2016-12-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786824#comment-15786824 ] Liang-Chi Hsieh commented on SPARK-19032: - There is a related discussion at dev mailing list:

[jira] [Commented] (SPARK-18933) Different log output between Terminal screen and stderr file

2016-12-29 Thread Sean Wong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786822#comment-15786822 ] Sean Wong commented on SPARK-18933: --- But there is no stderr or stdout file available for driver logs.

[jira] [Commented] (SPARK-15359) Mesos dispatcher should handle DRIVER_ABORTED status from mesosDriver.run()

2016-12-29 Thread Jared (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786813#comment-15786813 ] Jared commented on SPARK-15359: --- Hi, I tested the fix. However, it seemed the problem still existed. I1230

[jira] [Commented] (SPARK-19026) local directories cannot be cleanuped when create directory of "executor-***" throws IOException such as there is no more free disk space to create it etc.

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786773#comment-15786773 ] Apache Spark commented on SPARK-19026: -- User 'zuotingbing' has created a pull request for this

[jira] [Assigned] (SPARK-19026) local directories cannot be cleanuped when create directory of "executor-***" throws IOException such as there is no more free disk space to create it etc.

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19026: Assignee: (was: Apache Spark) > local directories cannot be cleanuped when create

[jira] [Assigned] (SPARK-19026) local directories cannot be cleanuped when create directory of "executor-***" throws IOException such as there is no more free disk space to create it etc.

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19026: Assignee: Apache Spark > local directories cannot be cleanuped when create directory of

[jira] [Updated] (SPARK-18974) FileInputDStream could not detected files which moved to the directory

2016-12-29 Thread Adam Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Wang updated SPARK-18974: -- Description: FileInputDStream use mod time to find new files, but if a file was moved into the

[jira] [Commented] (SPARK-18974) FileInputDStream could not detected files which moved to the directory

2016-12-29 Thread Adam Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786735#comment-15786735 ] Adam Wang commented on SPARK-18974: --- Thanks for reminding, I haven't tried before, I will try later >

[jira] [Commented] (SPARK-12757) Use reference counting to prevent blocks from being evicted during reads

2016-12-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786726#comment-15786726 ] Felix Cheung commented on SPARK-12757: -- ping. Still seeing a lot of these messages on Spark 2.1. Is

[jira] [Resolved] (SPARK-17766) Write ahead log exception on a toy project

2016-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17766. -- Resolution: Duplicate This has been fixed in SPARK-18617 > Write ahead log exception on a toy

[jira] [Created] (SPARK-19032) Non-deterministic results using aggregation first across multiple workers

2016-12-29 Thread Harry Weppner (JIRA)
Harry Weppner created SPARK-19032: - Summary: Non-deterministic results using aggregation first across multiple workers Key: SPARK-19032 URL: https://issues.apache.org/jira/browse/SPARK-19032 Project:

[jira] [Created] (SPARK-19031) JDBC Streaming Source

2016-12-29 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-19031: Summary: JDBC Streaming Source Key: SPARK-19031 URL: https://issues.apache.org/jira/browse/SPARK-19031 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-18942) Support output operations for kinesis

2016-12-29 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-18942. -- Resolution: Won't Fix > Support output operations for kinesis >

[jira] [Commented] (SPARK-18942) Support output operations for kinesis

2016-12-29 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786515#comment-15786515 ] Takeshi Yamamuro commented on SPARK-18942: -- okay, I'll put 'won't fix' in this ticket and

[jira] [Commented] (SPARK-18693) BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

2016-12-29 Thread Devesh Parekh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786390#comment-15786390 ] Devesh Parekh commented on SPARK-18693: --- I suggest this is more appropriately classified as a bug

[jira] [Commented] (SPARK-18930) Inserting in partitioned table - partitioned field should be last in select statement.

2016-12-29 Thread Egor Pahomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786368#comment-15786368 ] Egor Pahomov commented on SPARK-18930: -- I'm not sure, that such restriction buried in documentation

[jira] [Commented] (SPARK-18813) MLlib 2.2 Roadmap

2016-12-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786319#comment-15786319 ] Joseph K. Bradley commented on SPARK-18813: --- I just added links to the categories listed above

[jira] [Updated] (SPARK-18813) MLlib 2.2 Roadmap

2016-12-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18813: -- Description: *PROPOSAL: This includes a proposal for the 2.2 roadmap process for

[jira] [Commented] (SPARK-19026) local directories cannot be cleanuped when create directory of "executor-***" throws IOException such as there is no more free disk space to create it etc.

2016-12-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786276#comment-15786276 ] Sean Owen commented on SPARK-19026: --- Can you clarify? I'm not sure what you're proposing here. Maybe a

[jira] [Commented] (SPARK-18930) Inserting in partitioned table - partitioned field should be last in select statement.

2016-12-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786241#comment-15786241 ] Sean Owen commented on SPARK-18930: --- I don't know enough to say that myself. [~epahomov] what's the

[jira] [Updated] (SPARK-19003) Add Java examples in "Spark Streaming Guide", section "Design Patterns for using foreachRDD"

2016-12-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19003: -- Assignee: Tushar Adeshara > Add Java examples in "Spark Streaming Guide", section "Design Patterns

[jira] [Resolved] (SPARK-19003) Add Java examples in "Spark Streaming Guide", section "Design Patterns for using foreachRDD"

2016-12-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19003. --- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 Issue resolved by pull

[jira] [Updated] (SPARK-18693) BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

2016-12-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18693: -- Issue Type: Improvement (was: Bug) > BinaryClassificationEvaluator, RegressionEvaluator, and >

[jira] [Updated] (SPARK-18698) public constructor with uid for IndexToString-class

2016-12-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18698: -- Assignee: Ilya Matiach > public constructor with uid for IndexToString-class >

[jira] [Resolved] (SPARK-18698) public constructor with uid for IndexToString-class

2016-12-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-18698. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16436

[jira] [Created] (SPARK-19030) Dropped event errors being reported after SparkContext has been stopped

2016-12-29 Thread michael procopio (JIRA)
michael procopio created SPARK-19030: Summary: Dropped event errors being reported after SparkContext has been stopped Key: SPARK-19030 URL: https://issues.apache.org/jira/browse/SPARK-19030

[jira] [Commented] (SPARK-16402) JDBC source: Implement save API

2016-12-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786114#comment-15786114 ] Xiao Li commented on SPARK-16402: - Yes. > JDBC source: Implement save API >

[jira] [Closed] (SPARK-16402) JDBC source: Implement save API

2016-12-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-16402. --- Resolution: Duplicate > JDBC source: Implement save API > --- > >

[jira] [Commented] (SPARK-19029) Remove databaseName from SimpleCatalogRelation

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786109#comment-15786109 ] Apache Spark commented on SPARK-19029: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19029) Remove databaseName from SimpleCatalogRelation

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19029: Assignee: Xiao Li (was: Apache Spark) > Remove databaseName from SimpleCatalogRelation

[jira] [Assigned] (SPARK-19029) Remove databaseName from SimpleCatalogRelation

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19029: Assignee: Apache Spark (was: Xiao Li) > Remove databaseName from SimpleCatalogRelation

[jira] [Created] (SPARK-19029) Remove databaseName from SimpleCatalogRelation

2016-12-29 Thread Xiao Li (JIRA)
Xiao Li created SPARK-19029: --- Summary: Remove databaseName from SimpleCatalogRelation Key: SPARK-19029 URL: https://issues.apache.org/jira/browse/SPARK-19029 Project: Spark Issue Type:

[jira] [Updated] (SPARK-19012) CreateOrReplaceTempView throws org.apache.spark.sql.catalyst.parser.ParseException when viewName first char is numerical

2016-12-29 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-19012: -- Affects Version/s: 2.1.0 > CreateOrReplaceTempView throws >

[jira] [Commented] (SPARK-19012) CreateOrReplaceTempView throws org.apache.spark.sql.catalyst.parser.ParseException when viewName first char is numerical

2016-12-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786052#comment-15786052 ] Herman van Hovell commented on SPARK-19012: --- Ok, you could also start a table name with

[jira] [Resolved] (SPARK-19012) CreateOrReplaceTempView throws org.apache.spark.sql.catalyst.parser.ParseException when viewName first char is numerical

2016-12-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19012. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-18669) Update Apache docs regard watermarking in Structured Streaming

2016-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-18669. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > Update Apache docs

[jira] [Commented] (SPARK-19012) CreateOrReplaceTempView throws org.apache.spark.sql.catalyst.parser.ParseException when viewName first char is numerical

2016-12-29 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786015#comment-15786015 ] Dongjoon Hyun commented on SPARK-19012: --- Yep. I tried to update the annotation but unfortunately it

[jira] [Commented] (SPARK-18693) BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

2016-12-29 Thread Ilya Matiach (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786013#comment-15786013 ] Ilya Matiach commented on SPARK-18693: -- Many classifiers in ml don't seem to support weight columns

[jira] [Updated] (SPARK-18698) public constructor with uid for IndexToString-class

2016-12-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18698: -- Shepherd: Joseph K. Bradley Affects Version/s: (was: 2.0.2)

[jira] [Updated] (SPARK-18698) public constructor with uid for IndexToString-class

2016-12-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18698: -- Issue Type: Improvement (was: Wish) > public constructor with uid for

[jira] [Commented] (SPARK-18805) InternalMapWithStateDStream make java.lang.StackOverflowError

2016-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786000#comment-15786000 ] Shixiong Zhu commented on SPARK-18805: -- @etienne That should be not an infinite loop. The time is

[jira] [Commented] (SPARK-16402) JDBC source: Implement save API

2016-12-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785990#comment-15785990 ] Nicholas Chammas commented on SPARK-16402: -- [~JustinPihony], [~smilegator] - Does the resolution

[jira] [Updated] (SPARK-18693) BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

2016-12-29 Thread Devesh Parekh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devesh Parekh updated SPARK-18693: -- Description: The LogisticRegression and LinearRegression models support training with a weight

[jira] [Commented] (SPARK-18942) Support output operations for kinesis

2016-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785953#comment-15785953 ] Shixiong Zhu commented on SPARK-18942: -- Thanks for your prototype. Actually, you can just implement

[jira] [Commented] (SPARK-15493) Allow setting the quoteEscapingEnabled flag when writing CSV

2016-12-29 Thread Jacob Wellington (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785951#comment-15785951 ] Jacob Wellington commented on SPARK-15493: -- I'm running into an issue where this doesn't seem to

[jira] [Commented] (SPARK-18974) FileInputDStream could not detected files which moved to the directory

2016-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785930#comment-15785930 ] Shixiong Zhu commented on SPARK-18974: -- Do you want to try Structured Streaming? Its

[jira] [Updated] (SPARK-18359) Let user specify locale in CSV parsing

2016-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18359: - Component/s: (was: Spark Core) > Let user specify locale in CSV parsing >

[jira] [Commented] (SPARK-18404) RPC call from executor to driver blocks when getting map output locations (Netty Only)

2016-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785914#comment-15785914 ] Shixiong Zhu commented on SPARK-18404: -- That's pretty weird. It's a blocking call for both netty and

[jira] [Updated] (SPARK-19012) CreateOrReplaceTempView throws org.apache.spark.sql.catalyst.parser.ParseException when viewName first char is numerical

2016-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19012: - Component/s: (was: Spark Core) SQL > CreateOrReplaceTempView throws >

[jira] [Assigned] (SPARK-19028) Fixed non-thread-safe functions used in SessionCatalog

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19028: Assignee: Xiao Li (was: Apache Spark) > Fixed non-thread-safe functions used in

[jira] [Assigned] (SPARK-19028) Fixed non-thread-safe functions used in SessionCatalog

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19028: Assignee: Apache Spark (was: Xiao Li) > Fixed non-thread-safe functions used in

[jira] [Commented] (SPARK-19028) Fixed non-thread-safe functions used in SessionCatalog

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785827#comment-15785827 ] Apache Spark commented on SPARK-19028: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Created] (SPARK-19028) Fixed non-thread-safe functions used in SessionCatalog

2016-12-29 Thread Xiao Li (JIRA)
Xiao Li created SPARK-19028: --- Summary: Fixed non-thread-safe functions used in SessionCatalog Key: SPARK-19028 URL: https://issues.apache.org/jira/browse/SPARK-19028 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-18737) Serialization setting "spark.serializer" ignored in Spark 2.x

2016-12-29 Thread Josh Bacon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785715#comment-15785715 ] Josh Bacon commented on SPARK-18737: I think this issue may be related to the following issues:

[jira] [Commented] (SPARK-18883) FileNotFoundException on _temporary directory

2016-12-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785634#comment-15785634 ] Steve Loughran commented on SPARK-18883: thanks, good to know > FileNotFoundException on

[jira] [Assigned] (SPARK-18698) public constructor with uid for IndexToString-class

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18698: Assignee: Apache Spark > public constructor with uid for IndexToString-class >

[jira] [Commented] (SPARK-16494) Upgrade breeze version to 0.12

2016-12-29 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785629#comment-15785629 ] koert kuipers commented on SPARK-16494: --- i just ran into an issue because of this when trying to

[jira] [Assigned] (SPARK-18698) public constructor with uid for IndexToString-class

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18698: Assignee: (was: Apache Spark) > public constructor with uid for IndexToString-class >

[jira] [Commented] (SPARK-18698) public constructor with uid for IndexToString-class

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785627#comment-15785627 ] Apache Spark commented on SPARK-18698: -- User 'imatiach-msft' has created a pull request for this

[jira] [Assigned] (SPARK-19027) estimate size of object buffer for object hash aggregate

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19027: Assignee: Wenchen Fan (was: Apache Spark) > estimate size of object buffer for object

[jira] [Commented] (SPARK-19027) estimate size of object buffer for object hash aggregate

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785604#comment-15785604 ] Apache Spark commented on SPARK-19027: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19027) estimate size of object buffer for object hash aggregate

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19027: Assignee: Apache Spark (was: Wenchen Fan) > estimate size of object buffer for object

[jira] [Created] (SPARK-19027) estimate size of object buffer for object hash aggregate

2016-12-29 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-19027: --- Summary: estimate size of object buffer for object hash aggregate Key: SPARK-19027 URL: https://issues.apache.org/jira/browse/SPARK-19027 Project: Spark Issue

[jira] [Commented] (SPARK-17346) Kafka 0.10 support in Structured Streaming

2016-12-29 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785560#comment-15785560 ] koert kuipers commented on SPARK-17346: --- this ticket mentions kafka 0.10-based sinks for structured

[jira] [Commented] (SPARK-18698) public constructor with uid for IndexToString-class

2016-12-29 Thread Ilya Matiach (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785558#comment-15785558 ] Ilya Matiach commented on SPARK-18698: -- This looks like a minor bug... similar transformers have

[jira] [Commented] (SPARK-17645) Add feature selector methods based on: False Discovery Rate (FDR) and Family Wise Error rate (FWE)

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785524#comment-15785524 ] Apache Spark commented on SPARK-17645: -- User 'mpjlu' has created a pull request for this issue:

[jira] [Commented] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2016-12-29 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785496#comment-15785496 ] Nattavut Sutyanyong commented on SPARK-19017: - In 3-value logic, true OR unknown = true.

[jira] [Issue Comment Deleted] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2016-12-29 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nattavut Sutyanyong updated SPARK-19017: Comment: was deleted (was: In 3-value logic, true OR unknown = true. Using your

[jira] [Commented] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2016-12-29 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785494#comment-15785494 ] Nattavut Sutyanyong commented on SPARK-19017: - In 3-value logic, true OR unknown = true.

[jira] [Commented] (SPARK-18693) BinaryClassificationEvaluator, RegressionEvaluator, and MulticlassClassificationEvaluator should use sample weight data

2016-12-29 Thread Ilya Matiach (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785469#comment-15785469 ] Ilya Matiach commented on SPARK-18693: -- I can take a look into fixing this issue. >

[jira] [Commented] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2016-12-29 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785436#comment-15785436 ] Herman van Hovell commented on SPARK-19017: --- Ok, that is fair. Let me correct my mistake.

[jira] [Commented] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2016-12-29 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785417#comment-15785417 ] Nattavut Sutyanyong commented on SPARK-19017: - Using your interpretation, (2,1) not in (2,0)

[jira] [Commented] (SPARK-19026) local directories cannot be cleanuped when create directory of "executor-***" throws IOException such as there is no more free disk space to create it etc.

2016-12-29 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785220#comment-15785220 ] zuotingbing commented on SPARK-19026: - i will commit the code after this issues be accepted. > local

[jira] [Updated] (SPARK-19026) local directories cannot be cleanuped when create directory of "executor-***" throws IOException such as there is no more free disk space to create it etc.

2016-12-29 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-19026: Description: i set SPARK_LOCAL_DIRS variable like this:

[jira] [Created] (SPARK-19026) local directories cannot be cleanuped when create directory of "executor-***" throws IOException such as there is no more free disk space to create it etc.

2016-12-29 Thread zuotingbing (JIRA)
zuotingbing created SPARK-19026: --- Summary: local directories cannot be cleanuped when create directory of "executor-***" throws IOException such as there is no more free disk space to create it etc. Key: SPARK-19026

[jira] [Created] (SPARK-19025) Remove SQL builder for operators

2016-12-29 Thread Jiang Xingbo (JIRA)
Jiang Xingbo created SPARK-19025: Summary: Remove SQL builder for operators Key: SPARK-19025 URL: https://issues.apache.org/jira/browse/SPARK-19025 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-19024) Don't generate SQL query on CREATE/ALTER a view

2016-12-29 Thread Jiang Xingbo (JIRA)
Jiang Xingbo created SPARK-19024: Summary: Don't generate SQL query on CREATE/ALTER a view Key: SPARK-19024 URL: https://issues.apache.org/jira/browse/SPARK-19024 Project: Spark Issue Type:

[jira] [Created] (SPARK-19023) Memory leak on GraphX with an iterative algorithm and checkpoint on the graph

2016-12-29 Thread Julien MASSIOT (JIRA)
Julien MASSIOT created SPARK-19023: -- Summary: Memory leak on GraphX with an iterative algorithm and checkpoint on the graph Key: SPARK-19023 URL: https://issues.apache.org/jira/browse/SPARK-19023

[jira] [Assigned] (SPARK-19022) Fix tests dependent on OS due to different newline characters

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19022: Assignee: Apache Spark > Fix tests dependent on OS due to different newline characters >

[jira] [Assigned] (SPARK-19022) Fix tests dependent on OS due to different newline characters

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19022: Assignee: (was: Apache Spark) > Fix tests dependent on OS due to different newline

[jira] [Commented] (SPARK-19022) Fix tests dependent on OS due to different newline characters

2016-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784908#comment-15784908 ] Apache Spark commented on SPARK-19022: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-19022) Fix tests dependent on OS due to different newline characters

2016-12-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784894#comment-15784894 ] Hyukjin Kwon commented on SPARK-19022: -- It seems these are (almost) all instances across the tests

[jira] [Created] (SPARK-19022) Fix tests dependent on OS due to different newline characters

2016-12-29 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-19022: Summary: Fix tests dependent on OS due to different newline characters Key: SPARK-19022 URL: https://issues.apache.org/jira/browse/SPARK-19022 Project: Spark

[jira] [Updated] (SPARK-18801) Support resolve a nested view

2016-12-29 Thread Jiang Xingbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiang Xingbo updated SPARK-18801: - Summary: Support resolve a nested view (was: Add `View` operator to help resolve a view) >

[jira] [Updated] (SPARK-18801) Add `View` operator to help resolve a view

2016-12-29 Thread Jiang Xingbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiang Xingbo updated SPARK-18801: - Description: We should be able to resolve a nested view. The main advantage is that if you

[jira] [Updated] (SPARK-18801) Add `View` operator to help resolve a view

2016-12-29 Thread Jiang Xingbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiang Xingbo updated SPARK-18801: - Description: We should be able to resolve a nested view. The main advantage is that if you

  1   2   >