[jira] [Commented] (SPARK-23594) Add interpreted execution for GetExternalRowField expression

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387396#comment-16387396 ] Apache Spark commented on SPARK-23594: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23594) Add interpreted execution for GetExternalRowField expression

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23594: Assignee: Apache Spark > Add interpreted execution for GetExternalRowField expression >

[jira] [Assigned] (SPARK-23594) Add interpreted execution for GetExternalRowField expression

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23594: Assignee: (was: Apache Spark) > Add interpreted execution for GetExternalRowField

[jira] [Commented] (SPARK-23594) Add interpreted execution for GetExternalRowField expression

2018-03-05 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387334#comment-16387334 ] Takeshi Yamamuro commented on SPARK-23594: -- If nobody works on this, I'll take this. > Add

[jira] [Commented] (SPARK-23580) Interpreted mode fallback should be implemented for all expressions & projections

2018-03-05 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387315#comment-16387315 ] Takeshi Yamamuro commented on SPARK-23580: -- I'll join this work, too. > Interpreted mode

[jira] [Commented] (SPARK-23545) [Spark-Core] port opened by the SparkDriver is vulnerable for flooding attacks

2018-03-05 Thread sandeep katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387307#comment-16387307 ] sandeep katta commented on SPARK-23545: --- I will be working on this bug, Solution is as follows

[jira] [Commented] (SPARK-23537) Logistic Regression without standardization

2018-03-05 Thread Teng Peng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387276#comment-16387276 ] Teng Peng commented on SPARK-23537: --- This is a quiet interesting question and I do not have answer yet:

[jira] [Commented] (SPARK-23288) Incorrect number of written records in structured streaming

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387266#comment-16387266 ] Apache Spark commented on SPARK-23288: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Commented] (SPARK-23433) java.lang.IllegalStateException: more than one active taskSet for stage

2018-03-05 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387247#comment-16387247 ] guoxiaolongzte commented on SPARK-23433: I also encountered the same problem, who can solve it?

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 1.1.0

2018-03-05 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387245#comment-16387245 ] Cody Koeninger commented on SPARK-18057: It's probably easiest to keep the KIP discussion on the

[jira] [Commented] (SPARK-23582) Add interpreted execution to StaticInvoke expression

2018-03-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387230#comment-16387230 ] Kazuaki Ishizaki commented on SPARK-23582: -- I am working for this > Add interpreted execution

[jira] [Commented] (SPARK-23583) Add interpreted execution to Invoke expression

2018-03-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387231#comment-16387231 ] Kazuaki Ishizaki commented on SPARK-23583: -- I am working for this > Add interpreted execution

[jira] [Commented] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

2018-03-05 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387217#comment-16387217 ] Cody Koeninger commented on SPARK-19767: [~nafshartous] I think at this point people are more

[jira] [Commented] (SPARK-22446) Optimizer causing StringIndexerModel's indexer UDF to throw "Unseen label" exception incorrectly for filtered data.

2018-03-05 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387191#comment-16387191 ] Liang-Chi Hsieh commented on SPARK-22446: - Yeah, sounds good. > Optimizer causing

[jira] [Updated] (SPARK-23603) When the length of the json is in a range,get_json_object will result in missing tail data

2018-03-05 Thread dzcxzl (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-23603: --- Labels: (was: ca) > When the length of the json is in a range,get_json_object will result in > missing

[jira] [Updated] (SPARK-23603) When the length of the json is in a range,get_json_object will result in missing tail data

2018-03-05 Thread dzcxzl (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-23603: --- Labels: ca (was: ) > When the length of the json is in a range,get_json_object will result in > missing

[jira] [Assigned] (SPARK-23608) SHS needs synchronization between attachSparkUI and detachSparkUI functions

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23608: Assignee: Apache Spark > SHS needs synchronization between attachSparkUI and

[jira] [Assigned] (SPARK-23608) SHS needs synchronization between attachSparkUI and detachSparkUI functions

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23608: Assignee: (was: Apache Spark) > SHS needs synchronization between attachSparkUI and

[jira] [Commented] (SPARK-23608) SHS needs synchronization between attachSparkUI and detachSparkUI functions

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387176#comment-16387176 ] Apache Spark commented on SPARK-23608: -- User 'zhouyejoe' has created a pull request for this issue:

[jira] [Commented] (SPARK-23608) SHS needs synchronization between attachSparkUI and detachSparkUI functions

2018-03-05 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387175#comment-16387175 ] Ye Zhou commented on SPARK-23608: - Pull Request: https://github.com/apache/spark/pull/20744 > SHS needs

[jira] [Commented] (SPARK-23608) SHS needs synchronization between attachSparkUI and detachSparkUI functions

2018-03-05 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387162#comment-16387162 ] Ye Zhou commented on SPARK-23608: - I will post a pull request for this minor change. [~vanzin]

[jira] [Created] (SPARK-23608) SHS needs synchronization between attachSparkUI and detachSparkUI functions

2018-03-05 Thread Ye Zhou (JIRA)
Ye Zhou created SPARK-23608: --- Summary: SHS needs synchronization between attachSparkUI and detachSparkUI functions Key: SPARK-23608 URL: https://issues.apache.org/jira/browse/SPARK-23608 Project: Spark

[jira] [Comment Edited] (SPARK-23607) Use HDFS extended attributes to store application summary to improve the Spark History Server performance

2018-03-05 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387126#comment-16387126 ] Ye Zhou edited comment on SPARK-23607 at 3/6/18 1:42 AM: - [~vanzin]   [~zsxwing]

[jira] [Updated] (SPARK-23607) Use HDFS extended attributes to store application summary to improve the Spark History Server performance

2018-03-05 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-23607: Shepherd: (was: Marcelo Vanzin) > Use HDFS extended attributes to store application summary to improve

[jira] [Commented] (SPARK-23607) Use HDFS extended attributes to store application summary to improve the Spark History Server performance

2018-03-05 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387126#comment-16387126 ] Ye Zhou commented on SPARK-23607: - [~vanzin] Any comments? Thanks. > Use HDFS extended attributes to

[jira] [Created] (SPARK-23607) Use HDFS extended attributes to store application summary to improve the Spark History Server performance

2018-03-05 Thread Ye Zhou (JIRA)
Ye Zhou created SPARK-23607: --- Summary: Use HDFS extended attributes to store application summary to improve the Spark History Server performance Key: SPARK-23607 URL: https://issues.apache.org/jira/browse/SPARK-23607

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-03-05 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387083#comment-16387083 ] Edwina Lu commented on SPARK-23206: --- [~assia6], sorry for the delay. I'm planning to submit a pull

[jira] [Commented] (SPARK-16630) Blacklist a node if executors won't launch on it.

2018-03-05 Thread Attila Zsolt Piros (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387071#comment-16387071 ] Attila Zsolt Piros commented on SPARK-16630: Of course we can consider

[jira] [Commented] (SPARK-21812) PySpark ML Models should not depend transfering params from Java

2018-03-05 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387068#comment-16387068 ] Bryan Cutler commented on SPARK-21812: -- Adding SPARK-15009 as an example of how to restructure the

[jira] [Assigned] (SPARK-23604) ParquetInteroperabilityTest timestamp test should use Statistics.hasNonNullValue

2018-03-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23604: -- Assignee: Henry Robinson > ParquetInteroperabilityTest timestamp test should use >

[jira] [Resolved] (SPARK-23604) ParquetInteroperabilityTest timestamp test should use Statistics.hasNonNullValue

2018-03-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23604. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20740

[jira] [Created] (SPARK-23606) Flakey FileBasedDataSourceSuite

2018-03-05 Thread Henry Robinson (JIRA)
Henry Robinson created SPARK-23606: -- Summary: Flakey FileBasedDataSourceSuite Key: SPARK-23606 URL: https://issues.apache.org/jira/browse/SPARK-23606 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-23586) Add interpreted execution for WrapOption expression

2018-03-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-23586. --- Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 2.4.0 > Add

[jira] [Commented] (SPARK-23020) Re-enable Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387047#comment-16387047 ] Apache Spark commented on SPARK-23020: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-23562) RFormula handleInvalid should handle invalid values in non-string columns.

2018-03-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387045#comment-16387045 ] Joseph K. Bradley commented on SPARK-23562: --- [~yogeshgarg] is going to work on this > RFormula

[jira] [Commented] (SPARK-23020) Re-enable Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

2018-03-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387036#comment-16387036 ] Marcelo Vanzin commented on SPARK-23020: Things look pretty stable on master, so I'll post a

[jira] [Assigned] (SPARK-23581) Add an interpreted version of GenerateUnsafeProjection

2018-03-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-23581: - Assignee: Herman van Hovell > Add an interpreted version of

[jira] [Commented] (SPARK-23580) Interpreted mode fallback should be implemented for all expressions & projections

2018-03-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386991#comment-16386991 ] Kazuaki Ishizaki commented on SPARK-23580: -- Sure, I will work for them > Interpreted mode

[jira] [Resolved] (SPARK-18630) PySpark ML memory leak

2018-03-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-18630. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20724

[jira] [Assigned] (SPARK-18630) PySpark ML memory leak

2018-03-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-18630: - Assignee: yogesh garg > PySpark ML memory leak > -- > >

[jira] [Comment Edited] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 1.1.0

2018-03-05 Thread Richard Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386982#comment-16386982 ] Richard Yu edited comment on SPARK-18057 at 3/5/18 11:48 PM: - Hi all, I'm

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 1.1.0

2018-03-05 Thread Richard Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386982#comment-16386982 ] Richard Yu commented on SPARK-18057: Hi all, I'm currently working on KAFKA-6608 which is part of

[jira] [Updated] (SPARK-23604) ParquetInteroperabilityTest timestamp test should use Statistics.hasNonNullValue

2018-03-05 Thread Henry Robinson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Robinson updated SPARK-23604: --- Description: We ran into an issue with a downstream build of Spark running against a custom

[jira] [Commented] (SPARK-23572) Update security.md to cover new features

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386935#comment-16386935 ] Apache Spark commented on SPARK-23572: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23572) Update security.md to cover new features

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23572: Assignee: (was: Apache Spark) > Update security.md to cover new features >

[jira] [Assigned] (SPARK-23572) Update security.md to cover new features

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23572: Assignee: Apache Spark > Update security.md to cover new features >

[jira] [Resolved] (SPARK-23538) Simplify SSL configuration for https client

2018-03-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23538. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20723

[jira] [Assigned] (SPARK-23538) Simplify SSL configuration for https client

2018-03-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23538: -- Assignee: Marcelo Vanzin > Simplify SSL configuration for https client >

[jira] [Resolved] (SPARK-23040) BlockStoreShuffleReader's return Iterator isn't interruptible if aggregator or ordering is specified

2018-03-05 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23040. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20449

[jira] [Assigned] (SPARK-23040) BlockStoreShuffleReader's return Iterator isn't interruptible if aggregator or ordering is specified

2018-03-05 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23040: --- Assignee: Xianjin YE > BlockStoreShuffleReader's return Iterator isn't interruptible if

[jira] [Updated] (SPARK-23434) Spark should not warn `metadata directory` for a HDFS file path

2018-03-05 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-23434: Fix Version/s: 2.2.2 > Spark should not warn `metadata directory` for a HDFS file path >

[jira] [Updated] (SPARK-23605) Conflicting dependencies for janino in 2.3.0

2018-03-05 Thread Tao Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Liu updated SPARK-23605: Labels: maven (was: ) > Conflicting dependencies for janino in 2.3.0 >

[jira] [Created] (SPARK-23605) Conflicting dependencies for janino in 2.3.0

2018-03-05 Thread Tao Liu (JIRA)
Tao Liu created SPARK-23605: --- Summary: Conflicting dependencies for janino in 2.3.0 Key: SPARK-23605 URL: https://issues.apache.org/jira/browse/SPARK-23605 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-23434) Spark should not warn `metadata directory` for a HDFS file path

2018-03-05 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-23434: Fix Version/s: 2.3.1 > Spark should not warn `metadata directory` for a HDFS file path >

[jira] [Commented] (SPARK-23580) Interpreted mode fallback should be implemented for all expressions & projections

2018-03-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386839#comment-16386839 ] Xiao Li commented on SPARK-23580: - cc [~kiszk] Could you help this umbrella ticket? > Interpreted mode

[jira] [Commented] (SPARK-16630) Blacklist a node if executors won't launch on it.

2018-03-05 Thread Attila Zsolt Piros (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386789#comment-16386789 ] Attila Zsolt Piros commented on SPARK-16630: I have checked the existing sources and I would

[jira] [Resolved] (SPARK-23559) add epoch ID to data writer factory

2018-03-05 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23559. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 20710

[jira] [Assigned] (SPARK-23559) add epoch ID to data writer factory

2018-03-05 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-23559: - Assignee: Jose Torres > add epoch ID to data writer factory >

[jira] [Commented] (SPARK-23586) Add interpreted execution for WrapOption expression

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386780#comment-16386780 ] Apache Spark commented on SPARK-23586: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23604) ParquetInteroperabilityTest timestamp test should use Statistics.hasNonNullValue

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23604: Assignee: Apache Spark > ParquetInteroperabilityTest timestamp test should use >

[jira] [Commented] (SPARK-23604) ParquetInteroperabilityTest timestamp test should use Statistics.hasNonNullValue

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386769#comment-16386769 ] Apache Spark commented on SPARK-23604: -- User 'henryr' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23604) ParquetInteroperabilityTest timestamp test should use Statistics.hasNonNullValue

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23604: Assignee: (was: Apache Spark) > ParquetInteroperabilityTest timestamp test should use

[jira] [Created] (SPARK-23604) ParquetInteroperabilityTest timestamp test should use Statistics.hasNonNullValue

2018-03-05 Thread Henry Robinson (JIRA)
Henry Robinson created SPARK-23604: -- Summary: ParquetInteroperabilityTest timestamp test should use Statistics.hasNonNullValue Key: SPARK-23604 URL: https://issues.apache.org/jira/browse/SPARK-23604

[jira] [Updated] (SPARK-23457) Register task completion listeners first for ParquetFileFormat

2018-03-05 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-23457: Fix Version/s: 2.3.1 > Register task completion listeners first for ParquetFileFormat >

[jira] [Resolved] (SPARK-23585) Add interpreted execution for UnwrapOption expression

2018-03-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-23585. --- Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 2.4.0 > Add

[jira] [Resolved] (SPARK-22882) ML test for StructuredStreaming: spark.ml.classification

2018-03-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-22882. --- Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue

[jira] [Assigned] (SPARK-23500) Filters on named_structs could be pushed into scans

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23500: Assignee: Apache Spark > Filters on named_structs could be pushed into scans >

[jira] [Commented] (SPARK-23500) Filters on named_structs could be pushed into scans

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386604#comment-16386604 ] Apache Spark commented on SPARK-23500: -- User 'henryr' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23500) Filters on named_structs could be pushed into scans

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23500: Assignee: (was: Apache Spark) > Filters on named_structs could be pushed into scans >

[jira] [Commented] (SPARK-18791) Stream-Stream Joins

2018-03-05 Thread Yuriy Bondaruk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386583#comment-16386583 ] Yuriy Bondaruk commented on SPARK-18791: Shouldn't it be marked as resolved? Stream-stream joins

[jira] [Commented] (SPARK-15343) NoClassDefFoundError when initializing Spark with YARN

2018-03-05 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386554#comment-16386554 ] ASF GitHub Bot commented on SPARK-15343: Github user kr-arjun commented on the issue:

[jira] [Assigned] (SPARK-22882) ML test for StructuredStreaming: spark.ml.classification

2018-03-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-22882: - Assignee: Weichen Xu > ML test for StructuredStreaming: spark.ml.classification

[jira] [Commented] (SPARK-22883) ML test for StructuredStreaming: spark.ml.feature, A-M

2018-03-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386522#comment-16386522 ] Joseph K. Bradley commented on SPARK-22883: --- Preparing part 2 > ML test for

[jira] [Commented] (SPARK-22446) Optimizer causing StringIndexerModel's indexer UDF to throw "Unseen label" exception incorrectly for filtered data.

2018-03-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386520#comment-16386520 ] Joseph K. Bradley commented on SPARK-22446: --- Maybe not then unless someone complains? >

[jira] [Assigned] (SPARK-22430) Unknown tag warnings when building R docs with Roxygen 6.0.1

2018-03-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-22430: Assignee: Rekha Joshi > Unknown tag warnings when building R docs with Roxygen 6.0.1 >

[jira] [Resolved] (SPARK-22430) Unknown tag warnings when building R docs with Roxygen 6.0.1

2018-03-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-22430. -- Resolution: Fixed Fix Version/s: 2.4.0 Target Version/s: 2.4.0 > Unknown tag

[jira] [Commented] (SPARK-23528) Expose vital statistics of GaussianMixtureModel

2018-03-05 Thread Erich Schubert (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386318#comment-16386318 ] Erich Schubert commented on SPARK-23528: I had only been looking at the mllib API. There is no

[jira] [Commented] (SPARK-23585) Add interpreted execution for UnwrapOption expression

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386275#comment-16386275 ] Apache Spark commented on SPARK-23585: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23585) Add interpreted execution for UnwrapOption expression

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23585: Assignee: Apache Spark > Add interpreted execution for UnwrapOption expression >

[jira] [Assigned] (SPARK-23585) Add interpreted execution for UnwrapOption expression

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23585: Assignee: (was: Apache Spark) > Add interpreted execution for UnwrapOption expression

[jira] [Commented] (SPARK-23603) When the length of the json is in a range,get_json_object will result in missing tail data

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386239#comment-16386239 ] Apache Spark commented on SPARK-23603: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23603) When the length of the json is in a range,get_json_object will result in missing tail data

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23603: Assignee: Apache Spark > When the length of the json is in a range,get_json_object will

[jira] [Commented] (SPARK-23603) When the length of the json is in a range,get_json_object will result in missing tail data

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386235#comment-16386235 ] Apache Spark commented on SPARK-23603: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23603) When the length of the json is in a range,get_json_object will result in missing tail data

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23603: Assignee: (was: Apache Spark) > When the length of the json is in a

[jira] [Commented] (SPARK-23598) WholeStageCodegen can lead to IllegalAccessError calling append for HashAggregateExec

2018-03-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386234#comment-16386234 ] Marco Gaido commented on SPARK-23598: - thanks for reporting this. Actually the one which you designed

[jira] [Created] (SPARK-23603) When the length of the json is in a range,get_json_object will result in missing tail data

2018-03-05 Thread dzcxzl (JIRA)
dzcxzl created SPARK-23603: -- Summary: When the length of the json is in a range,get_json_object will result in missing tail data Key: SPARK-23603 URL: https://issues.apache.org/jira/browse/SPARK-23603

[jira] [Created] (SPARK-23602) PrintToStderr should behave the same in interpreted mode

2018-03-05 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-23602: - Summary: PrintToStderr should behave the same in interpreted mode Key: SPARK-23602 URL: https://issues.apache.org/jira/browse/SPARK-23602 Project: Spark

[jira] [Commented] (SPARK-23600) conda_panda_example test fails to import panda lib with Spark 2.3

2018-03-05 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386177#comment-16386177 ] Hyukjin Kwon commented on SPARK-23600: -- BTW, where is "conda_panda_example.py" ? >

[jira] [Comment Edited] (SPARK-23600) conda_panda_example test fails to import panda lib with Spark 2.3

2018-03-05 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386169#comment-16386169 ] Hyukjin Kwon edited comment on SPARK-23600 at 3/5/18 3:01 PM: -- Let's don't

[jira] [Updated] (SPARK-23600) conda_panda_example test fails to import panda lib with Spark 2.3

2018-03-05 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23600: - Priority: Major (was: Critical) Fix Version/s: (was: 2.3.0) Let's don't set the

[jira] [Resolved] (SPARK-23329) Update the function descriptions with the arguments and returned values of the trigonometric functions

2018-03-05 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23329. -- Resolution: Fixed Fix Version/s: 2.4.0 2.3.1 Fixed in

[jira] [Assigned] (SPARK-23329) Update the function descriptions with the arguments and returned values of the trigonometric functions

2018-03-05 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23329: Assignee: Mihaly Toth > Update the function descriptions with the arguments and returned

[jira] [Resolved] (SPARK-23566) Arguement name fix

2018-03-05 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23566. -- Resolution: Fixed Fix Version/s: 2.4.0 Fixed in

[jira] [Assigned] (SPARK-23566) Arguement name fix

2018-03-05 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23566: Assignee: Anirudh > Arguement name fix > -- > > Key:

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

2018-03-05 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386125#comment-16386125 ] Yuming Wang commented on SPARK-20202: - How about upgrade Hive directly to 2.3.2. In fact, I've

[jira] [Commented] (SPARK-23601) Remove .md5 files from release

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386106#comment-16386106 ] Apache Spark commented on SPARK-23601: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23601) Remove .md5 files from release

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23601: Assignee: Apache Spark (was: Sean Owen) > Remove .md5 files from release >

[jira] [Assigned] (SPARK-23601) Remove .md5 files from release

2018-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23601: Assignee: Sean Owen (was: Apache Spark) > Remove .md5 files from release >

[jira] [Commented] (SPARK-23598) WholeStageCodegen can lead to IllegalAccessError calling append for HashAggregateExec

2018-03-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386100#comment-16386100 ] Herman van Hovell commented on SPARK-23598: --- [~dvogelbacher] Do you have some code which we can

[jira] [Created] (SPARK-23601) Remove .md5 files from release

2018-03-05 Thread Sean Owen (JIRA)
Sean Owen created SPARK-23601: - Summary: Remove .md5 files from release Key: SPARK-23601 URL: https://issues.apache.org/jira/browse/SPARK-23601 Project: Spark Issue Type: Task

  1   2   >