[jira] [Assigned] (SPARK-25129) Revert mapping of com.databricks.spark.avro

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25129: Assignee: Apache Spark > Revert mapping of com.databricks.spark.avro > --

[jira] [Commented] (SPARK-25129) Revert mapping of com.databricks.spark.avro

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582035#comment-16582035 ] Apache Spark commented on SPARK-25129: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-25129) Revert mapping of com.databricks.spark.avro

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25129: Assignee: (was: Apache Spark) > Revert mapping of com.databricks.spark.avro > ---

[jira] [Comment Edited] (SPARK-23194) from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls

2018-08-15 Thread Denis Bolshakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582025#comment-16582025 ] Denis Bolshakov edited comment on SPARK-23194 at 8/16/18 6:33 AM:

[jira] [Created] (SPARK-25129) Revert mapping of com.databricks.spark.avro

2018-08-15 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-25129: -- Summary: Revert mapping of com.databricks.spark.avro Key: SPARK-25129 URL: https://issues.apache.org/jira/browse/SPARK-25129 Project: Spark Issue Type: S

[jira] [Commented] (SPARK-23194) from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls

2018-08-15 Thread Denis Bolshakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582025#comment-16582025 ] Denis Bolshakov commented on SPARK-23194: - [~cloud_fan], [~hyukjin.kwon], do you

[jira] [Resolved] (SPARK-25082) Documentation for Spark Function expm1 is incomplete

2018-08-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25082. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22115 [https://gi

[jira] [Assigned] (SPARK-25082) Documentation for Spark Function expm1 is incomplete

2018-08-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25082: Assignee: Bo Meng > Documentation for Spark Function expm1 is incomplete > --

[jira] [Commented] (SPARK-24918) Executor Plugin API

2018-08-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582009#comment-16582009 ] Felix Cheung commented on SPARK-24918: -- I'd tend to agree with opt in - too many mi

[jira] [Commented] (SPARK-23817) Migrate ORC file format read path to data source V2

2018-08-15 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582005#comment-16582005 ] Gengliang Wang commented on SPARK-23817: [~dongjoon] Thanks! This issue is still

[jira] [Commented] (SPARK-20568) Delete files after processing in structured streaming

2018-08-15 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581979#comment-16581979 ] Jungtaek Lim commented on SPARK-20568: -- For me, the feature looks like the missing

[jira] [Reopened] (SPARK-23817) Migrate ORC file format read path to data source V2

2018-08-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-23817: --- Ping, [~smilegator] and [~Gengliang.Wang] . > Migrate ORC file format read path to data source

[jira] [Commented] (SPARK-25071) BuildSide is coming not as expected with join queries

2018-08-15 Thread Ayush Anubhava (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581959#comment-16581959 ] Ayush Anubhava commented on SPARK-25071: Moreover, any insight would be consider

[jira] [Commented] (SPARK-25071) BuildSide is coming not as expected with join queries

2018-08-15 Thread Ayush Anubhava (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581957#comment-16581957 ] Ayush Anubhava commented on SPARK-25071: Thanks for the information. Please, pon

[jira] [Resolved] (SPARK-25126) OrcFileOperator.getFileReader: avoid creating OrcFile.Reader for all orc files

2018-08-15 Thread Rao Fu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rao Fu resolved SPARK-25126. Resolution: Not A Problem > OrcFileOperator.getFileReader: avoid creating OrcFile.Reader for all orc files

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

2018-08-15 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581929#comment-16581929 ] Gengliang Wang commented on SPARK-24924: As package "org.apache.spark.sql.avro" 

[jira] [Commented] (SPARK-24882) data source v2 API improvement

2018-08-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581851#comment-16581851 ] Wenchen Fan commented on SPARK-24882: - The proposal LGTM. Do we have to include it i

[jira] [Resolved] (SPARK-16617) Upgrade to Avro 1.8.x

2018-08-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16617. - Resolution: Duplicate Fix Version/s: 2.4.0 > Upgrade to Avro 1.8.x >

[jira] [Commented] (SPARK-23654) Cut jets3t as a dependency of spark-core

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581833#comment-16581833 ] Apache Spark commented on SPARK-23654: -- User 'steveloughran' has created a pull req

[jira] [Updated] (SPARK-25104) Validate user specified output schema

2018-08-15 Thread Li Yuanjian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-25104: Description: With code changes in  [https://github.com/apache/spark/pull/21847|https://github.com/

[jira] [Commented] (SPARK-25078) Standalone does not work with spark.authenticate.secret and deploy-mode=cluster

2018-08-15 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581781#comment-16581781 ] Bo Meng commented on SPARK-25078: - what is your suggestion to improve this? > Standalon

[jira] [Updated] (SPARK-25100) Using KryoSerializer and setting registrationRequired true can lead job failed

2018-08-15 Thread Li Yuanjian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-25100: Description: When spark.serializer is `org.apache.spark.serializer.KryoSerializer` and  `spark.kr

[jira] [Commented] (SPARK-25082) Documentation for Spark Function expm1 is incomplete

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581763#comment-16581763 ] Apache Spark commented on SPARK-25082: -- User 'bomeng' has created a pull request fo

[jira] [Assigned] (SPARK-25082) Documentation for Spark Function expm1 is incomplete

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25082: Assignee: Apache Spark > Documentation for Spark Function expm1 is incomplete > -

[jira] [Assigned] (SPARK-25082) Documentation for Spark Function expm1 is incomplete

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25082: Assignee: (was: Apache Spark) > Documentation for Spark Function expm1 is incomplete

[jira] [Commented] (SPARK-24882) data source v2 API improvement

2018-08-15 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581725#comment-16581725 ] Ryan Blue commented on SPARK-24882: --- [~cloud_fan], I've been building more v2 logical

[jira] [Updated] (SPARK-24685) Adjust release scripts to build all versions for older releases

2018-08-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-24685: --- Docs Text: (was: Testing, please ignore.) > Adjust release scripts to build all versions f

[jira] [Updated] (SPARK-24685) Adjust release scripts to build all versions for older releases

2018-08-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-24685: --- Docs Text: Testing, please ignore. > Adjust release scripts to build all versions for older

[jira] [Assigned] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24938: Assignee: (was: Apache Spark) > Understand usage of netty's onheap memory use, even w

[jira] [Assigned] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24938: Assignee: Apache Spark > Understand usage of netty's onheap memory use, even with offheap

[jira] [Commented] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581676#comment-16581676 ] Apache Spark commented on SPARK-24938: -- User 'NiharS' has created a pull request fo

[jira] [Commented] (SPARK-25111) increment kinesis client/producer lib versions & aws-sdk to match

2018-08-15 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581663#comment-16581663 ] Steve Loughran commented on SPARK-25111: FWIW, it'd be interesting to do a follo

[jira] [Commented] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

2018-08-15 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581658#comment-16581658 ] Steve Loughran commented on SPARK-24787: yes,, hsync updating the file length is

[jira] [Resolved] (SPARK-24685) Adjust release scripts to build all versions for older releases

2018-08-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24685. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21661 [https:

[jira] [Assigned] (SPARK-24685) Adjust release scripts to build all versions for older releases

2018-08-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24685: -- Assignee: Marcelo Vanzin > Adjust release scripts to build all versions for older rel

[jira] [Commented] (SPARK-25127) DataSourceV2: Remove SupportsPushDownCatalystFilters

2018-08-15 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581638#comment-16581638 ] Ryan Blue commented on SPARK-25127: --- FYI [~rxin]. > DataSourceV2: Remove SupportsPush

[jira] [Updated] (SPARK-25125) Spark SQL percentile_approx takes longer than Hive version for large datasets

2018-08-15 Thread Mir Ali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mir Ali updated SPARK-25125: Description: The percentile_approx function in Spark SQL takes much longer than the previous Hive impleme

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

2018-08-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581608#comment-16581608 ] Thomas Graves commented on SPARK-24924: --- I'd be ok with that but CSV has been that

[jira] [Assigned] (SPARK-25126) OrcFileOperator.getFileReader: avoid creating OrcFile.Reader for all orc files

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25126: Assignee: (was: Apache Spark) > OrcFileOperator.getFileReader: avoid creating OrcFile

[jira] [Commented] (SPARK-25126) OrcFileOperator.getFileReader: avoid creating OrcFile.Reader for all orc files

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581578#comment-16581578 ] Apache Spark commented on SPARK-25126: -- User 'raofu' has created a pull request for

[jira] [Assigned] (SPARK-25126) OrcFileOperator.getFileReader: avoid creating OrcFile.Reader for all orc files

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25126: Assignee: Apache Spark > OrcFileOperator.getFileReader: avoid creating OrcFile.Reader for

[jira] [Resolved] (SPARK-24819) Fail fast when no enough slots to launch the barrier stage on job submitted

2018-08-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-24819. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22001 [https://

[jira] [Assigned] (SPARK-24819) Fail fast when no enough slots to launch the barrier stage on job submitted

2018-08-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-24819: - Assignee: Jiang Xingbo > Fail fast when no enough slots to launch the barrier stage on

[jira] [Commented] (SPARK-23243) Shuffle+Repartition on an RDD could lead to incorrect answers

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581551#comment-16581551 ] Apache Spark commented on SPARK-23243: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

2018-08-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581546#comment-16581546 ] Dongjoon Hyun commented on SPARK-24924: --- [~tgraves] . In that case, for consistenc

[jira] [Commented] (SPARK-25128) multiple simultaneous job submissions against k8s backend cause driver pods to hang

2018-08-15 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581537#comment-16581537 ] Erik Erlandson commented on SPARK-25128: [~mcheah], [~liyinan926], wdyt? > mult

[jira] [Created] (SPARK-25128) multiple simultaneous job submissions against k8s backend cause driver pods to hang

2018-08-15 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-25128: -- Summary: multiple simultaneous job submissions against k8s backend cause driver pods to hang Key: SPARK-25128 URL: https://issues.apache.org/jira/browse/SPARK-25128

[jira] [Updated] (SPARK-25125) Spark SQL percentile_approx takes longer than Hive version for large datasets

2018-08-15 Thread Mir Ali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mir Ali updated SPARK-25125: Description: The percentile_approx function in Spark SQL takes much longer than the previous Hive impleme

[jira] [Updated] (SPARK-25026) Binary releases should contain some copy of compiled external integration modules

2018-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25026: -- Priority: Minor (was: Major) > Binary releases should contain some copy of compiled external integrat

[jira] [Resolved] (SPARK-25026) Binary releases should contain some copy of compiled external integration modules

2018-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25026. --- Resolution: Won't Fix > Binary releases should contain some copy of compiled external integration >

[jira] [Created] (SPARK-25127) DataSourceV2: Remove SupportsPushDownCatalystFilters

2018-08-15 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-25127: - Summary: DataSourceV2: Remove SupportsPushDownCatalystFilters Key: SPARK-25127 URL: https://issues.apache.org/jira/browse/SPARK-25127 Project: Spark Issue Type: Su

[jira] [Created] (SPARK-25126) OrcFileOperator.getFileReader: avoid creating OrcFile.Reader for all orc files

2018-08-15 Thread Rao Fu (JIRA)
Rao Fu created SPARK-25126: -- Summary: OrcFileOperator.getFileReader: avoid creating OrcFile.Reader for all orc files Key: SPARK-25126 URL: https://issues.apache.org/jira/browse/SPARK-25126 Project: Spark

[jira] [Updated] (SPARK-25125) Spark SQL percentile_approx takes longer than Hive version for large datasets

2018-08-15 Thread Mir Ali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mir Ali updated SPARK-25125: Description: The percentile_approx function in Spark SQL takes much longer than the previous Hive impleme

[jira] [Updated] (SPARK-25125) Spark SQL percentile_approx takes longer than Hive version for large datasets

2018-08-15 Thread Mir Ali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mir Ali updated SPARK-25125: Description: The percentile_approx function in Spark SQL takes much longer than the previous Hive impleme

[jira] [Updated] (SPARK-25125) Spark SQL percentile_approx takes longer than Hive version for large datasets

2018-08-15 Thread Mir Ali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mir Ali updated SPARK-25125: Description: The percentile_approx function in Spark SQL takes much longer than the previous Hive impleme

[jira] [Created] (SPARK-25125) Spark SQL percentile_approx takes longer than Hive version for large datasets

2018-08-15 Thread Mir Ali (JIRA)
Mir Ali created SPARK-25125: --- Summary: Spark SQL percentile_approx takes longer than Hive version for large datasets Key: SPARK-25125 URL: https://issues.apache.org/jira/browse/SPARK-25125 Project: Spark

[jira] [Commented] (SPARK-23908) High-order function: transform(array, function) → array

2018-08-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581364#comment-16581364 ] Marco Gaido commented on SPARK-23908: - [~huaxingao] they are not exposed through the

[jira] [Resolved] (SPARK-25111) increment kinesis client/producer lib versions & aws-sdk to match

2018-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25111. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22099 [https://github.c

[jira] [Assigned] (SPARK-25111) increment kinesis client/producer lib versions & aws-sdk to match

2018-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25111: - Assignee: Steve Loughran > increment kinesis client/producer lib versions & aws-sdk to match >

[jira] [Commented] (SPARK-23908) High-order function: transform(array, function) → array

2018-08-15 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581354#comment-16581354 ] Huaxin Gao commented on SPARK-23908: [~ueshin]  I am just wondering if the newly add

[jira] [Updated] (SPARK-25088) Rest Server default & doc updates

2018-08-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25088: Labels: release-notes (was: ) > Rest Server default & doc updates > - > >

[jira] [Commented] (SPARK-9610) Class and instance weighting for ML

2018-08-15 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581253#comment-16581253 ] Barry Becker commented on SPARK-9610: - All ML models should support having and option

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

2018-08-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581117#comment-16581117 ] Thomas Graves commented on SPARK-24924: --- [~cloud_fan] [~hyukjin.kwon] seems no one

[jira] [Created] (SPARK-25124) VectorSizeHint.size is buggy, breaking streaming pipeline

2018-08-15 Thread Timothy Hunter (JIRA)
Timothy Hunter created SPARK-25124: -- Summary: VectorSizeHint.size is buggy, breaking streaming pipeline Key: SPARK-25124 URL: https://issues.apache.org/jira/browse/SPARK-25124 Project: Spark

[jira] [Commented] (SPARK-25123) SimpleExprValue may cause the loss of a reference

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580868#comment-16580868 ] Apache Spark commented on SPARK-25123: -- User 'mgaido91' has created a pull request

[jira] [Assigned] (SPARK-25123) SimpleExprValue may cause the loss of a reference

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25123: Assignee: (was: Apache Spark) > SimpleExprValue may cause the loss of a reference > -

[jira] [Assigned] (SPARK-25123) SimpleExprValue may cause the loss of a reference

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25123: Assignee: Apache Spark > SimpleExprValue may cause the loss of a reference >

[jira] [Created] (SPARK-25123) SimpleExprValue may cause the loss of a reference

2018-08-15 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-25123: --- Summary: SimpleExprValue may cause the loss of a reference Key: SPARK-25123 URL: https://issues.apache.org/jira/browse/SPARK-25123 Project: Spark Issue Type: B

[jira] [Assigned] (SPARK-25122) Deduplication of supports equals code

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25122: Assignee: (was: Apache Spark) > Deduplication of supports equals code > -

[jira] [Assigned] (SPARK-25122) Deduplication of supports equals code

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25122: Assignee: Apache Spark > Deduplication of supports equals code >

[jira] [Commented] (SPARK-25122) Deduplication of supports equals code

2018-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580827#comment-16580827 ] Apache Spark commented on SPARK-25122: -- User 'mn-mikke' has created a pull request

[jira] [Updated] (SPARK-25122) Deduplication of supports equals code

2018-08-15 Thread Marek Novotny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marek Novotny updated SPARK-25122: -- Description: The method {{*supportEquals}} determining whether elements of a data type could

[jira] [Created] (SPARK-25122) Deduplication of supports equals code

2018-08-15 Thread Marek Novotny (JIRA)
Marek Novotny created SPARK-25122: - Summary: Deduplication of supports equals code Key: SPARK-25122 URL: https://issues.apache.org/jira/browse/SPARK-25122 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-25092) Add RewriteExceptAll, RewriteIntersectAll and RewriteCorrelatedScalarSubquery in the list of nonExcludableRules

2018-08-15 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Biswal updated SPARK-25092: - Summary: Add RewriteExceptAll, RewriteIntersectAll and RewriteCorrelatedScalarSubquery in the li