[jira] [Assigned] (SPARK-24886) Increase Jenkins build time

[ https://issues.apache.org/jira/browse/SPARK-24886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24886: Assignee: Apache Spark (was: Hyukjin Kwon) > Increase Jenkins build time >

[jira] [Assigned] (SPARK-24886) Increase Jenkins build time

[ https://issues.apache.org/jira/browse/SPARK-24886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24886: Assignee: Hyukjin Kwon (was: Apache Spark) > Increase Jenkins build time >

[jira] [Reopened] (SPARK-24886) Increase Jenkins build time

[ https://issues.apache.org/jira/browse/SPARK-24886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-24886: -- Assignee: Hyukjin Kwon Reopened since we hit the time limit issue. > Increase Jenkins

[jira] [Commented] (SPARK-25040) Empty string for double and float types should be nulls in JSON

[ https://issues.apache.org/jira/browse/SPARK-25040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571126#comment-16571126 ] Apache Spark commented on SPARK-25040: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-25040) Empty string for double and float types should be nulls in JSON

[ https://issues.apache.org/jira/browse/SPARK-25040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25040: Assignee: (was: Apache Spark) > Empty string for double and float types should be

[jira] [Assigned] (SPARK-25040) Empty string for double and float types should be nulls in JSON

[ https://issues.apache.org/jira/browse/SPARK-25040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25040: Assignee: Apache Spark > Empty string for double and float types should be nulls in

[jira] [Created] (SPARK-25040) Empty string for double and float types should be nulls in JSON

Hyukjin Kwon created SPARK-25040: Summary: Empty string for double and float types should be nulls in JSON Key: SPARK-25040 URL: https://issues.apache.org/jira/browse/SPARK-25040 Project: Spark

[jira] [Created] (SPARK-25039) Binary comparison behavior should refer to Teradata

Yuming Wang created SPARK-25039: --- Summary: Binary comparison behavior should refer to Teradata Key: SPARK-25039 URL: https://issues.apache.org/jira/browse/SPARK-25039 Project: Spark Issue

[jira] [Commented] (SPARK-23940) High-order function: transform_values(map, function) → map

[ https://issues.apache.org/jira/browse/SPARK-23940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571095#comment-16571095 ] Neha Patil commented on SPARK-23940: Will Publish PR by tomorrow. > High-order function:

[jira] [Assigned] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25038: Assignee: (was: Apache Spark) > Accelerate Spark Plan generation when Spark SQL read

[jira] [Commented] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571081#comment-16571081 ] Apache Spark commented on SPARK-25038: -- User 'habren' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25038: Assignee: Apache Spark > Accelerate Spark Plan generation when Spark SQL read large

[jira] [Commented] (SPARK-25037) plan.transformAllExpressions() doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571047#comment-16571047 ] Hyukjin Kwon commented on SPARK-25037: -- If it's an actual issue after the discussion in the mailing

[jira] [Resolved] (SPARK-25037) plan.transformAllExpressions() doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25037. -- Resolution: Invalid > plan.transformAllExpressions() doesn't transform expressions in nested

[jira] [Commented] (SPARK-25037) plan.transformAllExpressions() doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571046#comment-16571046 ] Hyukjin Kwon commented on SPARK-25037: -- In that case, let's ask this into mailing list

[jira] [Assigned] (SPARK-25018) Use `Co-Authored-By` git trailer in `merge_spark_pr.py`

[ https://issues.apache.org/jira/browse/SPARK-25018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25018: Assignee: DB Tsai > Use `Co-Authored-By` git trailer in `merge_spark_pr.py` >

[jira] [Resolved] (SPARK-25018) Use `Co-Authored-By` git trailer in `merge_spark_pr.py`

[ https://issues.apache.org/jira/browse/SPARK-25018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25018. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21991

[jira] [Resolved] (SPARK-24748) Support for reporting custom metrics via Streaming Query Progress

[ https://issues.apache.org/jira/browse/SPARK-24748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24748. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21721

[jira] [Assigned] (SPARK-24748) Support for reporting custom metrics via Streaming Query Progress

[ https://issues.apache.org/jira/browse/SPARK-24748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24748: Assignee: Arun Mahadevan > Support for reporting custom metrics via Streaming Query

[jira] [Commented] (SPARK-25037) plan.transformAllExpressions() doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571034#comment-16571034 ] Chris O'Hara commented on SPARK-25037: -- If this is expected behavior then please disregard! We

[jira] [Comment Edited] (SPARK-25037) plan.transformAllExpressions() doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571032#comment-16571032 ] Chris O'Hara edited comment on SPARK-25037 at 8/7/18 2:17 AM: -- You're right

[jira] [Commented] (SPARK-25037) plan.transformAllExpressions() doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571032#comment-16571032 ] Chris O'Hara commented on SPARK-25037: -- You're right – we ran into the issue with a custom

[jira] [Assigned] (SPARK-24637) Add metrics regarding state and watermark to dropwizard metrics

[ https://issues.apache.org/jira/browse/SPARK-24637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24637: Assignee: Jungtaek Lim > Add metrics regarding state and watermark to dropwizard metrics

[jira] [Resolved] (SPARK-24637) Add metrics regarding state and watermark to dropwizard metrics

[ https://issues.apache.org/jira/browse/SPARK-24637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24637. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21622

[jira] [Commented] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571027#comment-16571027 ] Jason Guo commented on SPARK-25038: --- [~hyukjin.kwon] Gotcha I will create a PR for this today >

[jira] [Commented] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571026#comment-16571026 ] Hyukjin Kwon commented on SPARK-25038: -- (please avoid to set Critical+ which is usually reserved

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25038: - Priority: Major (was: Critical) > Accelerate Spark Plan generation when Spark SQL read large

[jira] [Commented] (SPARK-25037) plan.transformAllExpressions() doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571024#comment-16571024 ] Hyukjin Kwon commented on SPARK-25037: -- Does this cause an actual issue? catalyst modules are meant

[jira] [Resolved] (SPARK-25028) AnalyzePartitionCommand failed with NPE if value is null

[ https://issues.apache.org/jira/browse/SPARK-25028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25028. -- Resolution: Cannot Reproduce Please reopen if there are reproduce steps updated, or anyone is

[jira] [Commented] (SPARK-25026) Binary releases should contain some copy of compiled external integration modules

[ https://issues.apache.org/jira/browse/SPARK-25026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571018#comment-16571018 ] Hyukjin Kwon commented on SPARK-25026: -- I was wondering about this too. +1 for somehow including

[jira] [Commented] (SPARK-25013) JDBC urls with jdbc:mariadb don't work as expected

[ https://issues.apache.org/jira/browse/SPARK-25013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571017#comment-16571017 ] Hyukjin Kwon commented on SPARK-25013: -- isn't it possible to let someone implement this and leave

[jira] [Resolved] (SPARK-25012) dataframe creation results in matcherror

[ https://issues.apache.org/jira/browse/SPARK-25012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25012. -- Resolution: Duplicate > dataframe creation results in matcherror >

[jira] [Commented] (SPARK-25028) AnalyzePartitionCommand failed with NPE if value is null

[ https://issues.apache.org/jira/browse/SPARK-25028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571005#comment-16571005 ] Achuth Narayan Rajagopal commented on SPARK-25028: -- [~igreenfi], Can you share a test

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Description: When Spark SQL read large amount of data, it take a long time (more than 10 minutes) to

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Attachment: (was: job start original.png) > Accelerate Spark Plan generation when Spark SQL read

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Attachment: (was: issue sql original.png) > Accelerate Spark Plan generation when Spark SQL read

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Attachment: job start original.png job start optimized.png issue sql

[jira] [Commented] (SPARK-25030) SparkSubmit.doSubmit will not return result if the mainClass submitted creates a Timer()

[ https://issues.apache.org/jira/browse/SPARK-25030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570995#comment-16570995 ] Saisai Shao commented on SPARK-25030: - I would like to see more about this issue. >

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Description: When Spark SQL read large amount of data, it take a long time (more than 10 minutes) to

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Attachment: issue sql original.png > Accelerate Spark Plan generation when Spark SQL read large

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Attachment: job start original.png > Accelerate Spark Plan generation when Spark SQL read large

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Attachment: start.png issue.png > Accelerate Spark Plan generation when Spark SQL

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Attachment: (was: start.png) > Accelerate Spark Plan generation when Spark SQL read large amount

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Attachment: (was: issue.png) > Accelerate Spark Plan generation when Spark SQL read large amount

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Description: When Spark SQL read large amount of data, it take a long time (more than 10 minutes) to

[jira] [Updated] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

[ https://issues.apache.org/jira/browse/SPARK-25038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-25038: -- Description: When Spark SQL read large amount of data, it take a long time (more than 10 minutes) to

[jira] [Created] (SPARK-25038) Accelerate Spark Plan generation when Spark SQL read large amount of data

Jason Guo created SPARK-25038: - Summary: Accelerate Spark Plan generation when Spark SQL read large amount of data Key: SPARK-25038 URL: https://issues.apache.org/jira/browse/SPARK-25038 Project: Spark

[jira] [Updated] (SPARK-24906) Adaptively set split size for columnar file to ensure the task read data size fit expectation

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-24906: -- Summary: Adaptively set split size for columnar file to ensure the task read data size fit

[jira] [Updated] (SPARK-25030) SparkSubmit.doSubmit will not return result if the mainClass submitted creates a Timer()

[ https://issues.apache.org/jira/browse/SPARK-25030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25030: -- Summary: SparkSubmit.doSubmit will not return result if the mainClass submitted creates a

[jira] [Commented] (SPARK-25030) SparkSubmit will not return result if the mainClass submitted creates a Timer()

[ https://issues.apache.org/jira/browse/SPARK-25030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570951#comment-16570951 ] Xiangrui Meng commented on SPARK-25030: --- [~jiangxb1987] Could you create a PR to demonstrate the

[jira] [Updated] (SPARK-25037) plan.transformAllExpressions() doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris O'Hara updated SPARK-25037: - Description: Given the following LogicalPlan: {code:java} scala> val plan = spark.sql("SELECT 1

[jira] [Updated] (SPARK-25037) plan.transformAllExpressions() doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris O'Hara updated SPARK-25037: - Description: Given the following LogicalPlan: {code:java} scala> val plan = spark.sql("SELECT 1

[jira] [Updated] (SPARK-25031) The schema of MapType can not be printed correctly

[ https://issues.apache.org/jira/browse/SPARK-25031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25031: Target Version/s: 2.4.0 > The schema of MapType can not be printed correctly >

[jira] [Assigned] (SPARK-23938) High-order function: map_zip_with(map, map, function) → map

[ https://issues.apache.org/jira/browse/SPARK-23938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23938: Assignee: Apache Spark > High-order function: map_zip_with(map, map, function V3>) → map

[jira] [Commented] (SPARK-23938) High-order function: map_zip_with(map, map, function) → map

[ https://issues.apache.org/jira/browse/SPARK-23938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570932#comment-16570932 ] Apache Spark commented on SPARK-23938: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23938) High-order function: map_zip_with(map, map, function) → map

[ https://issues.apache.org/jira/browse/SPARK-23938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23938: Assignee: (was: Apache Spark) > High-order function: map_zip_with(map, map, function

[jira] [Resolved] (SPARK-24996) Use DSL to simplify DeclarativeAggregate

[ https://issues.apache.org/jira/browse/SPARK-24996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24996. - Resolution: Fixed Fix Version/s: 2.4.0 > Use DSL to simplify DeclarativeAggregate >

[jira] [Assigned] (SPARK-24996) Use DSL to simplify DeclarativeAggregate

[ https://issues.apache.org/jira/browse/SPARK-24996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24996: --- Assignee: Marco Gaido > Use DSL to simplify DeclarativeAggregate >

[jira] [Updated] (SPARK-25037) plan.transformAllExpressions() doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris O'Hara updated SPARK-25037: - Summary: plan.transformAllExpressions() doesn't transform expressions in nested

[jira] [Updated] (SPARK-25037) plan.transformAllExpressions doesn't transform expressions in nested SubqueryExpression plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris O'Hara updated SPARK-25037: - Summary: plan.transformAllExpressions doesn't transform expressions in nested

[jira] [Updated] (SPARK-25037) plan.transformAllExpressions doesn't transform expressions in subquery plans

[ https://issues.apache.org/jira/browse/SPARK-25037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris O'Hara updated SPARK-25037: - Description: Given the following LogicalPlan, containing a SubqueryAlias and

[jira] [Created] (SPARK-25037) plan.transformAllExpressions doesn't transform expressions in subquery plans

Chris O'Hara created SPARK-25037: Summary: plan.transformAllExpressions doesn't transform expressions in subquery plans Key: SPARK-25037 URL: https://issues.apache.org/jira/browse/SPARK-25037

[jira] [Comment Edited] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570880#comment-16570880 ] Dongjoon Hyun edited comment on SPARK-24924 at 8/6/18 10:50 PM: Yep. It

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570880#comment-16570880 ] Dongjoon Hyun commented on SPARK-24924: --- Yep. It will work if those 3rd-party packages are rebuilt

[jira] [Resolved] (SPARK-24161) Enable debug package feature on structured streaming

[ https://issues.apache.org/jira/browse/SPARK-24161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-24161. -- Resolution: Fixed Assignee: Jungtaek Lim Fix Version/s: 2.4.0 > Enable debug

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570840#comment-16570840 ] Thomas Graves commented on SPARK-24924: --- so officially the spark api compatibility is only at the

[jira] [Resolved] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-24948. - Resolution: Fixed > SHS filters wrongly some applications due to permission

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-24948: Fix Version/s: 2.4.0 > SHS filters wrongly some applications due to permission

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570770#comment-16570770 ] Sean Owen commented on SPARK-25029: --- Thanks [~shaneknapp] – looks good in that it runs and shows one

[jira] [Commented] (SPARK-24786) Executors not being released after all cached data is unpersisted

[ https://issues.apache.org/jira/browse/SPARK-24786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570755#comment-16570755 ] Apache Spark commented on SPARK-24786: -- User 'dhruve' has created a pull request for this issue:

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570756#comment-16570756 ] Dongjoon Hyun commented on SPARK-24924: --- 1. Theoretically, Spark 2.4 should handle both Hive

[jira] [Assigned] (SPARK-24786) Executors not being released after all cached data is unpersisted

[ https://issues.apache.org/jira/browse/SPARK-24786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24786: Assignee: (was: Apache Spark) > Executors not being released after all cached data

[jira] [Assigned] (SPARK-24786) Executors not being released after all cached data is unpersisted

[ https://issues.apache.org/jira/browse/SPARK-24786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24786: Assignee: Apache Spark > Executors not being released after all cached data is

[jira] [Commented] (SPARK-20286) dynamicAllocation.executorIdleTimeout is ignored after unpersist

[ https://issues.apache.org/jira/browse/SPARK-20286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570754#comment-16570754 ] Apache Spark commented on SPARK-20286: -- User 'dhruve' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20286) dynamicAllocation.executorIdleTimeout is ignored after unpersist

[ https://issues.apache.org/jira/browse/SPARK-20286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20286: Assignee: (was: Apache Spark) > dynamicAllocation.executorIdleTimeout is ignored

[jira] [Assigned] (SPARK-20286) dynamicAllocation.executorIdleTimeout is ignored after unpersist

[ https://issues.apache.org/jira/browse/SPARK-20286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20286: Assignee: Apache Spark > dynamicAllocation.executorIdleTimeout is ignored after

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570736#comment-16570736 ] Thomas Graves commented on SPARK-24924: --- so if the user includes the databricks jar and they

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570712#comment-16570712 ] shane knapp commented on SPARK-25029: - set up a build to help test this out:

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570702#comment-16570702 ] Dongjoon Hyun commented on SPARK-24924: --- For Hive tables, the format name is stored as a table

[jira] [Updated] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25029: -- Priority: Blocker (was: Major) Temporarily making this a blocker for 2.4, as Scala 2.12 support is a

[jira] [Commented] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570684#comment-16570684 ] Apache Spark commented on SPARK-25036: -- User 'kiszk' has created a pull request for this issue:

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570666#comment-16570666 ] Sean Owen commented on SPARK-25029: --- [~skonto] I tried basically that and it looks like resolves the

[jira] [Assigned] (SPARK-23939) High-order function: transform_keys(map, function) → map

[ https://issues.apache.org/jira/browse/SPARK-23939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23939: Assignee: (was: Apache Spark) > High-order function: transform_keys(map, function) →

[jira] [Assigned] (SPARK-23939) High-order function: transform_keys(map, function) → map

[ https://issues.apache.org/jira/browse/SPARK-23939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23939: Assignee: Apache Spark > High-order function: transform_keys(map, function) → > map >

[jira] [Commented] (SPARK-23939) High-order function: transform_keys(map, function) → map

[ https://issues.apache.org/jira/browse/SPARK-23939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570657#comment-16570657 ] Apache Spark commented on SPARK-23939: -- User 'codeatri' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25036: Assignee: (was: Apache Spark) > Scala 2.12 issues: Compilation error with sbt >

[jira] [Assigned] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25036: Assignee: Apache Spark > Scala 2.12 issues: Compilation error with sbt >

[jira] [Commented] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570653#comment-16570653 ] Apache Spark commented on SPARK-25036: -- User 'kiszk' has created a pull request for this issue:

[jira] [Resolved] (SPARK-25019) The published spark sql pom does not exclude the normal version of orc-core

[ https://issues.apache.org/jira/browse/SPARK-25019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-25019. -- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.4.0

[jira] [Resolved] (SPARK-24992) spark should randomize yarn local dir selection

[ https://issues.apache.org/jira/browse/SPARK-24992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-24992. --- Resolution: Fixed Assignee: Hieu Tri Huynh Fix Version/s: 2.4.0 > spark

[jira] [Created] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

Kazuaki Ishizaki created SPARK-25036: Summary: Scala 2.12 issues: Compilation error with sbt Key: SPARK-25036 URL: https://issues.apache.org/jira/browse/SPARK-25036 Project: Spark Issue

[jira] [Issue Comment Deleted] (SPARK-24229) Upgrade to the latest Apache Thrift 0.10.0 release

[ https://issues.apache.org/jira/browse/SPARK-24229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushant Pritmani updated SPARK-24229: - Comment: was deleted (was: Can we have the latest version of 'libfb303' also? As the

[jira] [Commented] (SPARK-24229) Upgrade to the latest Apache Thrift 0.10.0 release

[ https://issues.apache.org/jira/browse/SPARK-24229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570634#comment-16570634 ] Sushant Pritmani commented on SPARK-24229: -- Can we have the latest version of 'libfb303' also? 

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570638#comment-16570638 ] Thomas Graves commented on SPARK-24924: --- So something I just thought of that I want to clarify, is

[jira] [Comment Edited] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570617#comment-16570617 ] Dongjoon Hyun edited comment on SPARK-24924 at 8/6/18 6:39 PM: --- Thank you

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570617#comment-16570617 ] Dongjoon Hyun commented on SPARK-24924: --- Thank you for confirming and giving the right direction

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570586#comment-16570586 ] Thomas Graves commented on SPARK-24924: --- For compatibility we can't remove it unless major

[jira] [Assigned] (SPARK-24822) Python support for barrier execution mode

[ https://issues.apache.org/jira/browse/SPARK-24822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24822: Assignee: Apache Spark > Python support for barrier execution mode >

[jira] [Assigned] (SPARK-24822) Python support for barrier execution mode

[ https://issues.apache.org/jira/browse/SPARK-24822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24822: Assignee: (was: Apache Spark) > Python support for barrier execution mode >

[jira] [Commented] (SPARK-24822) Python support for barrier execution mode

[ https://issues.apache.org/jira/browse/SPARK-24822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570576#comment-16570576 ] Apache Spark commented on SPARK-24822: -- User 'jiangxb1987' has created a pull request for this

  1   2   >