[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550266#comment-16550266 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] I'm still not sure how to handle memory per

[jira] [Resolved] (SPARK-24861) create corrected temp directories in RateSourceSuite

[ https://issues.apache.org/jira/browse/SPARK-24861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24861. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21817

[jira] [Commented] (SPARK-24871) Refactor Concat and MapConcat to avoid creating concatenator object for each row.

[ https://issues.apache.org/jira/browse/SPARK-24871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550218#comment-16550218 ] Apache Spark commented on SPARK-24871: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24871) Refactor Concat and MapConcat to avoid creating concatenator object for each row.

[ https://issues.apache.org/jira/browse/SPARK-24871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24871: Assignee: Apache Spark > Refactor Concat and MapConcat to avoid creating concatenator

[jira] [Assigned] (SPARK-24871) Refactor Concat and MapConcat to avoid creating concatenator object for each row.

[ https://issues.apache.org/jira/browse/SPARK-24871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24871: Assignee: (was: Apache Spark) > Refactor Concat and MapConcat to avoid creating

[jira] [Created] (SPARK-24871) Refactor Concat and MapConcat to avoid creating concatenator object for each row.

Takuya Ueshin created SPARK-24871: - Summary: Refactor Concat and MapConcat to avoid creating concatenator object for each row. Key: SPARK-24871 URL: https://issues.apache.org/jira/browse/SPARK-24871

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550182#comment-16550182 ] Saisai Shao commented on SPARK-24723: - Hi [~mengxr], I don't think YARN has such feature to 

[jira] [Assigned] (SPARK-24195) sc.addFile for local:/ path is broken

[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24195: --- Assignee: Li Yuanjian > sc.addFile for local:/ path is broken >

[jira] [Resolved] (SPARK-24195) sc.addFile for local:/ path is broken

[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24195. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21533

[jira] [Commented] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550167#comment-16550167 ] Saisai Shao commented on SPARK-24307: - Issue resolved by pull request 21440

[jira] [Resolved] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24307. - Resolution: Fixed Assignee: Imran Rashid > Support sending messages over 2GB from memory

[jira] [Updated] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24307: Fix Version/s: 2.4.0 > Support sending messages over 2GB from memory >

[jira] [Updated] (SPARK-24037) stateful operators

[ https://issues.apache.org/jira/browse/SPARK-24037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24037: Fix Version/s: (was: 2.4.0) > stateful operators > -- > >

[jira] [Updated] (SPARK-24037) stateful operators

[ https://issues.apache.org/jira/browse/SPARK-24037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24037: Fix Version/s: 2.4.0 > stateful operators > -- > > Key:

[jira] [Comment Edited] (SPARK-24859) Predicates pushdown on outer joins

[ https://issues.apache.org/jira/browse/SPARK-24859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550159#comment-16550159 ] Hyukjin Kwon edited comment on SPARK-24859 at 7/20/18 3:04 AM: --- Does the

[jira] [Commented] (SPARK-24859) Predicates pushdown on outer joins

[ https://issues.apache.org/jira/browse/SPARK-24859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550159#comment-16550159 ] Hyukjin Kwon commented on SPARK-24859: -- Does the same thing happens in Apache Spark's master

[jira] [Commented] (SPARK-24859) Predicates pushdown on outer joins

[ https://issues.apache.org/jira/browse/SPARK-24859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550158#comment-16550158 ] Hyukjin Kwon commented on SPARK-24859: -- Please avoid set {{Criticial}}+ which is usually reserved

[jira] [Updated] (SPARK-24859) Predicates pushdown on outer joins

[ https://issues.apache.org/jira/browse/SPARK-24859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24859: - Priority: Major (was: Critical) > Predicates pushdown on outer joins >

[jira] [Resolved] (SPARK-24856) spark need upgrade Guava for use gRPC

[ https://issues.apache.org/jira/browse/SPARK-24856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24856. -- Resolution: Duplicate Please search JIRAs before filling it. > spark need upgrade Guava for

[jira] [Commented] (SPARK-24857) required the sample code test the spark steaming job in kubernates and write the data in remote hdfs file system

[ https://issues.apache.org/jira/browse/SPARK-24857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550153#comment-16550153 ] Hyukjin Kwon commented on SPARK-24857: -- [~kmkrishna1...@gmail.com], mind clarifying the input

[jira] [Assigned] (SPARK-24870) Cache can't work normally if there are case letters in SQL

[ https://issues.apache.org/jira/browse/SPARK-24870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24870: Assignee: (was: Apache Spark) > Cache can't work normally if there are case letters

[jira] [Commented] (SPARK-24870) Cache can't work normally if there are case letters in SQL

[ https://issues.apache.org/jira/browse/SPARK-24870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550131#comment-16550131 ] Apache Spark commented on SPARK-24870: -- User 'eatoncys' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24870) Cache can't work normally if there are case letters in SQL

[ https://issues.apache.org/jira/browse/SPARK-24870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24870: Assignee: Apache Spark > Cache can't work normally if there are case letters in SQL >

[jira] [Created] (SPARK-24870) Cache can't work normally if there are case letters in SQL

eaton created SPARK-24870: - Summary: Cache can't work normally if there are case letters in SQL Key: SPARK-24870 URL: https://issues.apache.org/jira/browse/SPARK-24870 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-24847) ScalaReflection#schemaFor occasionally fails to detect schema for Seq of type alias

[ https://issues.apache.org/jira/browse/SPARK-24847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550091#comment-16550091 ] Liang-Chi Hsieh commented on SPARK-24847: - I can't reproduce this currently. >

[jira] [Commented] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550060#comment-16550060 ] Dongjoon Hyun commented on SPARK-24523: --- Hi, [~umayr_nuna]. Do you still see the same situation

[jira] [Resolved] (SPARK-24726) Discuss necessary info and access in barrier mode + Standalone

[ https://issues.apache.org/jira/browse/SPARK-24726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-24726. --- Resolution: Resolved Target Version/s: 2.4.0 (was: 3.0.0) > Discuss necessary

[jira] [Commented] (SPARK-24726) Discuss necessary info and access in barrier mode + Standalone

[ https://issues.apache.org/jira/browse/SPARK-24726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550050#comment-16550050 ] Xiangrui Meng commented on SPARK-24726: --- I'm closing this ticket as resolved since with

[jira] [Assigned] (SPARK-24726) Discuss necessary info and access in barrier mode + Standalone

[ https://issues.apache.org/jira/browse/SPARK-24726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-24726: - Assignee: Xiangrui Meng > Discuss necessary info and access in barrier mode +

[jira] [Updated] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24723: -- Description: In barrier mode, to run hybrid distributed DL training jobs, we need to provide

[jira] [Commented] (SPARK-24724) Discuss necessary info and access in barrier mode + Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-24724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550049#comment-16550049 ] Xiangrui Meng commented on SPARK-24724: --- [~liyinan926] Any updates? > Discuss necessary info and

[jira] [Assigned] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-24723: - Assignee: Saisai Shao > Discuss necessary info and access in barrier mode + YARN >

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550048#comment-16550048 ] Xiangrui Meng commented on SPARK-24723: --- [~jerryshao] Does YARN have the feature that will by

[jira] [Commented] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

[ https://issues.apache.org/jira/browse/SPARK-24869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550046#comment-16550046 ] Xiao Li commented on SPARK-24869: - Thanks! [~maropu] > SaveIntoDataSourceCommand's input Dataset does

[jira] [Assigned] (SPARK-24865) Remove AnalysisBarrier

[ https://issues.apache.org/jira/browse/SPARK-24865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24865: Assignee: (was: Apache Spark) > Remove AnalysisBarrier > -- > >

[jira] [Assigned] (SPARK-24865) Remove AnalysisBarrier

[ https://issues.apache.org/jira/browse/SPARK-24865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24865: Assignee: Apache Spark > Remove AnalysisBarrier > -- > >

[jira] [Commented] (SPARK-24865) Remove AnalysisBarrier

[ https://issues.apache.org/jira/browse/SPARK-24865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550047#comment-16550047 ] Apache Spark commented on SPARK-24865: -- User 'rxin' has created a pull request for this issue:

[jira] [Updated] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24867: Priority: Blocker (was: Major) > Add AnalysisBarrier to DataFrameWriter >

[jira] [Assigned] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24867: Assignee: Xiao Li (was: Apache Spark) > Add AnalysisBarrier to DataFrameWriter >

[jira] [Commented] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550034#comment-16550034 ] Apache Spark commented on SPARK-24867: -- User 'gatorsmile' has created a pull request for this

[jira] [Assigned] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24867: Assignee: Apache Spark (was: Xiao Li) > Add AnalysisBarrier to DataFrameWriter >

[jira] [Commented] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

[ https://issues.apache.org/jira/browse/SPARK-24869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550029#comment-16550029 ] Takeshi Yamamuro commented on SPARK-24869: -- ok > SaveIntoDataSourceCommand's input Dataset

[jira] [Commented] (SPARK-6459) Warn when Column API is constructing trivially true equality

[ https://issues.apache.org/jira/browse/SPARK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550024#comment-16550024 ] nirav patel commented on SPARK-6459: [~zero323] why the example you gave should generate cartesian

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550023#comment-16550023 ] Xiangrui Meng commented on SPARK-24615: --- [~tgraves] Could you help link some past requests on

[jira] [Commented] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

[ https://issues.apache.org/jira/browse/SPARK-24869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550020#comment-16550020 ] Xiao Li commented on SPARK-24869: - cc [~maropu] Do you want to make a try? >

[jira] [Created] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

Xiao Li created SPARK-24869: --- Summary: SaveIntoDataSourceCommand's input Dataset does not use Cached Data Key: SPARK-24869 URL: https://issues.apache.org/jira/browse/SPARK-24869 Project: Spark

[jira] [Assigned] (SPARK-24868) add sequence function in Python

[ https://issues.apache.org/jira/browse/SPARK-24868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24868: Assignee: (was: Apache Spark) > add sequence function in Python >

[jira] [Commented] (SPARK-24868) add sequence function in Python

[ https://issues.apache.org/jira/browse/SPARK-24868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550014#comment-16550014 ] Apache Spark commented on SPARK-24868: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24868) add sequence function in Python

[ https://issues.apache.org/jira/browse/SPARK-24868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24868: Assignee: Apache Spark > add sequence function in Python >

[jira] [Created] (SPARK-24868) add sequence function in Python

Huaxin Gao created SPARK-24868: -- Summary: add sequence function in Python Key: SPARK-24868 URL: https://issues.apache.org/jira/browse/SPARK-24868 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

Xiao Li created SPARK-24867: --- Summary: Add AnalysisBarrier to DataFrameWriter Key: SPARK-24867 URL: https://issues.apache.org/jira/browse/SPARK-24867 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24866) Artifactual ROC scores when scaling up Random Forest classifier

[ https://issues.apache.org/jira/browse/SPARK-24866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evan Zamir updated SPARK-24866: --- Description: I'm encountering a very strange behavior that I can't explain away other than a bug

[jira] [Created] (SPARK-24866) Artifactual ROC scores when scaling up Random Forest classifier

Evan Zamir created SPARK-24866: -- Summary: Artifactual ROC scores when scaling up Random Forest classifier Key: SPARK-24866 URL: https://issues.apache.org/jira/browse/SPARK-24866 Project: Spark

[jira] [Updated] (SPARK-24865) Remove AnalysisBarrier

[ https://issues.apache.org/jira/browse/SPARK-24865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-24865: Affects Version/s: 2.3.0 > Remove AnalysisBarrier > -- > >

[jira] [Updated] (SPARK-24865) Remove AnalysisBarrier

[ https://issues.apache.org/jira/browse/SPARK-24865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-24865: Description: AnalysisBarrier was introduced in SPARK-20392 to improve analysis speed (don't

[jira] [Created] (SPARK-24865) Remove AnalysisBarrier

Reynold Xin created SPARK-24865: --- Summary: Remove AnalysisBarrier Key: SPARK-24865 URL: https://issues.apache.org/jira/browse/SPARK-24865 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-24864) Cannot resolve auto-generated column ordinals in a hive view

[ https://issues.apache.org/jira/browse/SPARK-24864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Madav updated SPARK-24864: --- Description: Spark job reading from a hive-view fails with analysis exception when

[jira] [Updated] (SPARK-24864) Cannot resolve auto-generated column ordinals in a hive view

[ https://issues.apache.org/jira/browse/SPARK-24864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Madav updated SPARK-24864: --- Summary: Cannot resolve auto-generated column ordinals in a hive view (was: Cannot

[jira] [Updated] (SPARK-24864) Cannot reference auto-generated column ordinals in a hive view

[ https://issues.apache.org/jira/browse/SPARK-24864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Madav updated SPARK-24864: --- Summary: Cannot reference auto-generated column ordinals in a hive view (was: Cannot

[jira] [Updated] (SPARK-24864) Cannot reference auto-generated column ordinals in a hive-view.

[ https://issues.apache.org/jira/browse/SPARK-24864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Madav updated SPARK-24864: --- Description: Spark job reading from a hive-view fails with analysis exception when

[jira] [Resolved] (SPARK-24846) Stabilize expression cannonicalization

[ https://issues.apache.org/jira/browse/SPARK-24846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-24846. --- Resolution: Fixed Fix Version/s: 2.4.0 Fixed by gvr's PR. I could not find

[jira] [Created] (SPARK-24864) Cannot reference auto-generated column ordinals in a hive-view.

Abhishek Madav created SPARK-24864: -- Summary: Cannot reference auto-generated column ordinals in a hive-view. Key: SPARK-24864 URL: https://issues.apache.org/jira/browse/SPARK-24864 Project: Spark

[jira] [Assigned] (SPARK-24863) Report offset lag as a custom metrics for Kafka structured streaming source

[ https://issues.apache.org/jira/browse/SPARK-24863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24863: Assignee: Apache Spark > Report offset lag as a custom metrics for Kafka structured

[jira] [Assigned] (SPARK-24863) Report offset lag as a custom metrics for Kafka structured streaming source

[ https://issues.apache.org/jira/browse/SPARK-24863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24863: Assignee: (was: Apache Spark) > Report offset lag as a custom metrics for Kafka

[jira] [Commented] (SPARK-24863) Report offset lag as a custom metrics for Kafka structured streaming source

[ https://issues.apache.org/jira/browse/SPARK-24863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549814#comment-16549814 ] Apache Spark commented on SPARK-24863: -- User 'arunmahadevan' has created a pull request for this

[jira] [Created] (SPARK-24863) Report offset lag as a custom metrics for Kafka structured streaming source

Arun Mahadevan created SPARK-24863: -- Summary: Report offset lag as a custom metrics for Kafka structured streaming source Key: SPARK-24863 URL: https://issues.apache.org/jira/browse/SPARK-24863

[jira] [Resolved] (SPARK-22187) Update unsaferow format for saved state such that we can set timeouts when state is null

[ https://issues.apache.org/jira/browse/SPARK-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-22187. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21739

[jira] [Created] (SPARK-24862) Spark Encoder is not consistent to scala case class semantic for multiple argument lists

Antonio Murgia created SPARK-24862: -- Summary: Spark Encoder is not consistent to scala case class semantic for multiple argument lists Key: SPARK-24862 URL: https://issues.apache.org/jira/browse/SPARK-24862

[jira] [Commented] (SPARK-24801) Empty byte[] arrays in spark.network.sasl.SaslEncryption$EncryptedMessage can waste a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-24801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549745#comment-16549745 ] Imran Rashid commented on SPARK-24801: -- [~felixcheung] [~mshen] > Empty byte[] arrays in

[jira] [Commented] (SPARK-15689) Data source API v2

[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549724#comment-16549724 ] Reynold Xin commented on SPARK-15689: - Is there an umbrella ticket on improving the dsv2 api? If

[jira] [Commented] (SPARK-24192) Invalid Spark URL in local spark session since upgrading from org.apache.spark:spark-sql_2.11:2.2.1 to org.apache.spark:spark-sql_2.11:2.3.0

[ https://issues.apache.org/jira/browse/SPARK-24192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549707#comment-16549707 ] Prabhu Joseph commented on SPARK-24192: --- We have faced this issue and this happens when the

[jira] [Commented] (SPARK-23908) High-order function: transform(array, function) → array

[ https://issues.apache.org/jira/browse/SPARK-23908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549628#comment-16549628 ] Frederick Reiss commented on SPARK-23908: - Thanks Herman, looking forward to seeing this

[jira] [Updated] (SPARK-24374) SPIP: Support Barrier Execution Mode in Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-24374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24374: -- Summary: SPIP: Support Barrier Execution Mode in Apache Spark (was: SPIP: Support Barrier

[jira] [Assigned] (SPARK-24860) Expose dynamic partition overwrite per write operation

[ https://issues.apache.org/jira/browse/SPARK-24860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24860: Assignee: Apache Spark > Expose dynamic partition overwrite per write operation >

[jira] [Assigned] (SPARK-24860) Expose dynamic partition overwrite per write operation

[ https://issues.apache.org/jira/browse/SPARK-24860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24860: Assignee: (was: Apache Spark) > Expose dynamic partition overwrite per write

[jira] [Commented] (SPARK-24860) Expose dynamic partition overwrite per write operation

[ https://issues.apache.org/jira/browse/SPARK-24860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549620#comment-16549620 ] Apache Spark commented on SPARK-24860: -- User 'koertkuipers' has created a pull request for this

[jira] [Commented] (SPARK-24861) create corrected temp directories in RateSourceSuite

[ https://issues.apache.org/jira/browse/SPARK-24861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549540#comment-16549540 ] Apache Spark commented on SPARK-24861: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24861) create corrected temp directories in RateSourceSuite

[ https://issues.apache.org/jira/browse/SPARK-24861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24861: Assignee: Apache Spark (was: Wenchen Fan) > create corrected temp directories in

[jira] [Assigned] (SPARK-24861) create corrected temp directories in RateSourceSuite

[ https://issues.apache.org/jira/browse/SPARK-24861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24861: Assignee: Wenchen Fan (was: Apache Spark) > create corrected temp directories in

[jira] [Commented] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

[ https://issues.apache.org/jira/browse/SPARK-24838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549504#comment-16549504 ] Qifan Pu commented on SPARK-24838: -- Thanks for the PR [~maurits]! Should we also fix it for Aggregate

[jira] [Created] (SPARK-24861) create corrected temp directories in RateSourceSuite

Wenchen Fan created SPARK-24861: --- Summary: create corrected temp directories in RateSourceSuite Key: SPARK-24861 URL: https://issues.apache.org/jira/browse/SPARK-24861 Project: Spark Issue

[jira] [Assigned] (SPARK-24424) Support ANSI-SQL compliant syntax for ROLLUP, CUBE and GROUPING SET

[ https://issues.apache.org/jira/browse/SPARK-24424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24424: Assignee: Apache Spark > Support ANSI-SQL compliant syntax for ROLLUP, CUBE and GROUPING

[jira] [Assigned] (SPARK-24424) Support ANSI-SQL compliant syntax for ROLLUP, CUBE and GROUPING SET

[ https://issues.apache.org/jira/browse/SPARK-24424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24424: Assignee: (was: Apache Spark) > Support ANSI-SQL compliant syntax for ROLLUP, CUBE

[jira] [Commented] (SPARK-24424) Support ANSI-SQL compliant syntax for ROLLUP, CUBE and GROUPING SET

[ https://issues.apache.org/jira/browse/SPARK-24424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549478#comment-16549478 ] Apache Spark commented on SPARK-24424: -- User 'dilipbiswal' has created a pull request for this

[jira] [Assigned] (SPARK-24755) Executor loss can cause task to not be resubmitted

[ https://issues.apache.org/jira/browse/SPARK-24755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-24755: - Assignee: Hieu Tri Huynh > Executor loss can cause task to not be resubmitted >

[jira] [Resolved] (SPARK-24755) Executor loss can cause task to not be resubmitted

[ https://issues.apache.org/jira/browse/SPARK-24755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-24755. --- Resolution: Fixed Fix Version/s: 2.3.3 2.4.0 > Executor loss can

[jira] [Commented] (SPARK-22151) PYTHONPATH not picked up from the spark.yarn.appMasterEnv properly

[ https://issues.apache.org/jira/browse/SPARK-22151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549356#comment-16549356 ] Thomas Graves commented on SPARK-22151: --- ok thanks, must have missed that. > PYTHONPATH not

[jira] [Created] (SPARK-24860) Expose dynamic partition overwrite per write operation

koert kuipers created SPARK-24860: - Summary: Expose dynamic partition overwrite per write operation Key: SPARK-24860 URL: https://issues.apache.org/jira/browse/SPARK-24860 Project: Spark

[jira] [Commented] (SPARK-22151) PYTHONPATH not picked up from the spark.yarn.appMasterEnv properly

[ https://issues.apache.org/jira/browse/SPARK-22151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549351#comment-16549351 ] Sean Owen commented on SPARK-22151: --- It wasn't marked as Resolved, but had a Fix version. I think you

[jira] [Commented] (SPARK-22151) PYTHONPATH not picked up from the spark.yarn.appMasterEnv properly

[ https://issues.apache.org/jira/browse/SPARK-22151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549348#comment-16549348 ] Thomas Graves commented on SPARK-22151: --- [~srowen] why did you blank out fixed version and

[jira] [Resolved] (SPARK-22151) PYTHONPATH not picked up from the spark.yarn.appMasterEnv properly

[ https://issues.apache.org/jira/browse/SPARK-22151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-22151. --- Resolution: Fixed Fix Version/s: 2.4.0 > PYTHONPATH not picked up from the

[jira] [Comment Edited] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549345#comment-16549345 ] Thomas Graves edited comment on SPARK-24615 at 7/19/18 2:28 PM: but my

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549345#comment-16549345 ] Thomas Graves commented on SPARK-24615: --- but my point is exactly that, it shouldn't be yet another

[jira] [Assigned] (SPARK-24858) Avoid unnecessary parquet footer reads

[ https://issues.apache.org/jira/browse/SPARK-24858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24858: Assignee: Gengliang Wang > Avoid unnecessary parquet footer reads >

[jira] [Resolved] (SPARK-24858) Avoid unnecessary parquet footer reads

[ https://issues.apache.org/jira/browse/SPARK-24858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24858. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21814

[jira] [Commented] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

[ https://issues.apache.org/jira/browse/SPARK-24838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549293#comment-16549293 ] Maurits van der Goes commented on SPARK-24838: -- Worked with [~hvanhovell] on a fix. This is

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549276#comment-16549276 ] Saisai Shao commented on SPARK-24615: - Thanks [~tgraves] for the suggestion.  {quote}Once I get to

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549266#comment-16549266 ] Thomas Graves commented on SPARK-24615: --- I think the usage for cpu/memory is the same.  You know

[jira] [Commented] (SPARK-12126) JDBC datasource processes filters only commonly pushed down.

[ https://issues.apache.org/jira/browse/SPARK-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549220#comment-16549220 ] Kyle Prifogle commented on SPARK-12126: --- Thanks [~hyukjin.kwon], I'll attach the relevant story

[jira] [Commented] (SPARK-16854) mapWithState Support for Python

[ https://issues.apache.org/jira/browse/SPARK-16854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549209#comment-16549209 ] Joost Verdoorn commented on SPARK-16854: mapWithState would be extremely helpful within python.

  1   2   >