[jira] [Updated] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24723: -- Description: In barrier mode, to run hybrid distributed DL training jobs, we need to provide

[jira] [Commented] (SPARK-24724) Discuss necessary info and access in barrier mode + Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-24724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550049#comment-16550049 ] Xiangrui Meng commented on SPARK-24724: --- [~liyinan926] Any updates? > Discuss necessary info and

[jira] [Commented] (SPARK-24870) Cache can't work normally if there are case letters in SQL

[ https://issues.apache.org/jira/browse/SPARK-24870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550131#comment-16550131 ] Apache Spark commented on SPARK-24870: -- User 'eatoncys' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24870) Cache can't work normally if there are case letters in SQL

[ https://issues.apache.org/jira/browse/SPARK-24870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24870: Assignee: Apache Spark > Cache can't work normally if there are case letters in SQL >

[jira] [Assigned] (SPARK-24870) Cache can't work normally if there are case letters in SQL

[ https://issues.apache.org/jira/browse/SPARK-24870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24870: Assignee: (was: Apache Spark) > Cache can't work normally if there are case letters

[jira] [Assigned] (SPARK-24195) sc.addFile for local:/ path is broken

[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24195: --- Assignee: Li Yuanjian > sc.addFile for local:/ path is broken >

[jira] [Created] (SPARK-24868) add sequence function in Python

Huaxin Gao created SPARK-24868: -- Summary: add sequence function in Python Key: SPARK-24868 URL: https://issues.apache.org/jira/browse/SPARK-24868 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550023#comment-16550023 ] Xiangrui Meng commented on SPARK-24615: --- [~tgraves] Could you help link some past requests on

[jira] [Commented] (SPARK-24523) InterruptedException when closing SparkContext

[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550060#comment-16550060 ] Dongjoon Hyun commented on SPARK-24523: --- Hi, [~umayr_nuna]. Do you still see the same situation

[jira] [Commented] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

[ https://issues.apache.org/jira/browse/SPARK-24869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550020#comment-16550020 ] Xiao Li commented on SPARK-24869: - cc [~maropu] Do you want to make a try? >

[jira] [Commented] (SPARK-24847) ScalaReflection#schemaFor occasionally fails to detect schema for Seq of type alias

[ https://issues.apache.org/jira/browse/SPARK-24847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550091#comment-16550091 ] Liang-Chi Hsieh commented on SPARK-24847: - I can't reproduce this currently. >

[jira] [Comment Edited] (SPARK-24859) Predicates pushdown on outer joins

[ https://issues.apache.org/jira/browse/SPARK-24859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550159#comment-16550159 ] Hyukjin Kwon edited comment on SPARK-24859 at 7/20/18 3:04 AM: --- Does the

[jira] [Commented] (SPARK-24859) Predicates pushdown on outer joins

[ https://issues.apache.org/jira/browse/SPARK-24859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550159#comment-16550159 ] Hyukjin Kwon commented on SPARK-24859: -- Does the same thing happens in Apache Spark's master

[jira] [Updated] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24307: Fix Version/s: 2.4.0 > Support sending messages over 2GB from memory >

[jira] [Resolved] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24307. - Resolution: Fixed Assignee: Imran Rashid > Support sending messages over 2GB from memory

[jira] [Commented] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550167#comment-16550167 ] Saisai Shao commented on SPARK-24307: - Issue resolved by pull request 21440

[jira] [Created] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

Xiao Li created SPARK-24869: --- Summary: SaveIntoDataSourceCommand's input Dataset does not use Cached Data Key: SPARK-24869 URL: https://issues.apache.org/jira/browse/SPARK-24869 Project: Spark

[jira] [Assigned] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24867: Assignee: Xiao Li (was: Apache Spark) > Add AnalysisBarrier to DataFrameWriter >

[jira] [Commented] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550034#comment-16550034 ] Apache Spark commented on SPARK-24867: -- User 'gatorsmile' has created a pull request for this

[jira] [Assigned] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24867: Assignee: Apache Spark (was: Xiao Li) > Add AnalysisBarrier to DataFrameWriter >

[jira] [Resolved] (SPARK-24726) Discuss necessary info and access in barrier mode + Standalone

[ https://issues.apache.org/jira/browse/SPARK-24726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-24726. --- Resolution: Resolved Target Version/s: 2.4.0 (was: 3.0.0) > Discuss necessary

[jira] [Commented] (SPARK-24726) Discuss necessary info and access in barrier mode + Standalone

[ https://issues.apache.org/jira/browse/SPARK-24726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550050#comment-16550050 ] Xiangrui Meng commented on SPARK-24726: --- I'm closing this ticket as resolved since with

[jira] [Assigned] (SPARK-24726) Discuss necessary info and access in barrier mode + Standalone

[ https://issues.apache.org/jira/browse/SPARK-24726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-24726: - Assignee: Xiangrui Meng > Discuss necessary info and access in barrier mode +

[jira] [Commented] (SPARK-24859) Predicates pushdown on outer joins

[ https://issues.apache.org/jira/browse/SPARK-24859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550158#comment-16550158 ] Hyukjin Kwon commented on SPARK-24859: -- Please avoid set {{Criticial}}+ which is usually reserved

[jira] [Updated] (SPARK-24859) Predicates pushdown on outer joins

[ https://issues.apache.org/jira/browse/SPARK-24859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24859: - Priority: Major (was: Critical) > Predicates pushdown on outer joins >

[jira] [Resolved] (SPARK-24856) spark need upgrade Guava for use gRPC

[ https://issues.apache.org/jira/browse/SPARK-24856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24856. -- Resolution: Duplicate Please search JIRAs before filling it. > spark need upgrade Guava for

[jira] [Updated] (SPARK-24037) stateful operators

[ https://issues.apache.org/jira/browse/SPARK-24037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24037: Fix Version/s: (was: 2.4.0) > stateful operators > -- > >

[jira] [Updated] (SPARK-24037) stateful operators

[ https://issues.apache.org/jira/browse/SPARK-24037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24037: Fix Version/s: 2.4.0 > stateful operators > -- > > Key:

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550182#comment-16550182 ] Saisai Shao commented on SPARK-24723: - Hi [~mengxr], I don't think YARN has such feature to 

[jira] [Assigned] (SPARK-24871) Refactor Concat and MapConcat to avoid creating concatenator object for each row.

[ https://issues.apache.org/jira/browse/SPARK-24871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24871: Assignee: (was: Apache Spark) > Refactor Concat and MapConcat to avoid creating

[jira] [Commented] (SPARK-24871) Refactor Concat and MapConcat to avoid creating concatenator object for each row.

[ https://issues.apache.org/jira/browse/SPARK-24871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550218#comment-16550218 ] Apache Spark commented on SPARK-24871: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24871) Refactor Concat and MapConcat to avoid creating concatenator object for each row.

[ https://issues.apache.org/jira/browse/SPARK-24871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24871: Assignee: Apache Spark > Refactor Concat and MapConcat to avoid creating concatenator

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550266#comment-16550266 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] I'm still not sure how to handle memory per

[jira] [Commented] (SPARK-24865) Remove AnalysisBarrier

[ https://issues.apache.org/jira/browse/SPARK-24865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550047#comment-16550047 ] Apache Spark commented on SPARK-24865: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-24870) Cache can't work normally if there are case letters in SQL

eaton created SPARK-24870: - Summary: Cache can't work normally if there are case letters in SQL Key: SPARK-24870 URL: https://issues.apache.org/jira/browse/SPARK-24870 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-24868) add sequence function in Python

[ https://issues.apache.org/jira/browse/SPARK-24868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24868: Assignee: Apache Spark > add sequence function in Python >

[jira] [Assigned] (SPARK-24868) add sequence function in Python

[ https://issues.apache.org/jira/browse/SPARK-24868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24868: Assignee: (was: Apache Spark) > add sequence function in Python >

[jira] [Commented] (SPARK-24868) add sequence function in Python

[ https://issues.apache.org/jira/browse/SPARK-24868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550014#comment-16550014 ] Apache Spark commented on SPARK-24868: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-24857) required the sample code test the spark steaming job in kubernates and write the data in remote hdfs file system

[ https://issues.apache.org/jira/browse/SPARK-24857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550153#comment-16550153 ] Hyukjin Kwon commented on SPARK-24857: -- [~kmkrishna1...@gmail.com], mind clarifying the input

[jira] [Resolved] (SPARK-24195) sc.addFile for local:/ path is broken

[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24195. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21533

[jira] [Updated] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24867: Priority: Blocker (was: Major) > Add AnalysisBarrier to DataFrameWriter >

[jira] [Commented] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

[ https://issues.apache.org/jira/browse/SPARK-24869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550046#comment-16550046 ] Xiao Li commented on SPARK-24869: - Thanks! [~maropu] > SaveIntoDataSourceCommand's input Dataset does

[jira] [Assigned] (SPARK-24865) Remove AnalysisBarrier

[ https://issues.apache.org/jira/browse/SPARK-24865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24865: Assignee: (was: Apache Spark) > Remove AnalysisBarrier > -- > >

[jira] [Assigned] (SPARK-24865) Remove AnalysisBarrier

[ https://issues.apache.org/jira/browse/SPARK-24865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24865: Assignee: Apache Spark > Remove AnalysisBarrier > -- > >

[jira] [Resolved] (SPARK-24861) create corrected temp directories in RateSourceSuite

[ https://issues.apache.org/jira/browse/SPARK-24861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24861. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21817

[jira] [Commented] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

[ https://issues.apache.org/jira/browse/SPARK-24869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550029#comment-16550029 ] Takeshi Yamamuro commented on SPARK-24869: -- ok > SaveIntoDataSourceCommand's input Dataset

[jira] [Assigned] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-24723: - Assignee: Saisai Shao > Discuss necessary info and access in barrier mode + YARN >

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550048#comment-16550048 ] Xiangrui Meng commented on SPARK-24723: --- [~jerryshao] Does YARN have the feature that will by

[jira] [Created] (SPARK-24871) Refactor Concat and MapConcat to avoid creating concatenator object for each row.

Takuya Ueshin created SPARK-24871: - Summary: Refactor Concat and MapConcat to avoid creating concatenator object for each row. Key: SPARK-24871 URL: https://issues.apache.org/jira/browse/SPARK-24871

[jira] [Commented] (SPARK-6459) Warn when Column API is constructing trivially true equality

[ https://issues.apache.org/jira/browse/SPARK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550024#comment-16550024 ] nirav patel commented on SPARK-6459: [~zero323] why the example you gave should generate cartesian

[jira] [Created] (SPARK-24866) Artifactual ROC scores when scaling up Random Forest classifier

Evan Zamir created SPARK-24866: -- Summary: Artifactual ROC scores when scaling up Random Forest classifier Key: SPARK-24866 URL: https://issues.apache.org/jira/browse/SPARK-24866 Project: Spark

[jira] [Updated] (SPARK-24866) Artifactual ROC scores when scaling up Random Forest classifier

[ https://issues.apache.org/jira/browse/SPARK-24866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evan Zamir updated SPARK-24866: --- Description: I'm encountering a very strange behavior that I can't explain away other than a bug

[jira] [Created] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

Xiao Li created SPARK-24867: --- Summary: Add AnalysisBarrier to DataFrameWriter Key: SPARK-24867 URL: https://issues.apache.org/jira/browse/SPARK-24867 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-15428) Disable support for multiple streaming aggregations

[ https://issues.apache.org/jira/browse/SPARK-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549174#comment-16549174 ] Joost Verdoorn edited comment on SPARK-15428 at 7/19/18 11:56 AM: -- I

[jira] [Commented] (SPARK-16854) mapWithState Support for Python

[ https://issues.apache.org/jira/browse/SPARK-16854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549209#comment-16549209 ] Joost Verdoorn commented on SPARK-16854: mapWithState would be extremely helpful within python.

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549266#comment-16549266 ] Thomas Graves commented on SPARK-24615: --- I think the usage for cpu/memory is the same.  You know

[jira] [Commented] (SPARK-24849) Convert StructType to DDL string

[ https://issues.apache.org/jira/browse/SPARK-24849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548878#comment-16548878 ] Maxim Gekk commented on SPARK-24849: [~maropu] This is a part of my work on customer's issue. There

[jira] [Commented] (SPARK-15428) Disable support for multiple streaming aggregations

[ https://issues.apache.org/jira/browse/SPARK-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549174#comment-16549174 ] Joost Verdoorn commented on SPARK-15428: I was wondering the same. Being able to do only one

[jira] [Resolved] (SPARK-11784) Support Timestamp filter pushdown in Parquet datasource

[ https://issues.apache.org/jira/browse/SPARK-11784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-11784. - Resolution: Duplicate > Support Timestamp filter pushdown in Parquet datasource >

[jira] [Commented] (SPARK-12126) JDBC datasource processes filters only commonly pushed down.

[ https://issues.apache.org/jira/browse/SPARK-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549220#comment-16549220 ] Kyle Prifogle commented on SPARK-12126: --- Thanks [~hyukjin.kwon], I'll attach the relevant story

[jira] [Resolved] (SPARK-24717) Split out min retain version of state for memory in HDFSBackedStateStoreProvider

[ https://issues.apache.org/jira/browse/SPARK-24717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-24717. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21700

[jira] [Updated] (SPARK-24859) Predicates pushdown on outer joins

[ https://issues.apache.org/jira/browse/SPARK-24859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johannes Mayer updated SPARK-24859: --- Description: I have two AVRO tables in Hive called FAct and DIm. Both are partitioned by a

[jira] [Created] (SPARK-24859) Predicates pushdown on outer joins

Johannes Mayer created SPARK-24859: -- Summary: Predicates pushdown on outer joins Key: SPARK-24859 URL: https://issues.apache.org/jira/browse/SPARK-24859 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-24858) Avoid unnecessary parquet footer reads

[ https://issues.apache.org/jira/browse/SPARK-24858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548880#comment-16548880 ] Apache Spark commented on SPARK-24858: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-24858) Avoid unnecessary parquet footer reads

[ https://issues.apache.org/jira/browse/SPARK-24858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24858: Assignee: (was: Apache Spark) > Avoid unnecessary parquet footer reads >

[jira] [Assigned] (SPARK-24858) Avoid unnecessary parquet footer reads

[ https://issues.apache.org/jira/browse/SPARK-24858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24858: Assignee: Apache Spark > Avoid unnecessary parquet footer reads >

[jira] [Assigned] (SPARK-24717) Split out min retain version of state for memory in HDFSBackedStateStoreProvider

[ https://issues.apache.org/jira/browse/SPARK-24717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-24717: - Assignee: Jungtaek Lim > Split out min retain version of state for memory in >

[jira] [Commented] (SPARK-23731) FileSourceScanExec throws NullPointerException in subexpression elimination

[ https://issues.apache.org/jira/browse/SPARK-23731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548938#comment-16548938 ] Apache Spark commented on SPARK-23731: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-24794) DriverWrapper should have both master addresses in -Dspark.master

[ https://issues.apache.org/jira/browse/SPARK-24794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24794: Assignee: (was: Apache Spark) > DriverWrapper should have both master addresses in

[jira] [Assigned] (SPARK-24794) DriverWrapper should have both master addresses in -Dspark.master

[ https://issues.apache.org/jira/browse/SPARK-24794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24794: Assignee: Apache Spark > DriverWrapper should have both master addresses in

[jira] [Commented] (SPARK-24794) DriverWrapper should have both master addresses in -Dspark.master

[ https://issues.apache.org/jira/browse/SPARK-24794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549019#comment-16549019 ] Apache Spark commented on SPARK-24794: -- User 'bsikander' has created a pull request for this issue:

[jira] [Created] (SPARK-24857) required the sample code test the spark steaming job in kubernates and write the data in remote hdfs file system

kumpatla murali krishna created SPARK-24857: --- Summary: required the sample code test the spark steaming job in kubernates and write the data in remote hdfs file system Key: SPARK-24857 URL:

[jira] [Closed] (SPARK-23984) PySpark Bindings for K8S

[ https://issues.apache.org/jira/browse/SPARK-23984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilan Filonenko closed SPARK-23984. -- > PySpark Bindings for K8S > > > Key: SPARK-23984 >

[jira] [Commented] (SPARK-23997) Configurable max number of buckets

[ https://issues.apache.org/jira/browse/SPARK-23997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548974#comment-16548974 ] Matthias Wolf commented on SPARK-23997: --- Is there any issue with having this limit configurable?

[jira] [Created] (SPARK-24858) Avoid unnecessary parquet footer reads

Gengliang Wang created SPARK-24858: -- Summary: Avoid unnecessary parquet footer reads Key: SPARK-24858 URL: https://issues.apache.org/jira/browse/SPARK-24858 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24701) SparkMaster WebUI allow all appids to be shown in detail on port 4040 rather than different ports per app

[ https://issues.apache.org/jira/browse/SPARK-24701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548920#comment-16548920 ] t oo commented on SPARK-24701: -- [~guoxiaolongzte] - see attached screenshot > SparkMaster WebUI allow all

[jira] [Updated] (SPARK-24701) SparkMaster WebUI allow all appids to be shown in detail on port 4040 rather than different ports per app

[ https://issues.apache.org/jira/browse/SPARK-24701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24701: - Attachment: spark_ports.png > SparkMaster WebUI allow all appids to be shown in detail on port 4040 rather >

[jira] [Assigned] (SPARK-24860) Expose dynamic partition overwrite per write operation

[ https://issues.apache.org/jira/browse/SPARK-24860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24860: Assignee: Apache Spark > Expose dynamic partition overwrite per write operation >

[jira] [Assigned] (SPARK-24860) Expose dynamic partition overwrite per write operation

[ https://issues.apache.org/jira/browse/SPARK-24860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24860: Assignee: (was: Apache Spark) > Expose dynamic partition overwrite per write

[jira] [Commented] (SPARK-24860) Expose dynamic partition overwrite per write operation

[ https://issues.apache.org/jira/browse/SPARK-24860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549620#comment-16549620 ] Apache Spark commented on SPARK-24860: -- User 'koertkuipers' has created a pull request for this

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549276#comment-16549276 ] Saisai Shao commented on SPARK-24615: - Thanks [~tgraves] for the suggestion.  {quote}Once I get to

[jira] [Resolved] (SPARK-22151) PYTHONPATH not picked up from the spark.yarn.appMasterEnv properly

[ https://issues.apache.org/jira/browse/SPARK-22151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-22151. --- Resolution: Fixed Fix Version/s: 2.4.0 > PYTHONPATH not picked up from the

[jira] [Created] (SPARK-24860) Expose dynamic partition overwrite per write operation

koert kuipers created SPARK-24860: - Summary: Expose dynamic partition overwrite per write operation Key: SPARK-24860 URL: https://issues.apache.org/jira/browse/SPARK-24860 Project: Spark

[jira] [Resolved] (SPARK-24755) Executor loss can cause task to not be resubmitted

[ https://issues.apache.org/jira/browse/SPARK-24755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-24755. --- Resolution: Fixed Fix Version/s: 2.3.3 2.4.0 > Executor loss can

[jira] [Assigned] (SPARK-24755) Executor loss can cause task to not be resubmitted

[ https://issues.apache.org/jira/browse/SPARK-24755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-24755: - Assignee: Hieu Tri Huynh > Executor loss can cause task to not be resubmitted >

[jira] [Created] (SPARK-24861) create corrected temp directories in RateSourceSuite

Wenchen Fan created SPARK-24861: --- Summary: create corrected temp directories in RateSourceSuite Key: SPARK-24861 URL: https://issues.apache.org/jira/browse/SPARK-24861 Project: Spark Issue

[jira] [Commented] (SPARK-23908) High-order function: transform(array, function) → array

[ https://issues.apache.org/jira/browse/SPARK-23908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549628#comment-16549628 ] Frederick Reiss commented on SPARK-23908: - Thanks Herman, looking forward to seeing this

[jira] [Assigned] (SPARK-24861) create corrected temp directories in RateSourceSuite

[ https://issues.apache.org/jira/browse/SPARK-24861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24861: Assignee: Wenchen Fan (was: Apache Spark) > create corrected temp directories in

[jira] [Commented] (SPARK-24861) create corrected temp directories in RateSourceSuite

[ https://issues.apache.org/jira/browse/SPARK-24861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549540#comment-16549540 ] Apache Spark commented on SPARK-24861: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24861) create corrected temp directories in RateSourceSuite

[ https://issues.apache.org/jira/browse/SPARK-24861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24861: Assignee: Apache Spark (was: Wenchen Fan) > create corrected temp directories in

[jira] [Updated] (SPARK-24374) SPIP: Support Barrier Execution Mode in Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-24374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24374: -- Summary: SPIP: Support Barrier Execution Mode in Apache Spark (was: SPIP: Support Barrier

[jira] [Commented] (SPARK-22151) PYTHONPATH not picked up from the spark.yarn.appMasterEnv properly

[ https://issues.apache.org/jira/browse/SPARK-22151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549356#comment-16549356 ] Thomas Graves commented on SPARK-22151: --- ok thanks, must have missed that. > PYTHONPATH not

[jira] [Commented] (SPARK-24424) Support ANSI-SQL compliant syntax for ROLLUP, CUBE and GROUPING SET

[ https://issues.apache.org/jira/browse/SPARK-24424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549478#comment-16549478 ] Apache Spark commented on SPARK-24424: -- User 'dilipbiswal' has created a pull request for this

[jira] [Assigned] (SPARK-24424) Support ANSI-SQL compliant syntax for ROLLUP, CUBE and GROUPING SET

[ https://issues.apache.org/jira/browse/SPARK-24424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24424: Assignee: Apache Spark > Support ANSI-SQL compliant syntax for ROLLUP, CUBE and GROUPING

[jira] [Assigned] (SPARK-24424) Support ANSI-SQL compliant syntax for ROLLUP, CUBE and GROUPING SET

[ https://issues.apache.org/jira/browse/SPARK-24424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24424: Assignee: (was: Apache Spark) > Support ANSI-SQL compliant syntax for ROLLUP, CUBE

[jira] [Commented] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

[ https://issues.apache.org/jira/browse/SPARK-24838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549293#comment-16549293 ] Maurits van der Goes commented on SPARK-24838: -- Worked with [~hvanhovell] on a fix. This is

[jira] [Resolved] (SPARK-24858) Avoid unnecessary parquet footer reads

[ https://issues.apache.org/jira/browse/SPARK-24858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24858. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21814

[jira] [Assigned] (SPARK-24858) Avoid unnecessary parquet footer reads

[ https://issues.apache.org/jira/browse/SPARK-24858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24858: Assignee: Gengliang Wang > Avoid unnecessary parquet footer reads >

[jira] [Comment Edited] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549345#comment-16549345 ] Thomas Graves edited comment on SPARK-24615 at 7/19/18 2:28 PM: but my

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549345#comment-16549345 ] Thomas Graves commented on SPARK-24615: --- but my point is exactly that, it shouldn't be yet another

  1   2   >