[jira] [Assigned] (SPARK-24076) very bad performance when shuffle.partition = 8192

[ https://issues.apache.org/jira/browse/SPARK-24076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24076: Assignee: Apache Spark > very bad performance when shuffle.partition = 8192 >

[jira] [Assigned] (SPARK-24076) very bad performance when shuffle.partition = 8192

[ https://issues.apache.org/jira/browse/SPARK-24076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24076: Assignee: (was: Apache Spark) > very bad performance when shuffle.partition = 8192 > -

[jira] [Commented] (SPARK-24076) very bad performance when shuffle.partition = 8192

[ https://issues.apache.org/jira/browse/SPARK-24076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451753#comment-16451753 ] Apache Spark commented on SPARK-24076: -- User 'yucai' has created a pull request for

[jira] [Commented] (SPARK-24075) [Mesos] Supervised driver upon failure will be retried indefinitely unless explicitly killed

[ https://issues.apache.org/jira/browse/SPARK-24075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451776#comment-16451776 ] Apache Spark commented on SPARK-24075: -- User 'nyogesh' has created a pull request fo

[jira] [Assigned] (SPARK-24075) [Mesos] Supervised driver upon failure will be retried indefinitely unless explicitly killed

[ https://issues.apache.org/jira/browse/SPARK-24075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24075: Assignee: (was: Apache Spark) > [Mesos] Supervised driver upon failure will be retried

[jira] [Assigned] (SPARK-24075) [Mesos] Supervised driver upon failure will be retried indefinitely unless explicitly killed

[ https://issues.apache.org/jira/browse/SPARK-24075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24075: Assignee: Apache Spark > [Mesos] Supervised driver upon failure will be retried indefinite

[jira] [Created] (SPARK-24083) Diagnostics message for uncaught exceptions should include the stacktrace

zhoukang created SPARK-24083: Summary: Diagnostics message for uncaught exceptions should include the stacktrace Key: SPARK-24083 URL: https://issues.apache.org/jira/browse/SPARK-24083 Project: Spark

[jira] [Assigned] (SPARK-24083) Diagnostics message for uncaught exceptions should include the stacktrace

[ https://issues.apache.org/jira/browse/SPARK-24083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24083: Assignee: (was: Apache Spark) > Diagnostics message for uncaught exceptions should inc

[jira] [Assigned] (SPARK-24083) Diagnostics message for uncaught exceptions should include the stacktrace

[ https://issues.apache.org/jira/browse/SPARK-24083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24083: Assignee: Apache Spark > Diagnostics message for uncaught exceptions should include the st

[jira] [Commented] (SPARK-24083) Diagnostics message for uncaught exceptions should include the stacktrace

[ https://issues.apache.org/jira/browse/SPARK-24083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451820#comment-16451820 ] Apache Spark commented on SPARK-24083: -- User 'caneGuy' has created a pull request fo

[jira] [Commented] (SPARK-24081) Spark SQL drops the table while writing into table in "overwrite" mode.

[ https://issues.apache.org/jira/browse/SPARK-24081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451913#comment-16451913 ] Hyukjin Kwon commented on SPARK-24081: -- Please avoid to set a block which is usually

[jira] [Updated] (SPARK-24081) Spark SQL drops the table while writing into table in "overwrite" mode.

[ https://issues.apache.org/jira/browse/SPARK-24081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24081: - Priority: Major (was: Blocker) > Spark SQL drops the table while writing into table in "overwri

[jira] [Comment Edited] (SPARK-24081) Spark SQL drops the table while writing into table in "overwrite" mode.

[ https://issues.apache.org/jira/browse/SPARK-24081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451913#comment-16451913 ] Hyukjin Kwon edited comment on SPARK-24081 at 4/25/18 8:55 AM:

[jira] [Assigned] (SPARK-23688) Refactor tests away from rate source

[ https://issues.apache.org/jira/browse/SPARK-23688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23688: Assignee: (was: Apache Spark) > Refactor tests away from rate source > ---

[jira] [Commented] (SPARK-23688) Refactor tests away from rate source

[ https://issues.apache.org/jira/browse/SPARK-23688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451991#comment-16451991 ] Apache Spark commented on SPARK-23688: -- User 'HeartSaVioR' has created a pull reques

[jira] [Assigned] (SPARK-23688) Refactor tests away from rate source

[ https://issues.apache.org/jira/browse/SPARK-23688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23688: Assignee: Apache Spark > Refactor tests away from rate source > --

[jira] [Commented] (SPARK-24058) Default Params in ML should be saved separately: Python API

[ https://issues.apache.org/jira/browse/SPARK-24058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452000#comment-16452000 ] Apache Spark commented on SPARK-24058: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-24012) Union of map and other compatible column

[ https://issues.apache.org/jira/browse/SPARK-24012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24012: --- Assignee: Lijia Liu > Union of map and other compatible column > ---

[jira] [Assigned] (SPARK-24058) Default Params in ML should be saved separately: Python API

[ https://issues.apache.org/jira/browse/SPARK-24058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24058: Assignee: (was: Apache Spark) > Default Params in ML should be saved separately: Pytho

[jira] [Assigned] (SPARK-24058) Default Params in ML should be saved separately: Python API

[ https://issues.apache.org/jira/browse/SPARK-24058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24058: Assignee: Apache Spark > Default Params in ML should be saved separately: Python API > ---

[jira] [Resolved] (SPARK-24012) Union of map and other compatible column

[ https://issues.apache.org/jira/browse/SPARK-24012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24012. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21100 [https://githu

[jira] [Commented] (SPARK-20894) Error while checkpointing to HDFS

[ https://issues.apache.org/jira/browse/SPARK-20894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452030#comment-16452030 ] Aydin Kocas commented on SPARK-20894: - having the same issue on 2.3 - what's the solu

[jira] [Commented] (SPARK-20894) Error while checkpointing to HDFS

[ https://issues.apache.org/jira/browse/SPARK-20894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452041#comment-16452041 ] Aydin Kocas commented on SPARK-20894: - removing the checkpoint location along with th

[jira] [Commented] (SPARK-24012) Union of map and other compatible column

[ https://issues.apache.org/jira/browse/SPARK-24012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452059#comment-16452059 ] Apache Spark commented on SPARK-24012: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-23880) table cache should be lazy and don't trigger any jobs.

[ https://issues.apache.org/jira/browse/SPARK-23880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23880: --- Assignee: Takeshi Yamamuro > table cache should be lazy and don't trigger any jobs. > --

[jira] [Resolved] (SPARK-23880) table cache should be lazy and don't trigger any jobs.

[ https://issues.apache.org/jira/browse/SPARK-23880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23880. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21018 [https://githu

[jira] [Commented] (SPARK-24070) TPC-DS Performance Tests for Parquet 1.10.0 Upgrade

[ https://issues.apache.org/jira/browse/SPARK-24070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452096#comment-16452096 ] Michał Świtakowski commented on SPARK-24070: [~maropu] I think you can just u

[jira] [Comment Edited] (SPARK-24070) TPC-DS Performance Tests for Parquet 1.10.0 Upgrade

[ https://issues.apache.org/jira/browse/SPARK-24070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452096#comment-16452096 ] Michał Świtakowski edited comment on SPARK-24070 at 4/25/18 11:43 AM: -

[jira] [Created] (SPARK-24084) Add job group id for query through Thrift Server

zhoukang created SPARK-24084: Summary: Add job group id for query through Thrift Server Key: SPARK-24084 URL: https://issues.apache.org/jira/browse/SPARK-24084 Project: Spark Issue Type: Improvem

[jira] [Updated] (SPARK-24084) Add job group id for query through spark-sql

[ https://issues.apache.org/jira/browse/SPARK-24084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-24084: - Summary: Add job group id for query through spark-sql (was: Add job group id for query through Thrift Se

[jira] [Updated] (SPARK-24084) Add job group id for query through spark-sql

[ https://issues.apache.org/jira/browse/SPARK-24084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-24084: - Description: For spark-sql we can add job group id for the same statement. (was: For thrift server we ca

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452196#comment-16452196 ] Steve Loughran commented on SPARK-18673: HIVE-16081 commit 93db527f47 contains th

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452215#comment-16452215 ] Steve Loughran commented on SPARK-18673: looking @ our local commit logs, the HDP

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: 1tes.zip > SQL which has large ‘case when’ expressions may cause code generation beyond

[jira] [Created] (SPARK-24085) Scalar subquery error

Alexey Baturin created SPARK-24085: -- Summary: Scalar subquery error Key: SPARK-24085 URL: https://issues.apache.org/jira/browse/SPARK-24085 Project: Spark Issue Type: Bug Component

[jira] [Assigned] (SPARK-23927) High-order function: sequence

[ https://issues.apache.org/jira/browse/SPARK-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23927: Assignee: (was: Apache Spark) > High-order function: sequence > --

[jira] [Assigned] (SPARK-23927) High-order function: sequence

[ https://issues.apache.org/jira/browse/SPARK-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23927: Assignee: Apache Spark > High-order function: sequence > - > >

[jira] [Commented] (SPARK-23927) High-order function: sequence

[ https://issues.apache.org/jira/browse/SPARK-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452246#comment-16452246 ] Apache Spark commented on SPARK-23927: -- User 'wajda' has created a pull request for

[jira] [Commented] (SPARK-23929) pandas_udf schema mapped by position and not by name

[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452282#comment-16452282 ] Tr3wory commented on SPARK-23929: - I think the problem is even more nuanced: in python th

[jira] [Created] (SPARK-24086) Exception while executing spark streaming examples

Chandra Hasan created SPARK-24086: - Summary: Exception while executing spark streaming examples Key: SPARK-24086 URL: https://issues.apache.org/jira/browse/SPARK-24086 Project: Spark Issue Ty

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: 1tes.zip) > SQL which has large ‘case when’ expressions may cause code generati

[jira] [Commented] (SPARK-24067) Backport SPARK-17147 to 2.3 (Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets (i.e. Log Compaction))

[ https://issues.apache.org/jira/browse/SPARK-24067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452337#comment-16452337 ] Cody Koeninger commented on SPARK-24067: Given the response on the dev list about

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452350#comment-16452350 ] Alex Wajda commented on SPARK-23933: Why can't {{map}} be overloaded? So that if you

[jira] [Commented] (SPARK-23929) pandas_udf schema mapped by position and not by name

[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452355#comment-16452355 ] Li Jin commented on SPARK-23929: [~tr3w] does using OrderedDict help in your case? > pan

[jira] [Commented] (SPARK-20087) Include accumulators / taskMetrics when sending TaskKilled to onTaskEnd listeners

[ https://issues.apache.org/jira/browse/SPARK-20087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452369#comment-16452369 ] Imran Rashid commented on SPARK-20087: -- Sound good to me, I'm in favor of the change

[jira] [Created] (SPARK-24087) Avoid shuffle when join keys are a super-set of bucket keys

yucai created SPARK-24087: - Summary: Avoid shuffle when join keys are a super-set of bucket keys Key: SPARK-24087 URL: https://issues.apache.org/jira/browse/SPARK-24087 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-24067) Backport SPARK-17147 to 2.3 (Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets (i.e. Log Compaction))

[ https://issues.apache.org/jira/browse/SPARK-24067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452396#comment-16452396 ] Sean Owen commented on SPARK-24067: --- It seems like a clear bug fix. Granted it's not tr

[jira] [Assigned] (SPARK-24087) Avoid shuffle when join keys are a super-set of bucket keys

[ https://issues.apache.org/jira/browse/SPARK-24087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24087: Assignee: Apache Spark > Avoid shuffle when join keys are a super-set of bucket keys > ---

[jira] [Commented] (SPARK-24087) Avoid shuffle when join keys are a super-set of bucket keys

[ https://issues.apache.org/jira/browse/SPARK-24087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452399#comment-16452399 ] Apache Spark commented on SPARK-24087: -- User 'yucai' has created a pull request for

[jira] [Assigned] (SPARK-24087) Avoid shuffle when join keys are a super-set of bucket keys

[ https://issues.apache.org/jira/browse/SPARK-24087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24087: Assignee: (was: Apache Spark) > Avoid shuffle when join keys are a super-set of bucket

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452404#comment-16452404 ] Kazuaki Ishizaki commented on SPARK-23933: -- Thank you for your comment. The curr

[jira] [Commented] (SPARK-23929) pandas_udf schema mapped by position and not by name

[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452459#comment-16452459 ] Tr3wory commented on SPARK-23929: - Yes, but that's not simpler than using "columns=[...]"

[jira] [Commented] (SPARK-22674) PySpark breaks serialization of namedtuple subclasses

[ https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452462#comment-16452462 ] Apache Spark commented on SPARK-22674: -- User 'superbobry' has created a pull request

[jira] [Assigned] (SPARK-22674) PySpark breaks serialization of namedtuple subclasses

[ https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22674: Assignee: (was: Apache Spark) > PySpark breaks serialization of namedtuple subclasses

[jira] [Assigned] (SPARK-22674) PySpark breaks serialization of namedtuple subclasses

[ https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22674: Assignee: Apache Spark > PySpark breaks serialization of namedtuple subclasses > -

[jira] [Created] (SPARK-24088) only HadoopRDD leverage HDFS Cache as preferred location

Xiaoju Wu created SPARK-24088: - Summary: only HadoopRDD leverage HDFS Cache as preferred location Key: SPARK-24088 URL: https://issues.apache.org/jira/browse/SPARK-24088 Project: Spark Issue Type

[jira] [Commented] (SPARK-24070) TPC-DS Performance Tests for Parquet 1.10.0 Upgrade

[ https://issues.apache.org/jira/browse/SPARK-24070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452527#comment-16452527 ] Xiao Li commented on SPARK-24070: - [~mswit] Thank you for your suggestions! This is very

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452576#comment-16452576 ] Jose Torres commented on SPARK-24036: - The broader Spark community is of course alway

[jira] [Commented] (SPARK-24067) Backport SPARK-17147 to 2.3 (Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets (i.e. Log Compaction))

[ https://issues.apache.org/jira/browse/SPARK-24067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452603#comment-16452603 ] Cody Koeninger commented on SPARK-24067: The original PR [https://github.com/apac

[jira] [Created] (SPARK-24089) DataFrame.write.mode(SaveMode.Append).insertInto(TABLE)

kumar created SPARK-24089: - Summary: DataFrame.write.mode(SaveMode.Append).insertInto(TABLE) Key: SPARK-24089 URL: https://issues.apache.org/jira/browse/SPARK-24089 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kumar updated SPARK-24089: -- Summary: DataFrame.write().mode(SaveMode.Append).insertInto(TABLE) (was: DataFrame.write.mode(SaveMode.Append

[jira] [Updated] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kumar updated SPARK-24089: -- Description: I am completely stuck with this issue, unable to progress further. For more info pls refer this p

[jira] [Commented] (SPARK-23874) Upgrade apache/arrow to 0.10.0

[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452666#comment-16452666 ] Bryan Cutler commented on SPARK-23874: -- [~smilegator] the Arrow community decided to

[jira] [Updated] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kumar updated SPARK-24089: -- Component/s: Java API > DataFrame.write().mode(SaveMode.Append).insertInto(TABLE) > --

[jira] [Created] (SPARK-24090) Kubernetes Backend Hotlist for Spark 2.4

Anirudh Ramanathan created SPARK-24090: -- Summary: Kubernetes Backend Hotlist for Spark 2.4 Key: SPARK-24090 URL: https://issues.apache.org/jira/browse/SPARK-24090 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-23850) We should not redact username|user|url from UI by default

[ https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23850: Assignee: (was: Apache Spark) > We should not redact username|user|url from UI by defa

[jira] [Assigned] (SPARK-23850) We should not redact username|user|url from UI by default

[ https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23850: Assignee: Apache Spark > We should not redact username|user|url from UI by default > -

[jira] [Commented] (SPARK-23850) We should not redact username|user|url from UI by default

[ https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452729#comment-16452729 ] Apache Spark commented on SPARK-23850: -- User 'vanzin' has created a pull request for

[jira] [Created] (SPARK-24091) Internally used ConfigMap prevents use of user-specified ConfigMaps carrying Spark configs files

Yinan Li created SPARK-24091: Summary: Internally used ConfigMap prevents use of user-specified ConfigMaps carrying Spark configs files Key: SPARK-24091 URL: https://issues.apache.org/jira/browse/SPARK-24091

[jira] [Updated] (SPARK-24091) Internally used ConfigMap prevents use of user-specified ConfigMaps carrying Spark configs files

[ https://issues.apache.org/jira/browse/SPARK-24091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li updated SPARK-24091: - Affects Version/s: (was: 2.3.0) 2.4.0 > Internally used ConfigMap prevents use

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452790#comment-16452790 ] Bruce Robbins commented on SPARK-23715: --- [~cloud_fan] I'll give separate answers f

[jira] [Comment Edited] (SPARK-23933) High-order function: map(array, array) → map

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452404#comment-16452404 ] Kazuaki Ishizaki edited comment on SPARK-23933 at 4/25/18 6:48 PM:

[jira] [Resolved] (SPARK-24050) StreamingQuery does not calculate input / processing rates in some cases

[ https://issues.apache.org/jira/browse/SPARK-24050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-24050. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21126 [https://g

[jira] [Updated] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24089: Priority: Critical (was: Blocker) > DataFrame.write().mode(SaveMode.Append).insertInto(TABLE) > -

[jira] [Commented] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452937#comment-16452937 ] Marco Gaido commented on SPARK-24089: - Blocker can be set only by commiters, I moved

[jira] [Commented] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452942#comment-16452942 ] Marco Gaido commented on SPARK-24089: - Anyway, for what I can see from your post on s

[jira] [Commented] (SPARK-22239) User-defined window functions with pandas udf

[ https://issues.apache.org/jira/browse/SPARK-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452977#comment-16452977 ] Li Jin commented on SPARK-22239: [~hvanhovell], I have done a bit further research of UDF

[jira] [Created] (SPARK-24092) spark.python.worker.reuse does not work?

David Figueroa created SPARK-24092: -- Summary: spark.python.worker.reuse does not work? Key: SPARK-24092 URL: https://issues.apache.org/jira/browse/SPARK-24092 Project: Spark Issue Type: Ques

[jira] [Updated] (SPARK-24092) spark.python.worker.reuse does not work?

[ https://issues.apache.org/jira/browse/SPARK-24092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Figueroa updated SPARK-24092: --- Description: {{spark.python.worker.reuse is true by default but even after explicitly settin

[jira] [Updated] (SPARK-24092) spark.python.worker.reuse does not work?

[ https://issues.apache.org/jira/browse/SPARK-24092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Figueroa updated SPARK-24092: --- Description: {{spark.python.worker.reuse is true by default but even after explicitly settin

[jira] [Created] (SPARK-24093) Make some fields of KafkaStreamWriter/InternalRowMicroBatchWriter visible to outside of the classes

Weiqing Yang created SPARK-24093: Summary: Make some fields of KafkaStreamWriter/InternalRowMicroBatchWriter visible to outside of the classes Key: SPARK-24093 URL: https://issues.apache.org/jira/browse/SPARK-2409

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453054#comment-16453054 ] Tavis Barr commented on SPARK-13446: I do not believe the issue causing the above sta

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453058#comment-16453058 ] Arun Mahadevan commented on SPARK-24036: Hi [~joseph.torres], I am also intereste

[jira] [Updated] (SPARK-24093) Make some fields of KafkaStreamWriter/InternalRowMicroBatchWriter visible to outside of the classes

[ https://issues.apache.org/jira/browse/SPARK-24093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiqing Yang updated SPARK-24093: - Description: To make third parties able to get the information of streaming writer, for example,

[jira] [Updated] (SPARK-24057) put the real data type in the AssertionError message

[ https://issues.apache.org/jira/browse/SPARK-24057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-24057: --- Issue Type: Improvement (was: Bug) > put the real data type in the AssertionError message >

[jira] [Assigned] (SPARK-24057) put the real data type in the AssertionError message

[ https://issues.apache.org/jira/browse/SPARK-24057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24057: Assignee: (was: Apache Spark) > put the real data type in the AssertionError message >

[jira] [Assigned] (SPARK-24057) put the real data type in the AssertionError message

[ https://issues.apache.org/jira/browse/SPARK-24057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24057: Assignee: Apache Spark > put the real data type in the AssertionError message > --

[jira] [Commented] (SPARK-24057) put the real data type in the AssertionError message

[ https://issues.apache.org/jira/browse/SPARK-24057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453092#comment-16453092 ] Apache Spark commented on SPARK-24057: -- User 'huaxingao' has created a pull request

[jira] [Resolved] (SPARK-23824) Make inpurityStats publicly accessible in ml.tree.Node

[ https://issues.apache.org/jira/browse/SPARK-23824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-23824. --- Resolution: Duplicate > Make inpurityStats publicly accessible in ml.tree.Node >

[jira] [Comment Edited] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452790#comment-16452790 ] Bruce Robbins edited comment on SPARK-23715 at 4/25/18 10:00 PM: --

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453209#comment-16453209 ] Jungtaek Lim commented on SPARK-24036: -- Maybe better to share what I've observed fro

[jira] [Comment Edited] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453209#comment-16453209 ] Jungtaek Lim edited comment on SPARK-24036 at 4/25/18 10:54 PM: ---

[jira] [Commented] (SPARK-22210) Online LDA variationalTopicInference should use random seed to have stable behavior

[ https://issues.apache.org/jira/browse/SPARK-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453228#comment-16453228 ] Joseph K. Bradley commented on SPARK-22210: --- [~lu.DB] Would you like to do this

[jira] [Commented] (SPARK-24070) TPC-DS Performance Tests for Parquet 1.10.0 Upgrade

[ https://issues.apache.org/jira/browse/SPARK-24070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453258#comment-16453258 ] Takeshi Yamamuro commented on SPARK-24070: -- Sure, I checked the numbers and see:

[jira] [Created] (SPARK-24094) Change description strings of v2 streaming sources to reflect the change

Tathagata Das created SPARK-24094: - Summary: Change description strings of v2 streaming sources to reflect the change Key: SPARK-24094 URL: https://issues.apache.org/jira/browse/SPARK-24094 Project: S

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453290#comment-16453290 ] Jungtaek Lim commented on SPARK-24036: -- Btw, I would like to say the idea for iterat

[jira] [Resolved] (SPARK-24069) Add array_max / array_min functions

[ https://issues.apache.org/jira/browse/SPARK-24069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24069. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21142 [https://git

[jira] [Assigned] (SPARK-24069) Add array_max / array_min functions

[ https://issues.apache.org/jira/browse/SPARK-24069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24069: Assignee: Hyukjin Kwon > Add array_max / array_min functions > ---

[jira] [Resolved] (SPARK-24092) spark.python.worker.reuse does not work?

[ https://issues.apache.org/jira/browse/SPARK-24092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24092. -- Resolution: Invalid Questions should go to mailing list. You could have a better and quicker an

[jira] [Resolved] (SPARK-24086) Exception while executing spark streaming examples

[ https://issues.apache.org/jira/browse/SPARK-24086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24086. -- Resolution: Invalid >From a quick look, that sounds because you didn't provide a profile for Ka

  1   2   >