[jira] [Assigned] (SPARK-28013) Upgrade to Kafka 2.2.1

[ https://issues.apache.org/jira/browse/SPARK-28013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28013: Assignee: (was: Apache Spark) > Upgrade to Kafka 2.2.1 > -- > >

[jira] [Assigned] (SPARK-28013) Upgrade to Kafka 2.2.1

[ https://issues.apache.org/jira/browse/SPARK-28013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28013: Assignee: Apache Spark > Upgrade to Kafka 2.2.1 > -- > >

[jira] [Created] (SPARK-28013) Upgrade to Kafka 2.2.1

Dongjoon Hyun created SPARK-28013: - Summary: Upgrade to Kafka 2.2.1 Key: SPARK-28013 URL: https://issues.apache.org/jira/browse/SPARK-28013 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-25520) The state of executors is KILLED on standalone

[ https://issues.apache.org/jira/browse/SPARK-25520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861760#comment-16861760 ] Dongjoon Hyun commented on SPARK-25520: --- In general, it looks like

[jira] [Comment Edited] (SPARK-25246) When the spark.eventLog.compress is enabled, the Application is not showing in the History server UI ('incomplete application' page), initially.

[ https://issues.apache.org/jira/browse/SPARK-25246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861755#comment-16861755 ] Dongjoon Hyun edited comment on SPARK-25246 at 6/12/19 5:10 AM: Hi,

[jira] [Resolved] (SPARK-25246) When the spark.eventLog.compress is enabled, the Application is not showing in the History server UI ('incomplete application' page), initially.

[ https://issues.apache.org/jira/browse/SPARK-25246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25246. --- Resolution: Not A Problem > When the spark.eventLog.compress is enabled, the Application is

[jira] [Commented] (SPARK-25246) When the spark.eventLog.compress is enabled, the Application is not showing in the History server UI ('incomplete application' page), initially.

[ https://issues.apache.org/jira/browse/SPARK-25246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861755#comment-16861755 ] Dongjoon Hyun commented on SPARK-25246: --- Hi, [~shahid]. As [~devaraj.k] mentioned, you can use a

[jira] [Comment Edited] (SPARK-27891) Long running spark jobs fail because of HDFS delegation token expires

[ https://issues.apache.org/jira/browse/SPARK-27891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861751#comment-16861751 ] hemshankar sahu edited comment on SPARK-27891 at 6/12/19 4:54 AM: --

[jira] [Updated] (SPARK-27891) Long running spark jobs fail because of HDFS delegation token expires

[ https://issues.apache.org/jira/browse/SPARK-27891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hemshankar sahu updated SPARK-27891: Priority: Critical (was: Major) Updating this as critical, as we use spark streaming

[jira] [Assigned] (SPARK-28012) Hive UDF supports literal struct type

[ https://issues.apache.org/jira/browse/SPARK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28012: Assignee: Apache Spark > Hive UDF supports literal struct type >

[jira] [Assigned] (SPARK-28012) Hive UDF supports literal struct type

[ https://issues.apache.org/jira/browse/SPARK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28012: Assignee: (was: Apache Spark) > Hive UDF supports literal struct type >

[jira] [Created] (SPARK-28012) Hive UDF supports literal struct type

dzcxzl created SPARK-28012: -- Summary: Hive UDF supports literal struct type Key: SPARK-28012 URL: https://issues.apache.org/jira/browse/SPARK-28012 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-27463) Support Dataframe Cogroup via Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861749#comment-16861749 ] Hyukjin Kwon commented on SPARK-27463: -- BTW, I think we have cogroup at Dataset in Scala side. How

[jira] [Commented] (SPARK-27463) Support Dataframe Cogroup via Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861747#comment-16861747 ] Hyukjin Kwon commented on SPARK-27463: -- [~d80tb7] are you still working on this? If there are some

[jira] [Created] (SPARK-28011) SQL parse error when there are too many aliases in the table

U Shaw created SPARK-28011: -- Summary: SQL parse error when there are too many aliases in the table Key: SPARK-28011 URL: https://issues.apache.org/jira/browse/SPARK-28011 Project: Spark Issue

[jira] [Commented] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861728#comment-16861728 ] Henry Yu commented on SPARK-27812: -- OK [~dongjoon] > kubernetes client import non-daemon thread which

[jira] [Updated] (SPARK-27741) Transitivity on predicate pushdown

[ https://issues.apache.org/jira/browse/SPARK-27741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] U Shaw updated SPARK-27741: --- Fix Version/s: 2.4.0 > Transitivity on predicate pushdown > --- > >

[jira] [Created] (SPARK-28010) Support ORDER BY ... USING syntax

Lantao Jin created SPARK-28010: -- Summary: Support ORDER BY ... USING syntax Key: SPARK-28010 URL: https://issues.apache.org/jira/browse/SPARK-28010 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-27701) Extend NestedColumnAliasing to more nested field cases

[ https://issues.apache.org/jira/browse/SPARK-27701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27701. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 3.0.0 This is

[jira] [Commented] (SPARK-27768) Infinity, -Infinity, NaN should be recognized in a case insensitive manner

[ https://issues.apache.org/jira/browse/SPARK-27768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861711#comment-16861711 ] Dongjoon Hyun commented on SPARK-27768: --- For this issue, I changed my mind to support this

[jira] [Commented] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861707#comment-16861707 ] Junjie Chen commented on SPARK-27499: - Yes, In KubernetesExecutorBuilder.scala, the

[jira] [Resolved] (SPARK-21136) Misleading error message for typo in SQL

[ https://issues.apache.org/jira/browse/SPARK-21136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-21136. -- Resolution: Fixed > Misleading error message for typo in SQL >

[jira] [Commented] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861646#comment-16861646 ] Hyukjin Kwon commented on SPARK-28006: -- The proposal itself looks making sense to me from a cursory

[jira] [Created] (SPARK-28009) PipedRDD: Block not locked for reading failure

Douglas Colkitt created SPARK-28009: --- Summary: PipedRDD: Block not locked for reading failure Key: SPARK-28009 URL: https://issues.apache.org/jira/browse/SPARK-28009 Project: Spark Issue

[jira] [Created] (SPARK-28008) Default values & column comments in AVRO schema converters

Mathew Wicks created SPARK-28008: Summary: Default values & column comments in AVRO schema converters Key: SPARK-28008 URL: https://issues.apache.org/jira/browse/SPARK-28008 Project: Spark

[jira] [Updated] (SPARK-28007) Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in Postgres/Redshift

[ https://issues.apache.org/jira/browse/SPARK-28007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-28007: --- Summary: Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in Postgres/Redshift

[jira] [Created] (SPARK-28007) Caret operator (^) means bitwise XOR in Spark and exponentiation in Postgres/Redshift

Josh Rosen created SPARK-28007: -- Summary: Caret operator (^) means bitwise XOR in Spark and exponentiation in Postgres/Redshift Key: SPARK-28007 URL: https://issues.apache.org/jira/browse/SPARK-28007

[jira] [Commented] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861559#comment-16861559 ] Li Jin commented on SPARK-28006: cc [~hyukjin.kwon] [~LI,Xiao] [~ueshin] [~bryanc] I think code wise

[jira] [Updated] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-28006: --- Description: Currently, pandas_udf supports "grouped aggregate" type that can be used with unbounded and

[jira] [Created] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

Li Jin created SPARK-28006: -- Summary: User-defined grouped transform pandas_udf for window operations Key: SPARK-28006 URL: https://issues.apache.org/jira/browse/SPARK-28006 Project: Spark Issue

[jira] [Assigned] (SPARK-28003) spark.createDataFrame with Arrow doesn't work with pandas.NaT

[ https://issues.apache.org/jira/browse/SPARK-28003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28003: Assignee: (was: Apache Spark) > spark.createDataFrame with Arrow doesn't work with

[jira] [Assigned] (SPARK-28003) spark.createDataFrame with Arrow doesn't work with pandas.NaT

[ https://issues.apache.org/jira/browse/SPARK-28003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28003: Assignee: Apache Spark > spark.createDataFrame with Arrow doesn't work with pandas.NaT

[jira] [Commented] (SPARK-28005) SparkRackResolver should not log for resolving empty list

[ https://issues.apache.org/jira/browse/SPARK-28005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861410#comment-16861410 ] Imran Rashid commented on SPARK-28005: -- cc [~cltlfcjin] > SparkRackResolver should not log for

[jira] [Created] (SPARK-28005) SparkRackResolver should not log for resolving empty list

Imran Rashid created SPARK-28005: Summary: SparkRackResolver should not log for resolving empty list Key: SPARK-28005 URL: https://issues.apache.org/jira/browse/SPARK-28005 Project: Spark

[jira] [Assigned] (SPARK-28004) Update jquery to 3.4.1

[ https://issues.apache.org/jira/browse/SPARK-28004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28004: Assignee: Sean Owen (was: Apache Spark) > Update jquery to 3.4.1 >

[jira] [Assigned] (SPARK-28004) Update jquery to 3.4.1

[ https://issues.apache.org/jira/browse/SPARK-28004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28004: Assignee: Apache Spark (was: Sean Owen) > Update jquery to 3.4.1 >

[jira] [Updated] (SPARK-28004) Update jquery to 3.4.1

[ https://issues.apache.org/jira/browse/SPARK-28004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28004: -- Description: We're using an old-ish jQuery, 1.12.4, and should probably update for Spark 3 to keep

[jira] [Created] (SPARK-28004) Update jquery to 3.4.1

Sean Owen created SPARK-28004: - Summary: Update jquery to 3.4.1 Key: SPARK-28004 URL: https://issues.apache.org/jira/browse/SPARK-28004 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-27360) Standalone cluster mode support for GPU-aware scheduling

[ https://issues.apache.org/jira/browse/SPARK-27360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861353#comment-16861353 ] Xiangrui Meng commented on SPARK-27360: --- Done. Thanks for taking the task! > Standalone cluster

[jira] [Deleted] (SPARK-27999) setup resources when Standalone Worker starts up

[ https://issues.apache.org/jira/browse/SPARK-27999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng deleted SPARK-27999: -- > setup resources when Standalone Worker starts up >

[jira] [Assigned] (SPARK-27369) Standalone worker can load resource conf and discover resources

[ https://issues.apache.org/jira/browse/SPARK-27369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-27369: -- Assignee: wuyi > Standalone worker can load resource conf and discover resources >

[jira] [Commented] (SPARK-26185) add weightCol in python MulticlassClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-26185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861261#comment-16861261 ] Huaxin Gao commented on SPARK-26185: resolved in another PR > add weightCol in python

[jira] [Updated] (SPARK-28003) spark.createDataFrame with Arrow doesn't work with pandas.NaT

[ https://issues.apache.org/jira/browse/SPARK-28003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-28003: --- Affects Version/s: (was: 2.3.2) 2.3.3 > spark.createDataFrame with Arrow doesn't

[jira] [Created] (SPARK-28003) spark.createDataFrame with Arrow doesn't work with pandas.NaT

Li Jin created SPARK-28003: -- Summary: spark.createDataFrame with Arrow doesn't work with pandas.NaT Key: SPARK-28003 URL: https://issues.apache.org/jira/browse/SPARK-28003 Project: Spark Issue

[jira] [Updated] (SPARK-28003) spark.createDataFrame with Arrow doesn't work with pandas.NaT

[ https://issues.apache.org/jira/browse/SPARK-28003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-28003: --- Affects Version/s: (was: 2.4.0) 2.3.2 2.4.3 >

[jira] [Assigned] (SPARK-27258) The value of "spark.app.name" or "--name" starts with number , which causes resourceName does not match regular expression

[ https://issues.apache.org/jira/browse/SPARK-27258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27258: Assignee: Apache Spark > The value of "spark.app.name" or "--name" starts with number ,

[jira] [Assigned] (SPARK-27258) The value of "spark.app.name" or "--name" starts with number , which causes resourceName does not match regular expression

[ https://issues.apache.org/jira/browse/SPARK-27258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27258: Assignee: (was: Apache Spark) > The value of "spark.app.name" or "--name" starts

[jira] [Assigned] (SPARK-28002) Support WITH clause column aliases

[ https://issues.apache.org/jira/browse/SPARK-28002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28002: Assignee: Apache Spark > Support WITH clause column aliases >

[jira] [Assigned] (SPARK-28002) Support WITH clause column aliases

[ https://issues.apache.org/jira/browse/SPARK-28002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28002: Assignee: (was: Apache Spark) > Support WITH clause column aliases >

[jira] [Created] (SPARK-28002) Support WITH clause column aliases

Peter Toth created SPARK-28002: -- Summary: Support WITH clause column aliases Key: SPARK-28002 URL: https://issues.apache.org/jira/browse/SPARK-28002 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-27917) Semantic equals of CaseWhen is failing with case sensitivity of column Names

[ https://issues.apache.org/jira/browse/SPARK-27917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27917: -- Fix Version/s: 2.4.4 > Semantic equals of CaseWhen is failing with case sensitivity of column

[jira] [Created] (SPARK-28001) Dataframe throws 'socket.timeout: timed out' exception

Marius Stanescu created SPARK-28001: --- Summary: Dataframe throws 'socket.timeout: timed out' exception Key: SPARK-28001 URL: https://issues.apache.org/jira/browse/SPARK-28001 Project: Spark

[jira] [Commented] (SPARK-9983) Local physical operators for query execution

[ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860976#comment-16860976 ] Lai Zhou commented on SPARK-9983: - [~rxin], we now use Calcite to build a high performance hive sql

[jira] [Created] (SPARK-28000) Add comments.sql

Lantao Jin created SPARK-28000: -- Summary: Add comments.sql Key: SPARK-28000 URL: https://issues.apache.org/jira/browse/SPARK-28000 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Comment Edited] (SPARK-9983) Local physical operators for query execution

[ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860976#comment-16860976 ] Lai Zhou edited comment on SPARK-9983 at 6/11/19 12:18 PM: --- [~rxin], we now use

[jira] [Comment Edited] (SPARK-9983) Local physical operators for query execution

[ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860976#comment-16860976 ] Lai Zhou edited comment on SPARK-9983 at 6/11/19 12:15 PM: --- [~rxin], we now use

[jira] [Assigned] (SPARK-27369) Standalone worker can load resource conf and discover resources

[ https://issues.apache.org/jira/browse/SPARK-27369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27369: Assignee: Apache Spark > Standalone worker can load resource conf and discover resources

[jira] [Assigned] (SPARK-27369) Standalone worker can load resource conf and discover resources

[ https://issues.apache.org/jira/browse/SPARK-27369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27369: Assignee: (was: Apache Spark) > Standalone worker can load resource conf and

[jira] [Comment Edited] (SPARK-9983) Local physical operators for query execution

[ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860959#comment-16860959 ] Lai Zhou edited comment on SPARK-9983 at 6/11/19 11:51 AM: --- [~rxin], `a

[jira] [Commented] (SPARK-9983) Local physical operators for query execution

[ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860959#comment-16860959 ] Lai Zhou commented on SPARK-9983: - [~rxin], `a hyper-optimized single-node version of DataFrame`, do you

[jira] [Reopened] (SPARK-27258) The value of "spark.app.name" or "--name" starts with number , which causes resourceName does not match regular expression

[ https://issues.apache.org/jira/browse/SPARK-27258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-27258: --- > The value of "spark.app.name" or "--name" starts with number , which causes > resourceName does not

[jira] [Commented] (SPARK-27360) Standalone cluster mode support for GPU-aware scheduling

[ https://issues.apache.org/jira/browse/SPARK-27360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860951#comment-16860951 ] wuyi commented on SPARK-27360: -- oops...I happened to find that sub-task 2 has changed to what I want after

[jira] [Created] (SPARK-27999) setup resources when Standalone Worker starts up

wuyi created SPARK-27999: Summary: setup resources when Standalone Worker starts up Key: SPARK-27999 URL: https://issues.apache.org/jira/browse/SPARK-27999 Project: Spark Issue Type: Sub-task

[jira] [Comment Edited] (SPARK-27258) The value of "spark.app.name" or "--name" starts with number , which causes resourceName does not match regular expression

[ https://issues.apache.org/jira/browse/SPARK-27258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860924#comment-16860924 ] Stavros Kontopoulos edited comment on SPARK-27258 at 6/11/19 11:03 AM:

[jira] [Comment Edited] (SPARK-27258) The value of "spark.app.name" or "--name" starts with number , which causes resourceName does not match regular expression

[ https://issues.apache.org/jira/browse/SPARK-27258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860924#comment-16860924 ] Stavros Kontopoulos edited comment on SPARK-27258 at 6/11/19 11:03 AM:

[jira] [Comment Edited] (SPARK-27258) The value of "spark.app.name" or "--name" starts with number , which causes resourceName does not match regular expression

[ https://issues.apache.org/jira/browse/SPARK-27258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860924#comment-16860924 ] Stavros Kontopoulos edited comment on SPARK-27258 at 6/11/19 11:02 AM:

[jira] [Comment Edited] (SPARK-27258) The value of "spark.app.name" or "--name" starts with number , which causes resourceName does not match regular expression

[ https://issues.apache.org/jira/browse/SPARK-27258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860924#comment-16860924 ] Stavros Kontopoulos edited comment on SPARK-27258 at 6/11/19 11:01 AM:

[jira] [Commented] (SPARK-27258) The value of "spark.app.name" or "--name" starts with number , which causes resourceName does not match regular expression

[ https://issues.apache.org/jira/browse/SPARK-27258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860924#comment-16860924 ] Stavros Kontopoulos commented on SPARK-27258: - [~hehuiyuan] the ticket needs to be

[jira] [Comment Edited] (SPARK-24875) MulticlassMetrics should offer a more efficient way to compute count by label

[ https://issues.apache.org/jira/browse/SPARK-24875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860922#comment-16860922 ] zhengruifeng edited comment on SPARK-24875 at 6/11/19 10:59 AM: The

[jira] [Commented] (SPARK-24875) MulticlassMetrics should offer a more efficient way to compute count by label

[ https://issues.apache.org/jira/browse/SPARK-24875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860922#comment-16860922 ] zhengruifeng commented on SPARK-24875: -- The dataset is usually much smaller than the training

[jira] [Commented] (SPARK-26185) add weightCol in python MulticlassClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-26185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860919#comment-16860919 ] zhengruifeng commented on SPARK-26185: -- Seems resolved? > add weightCol in python

[jira] [Updated] (SPARK-27759) Do not auto cast array to np.array in vectorized udf

[ https://issues.apache.org/jira/browse/SPARK-27759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] colin fang updated SPARK-27759: --- Description: {code:java} pd_df = pd.DataFrame({'x': np.random.rand(11, 3, 5).tolist()}) df =

[jira] [Assigned] (SPARK-27998) Column alias should support double quote string

[ https://issues.apache.org/jira/browse/SPARK-27998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27998: Assignee: Apache Spark > Column alias should support double quote string >

[jira] [Assigned] (SPARK-27998) Column alias should support double quote string

[ https://issues.apache.org/jira/browse/SPARK-27998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27998: Assignee: (was: Apache Spark) > Column alias should support double quote string >

[jira] [Commented] (SPARK-25360) Parallelized RDDs of Ranges could have known partitioner

[ https://issues.apache.org/jira/browse/SPARK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860759#comment-16860759 ] zhengruifeng commented on SPARK-25360: -- But I think it maybe worth to impl a direct version of

[jira] [Commented] (SPARK-25360) Parallelized RDDs of Ranges could have known partitioner

[ https://issues.apache.org/jira/browse/SPARK-25360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860756#comment-16860756 ] zhengruifeng commented on SPARK-25360: -- [~holdenk] i am afraid it is not doable to add a

[jira] [Resolved] (SPARK-27995) Note the difference between str of Python 2 and 3 at Arrow optimized

[ https://issues.apache.org/jira/browse/SPARK-27995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27995. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24838

[jira] [Assigned] (SPARK-27995) Note the difference between str of Python 2 and 3 at Arrow optimized

[ https://issues.apache.org/jira/browse/SPARK-27995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27995: Assignee: Hyukjin Kwon > Note the difference between str of Python 2 and 3 at Arrow

[jira] [Updated] (SPARK-27995) Note the difference between str of Python 2 and 3 at Arrow optimized

[ https://issues.apache.org/jira/browse/SPARK-27995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27995: - Summary: Note the difference between str of Python 2 and 3 at Arrow optimized (was: Note the

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860734#comment-16860734 ] HonglunChen commented on SPARK-18112: - [~hyukjin.kwon] Thanks, I did not remove the 1.2.1 jars, and

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860711#comment-16860711 ] Hyukjin Kwon commented on SPARK-18112: -- you should place 2.3.3 jars if you want do use 2.3.3. >

[jira] [Updated] (SPARK-27998) Column alias should support double quote string

[ https://issues.apache.org/jira/browse/SPARK-27998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27998: Affects Version/s: (was: 3.1.0) 3.0.0 > Column alias should support

[jira] [Created] (SPARK-27998) Column alias should support double quote string

Zhu, Lipeng created SPARK-27998: --- Summary: Column alias should support double quote string Key: SPARK-27998 URL: https://issues.apache.org/jira/browse/SPARK-27998 Project: Spark Issue Type:

[jira] [Commented] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860648#comment-16860648 ] Dongjoon Hyun commented on SPARK-27499: --- Oh, do you mean `SPARK_LOCAL_DIRS` doesn't work on the

[jira] [Resolved] (SPARK-27934) Add case.sql

[ https://issues.apache.org/jira/browse/SPARK-27934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27934. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 3.0.0 > Add case.sql >

[jira] [Resolved] (SPARK-27899) Make HiveMetastoreClient.getTableObjectsByName available in ExternalCatalog/SessionCatalog API

[ https://issues.apache.org/jira/browse/SPARK-27899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27899. - Resolution: Fixed Fix Version/s: 3.0.0 > Make HiveMetastoreClient.getTableObjectsByName

[jira] [Updated] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27499: -- Fix Version/s: (was: 2.4.0) > Support mapping spark.local.dir to hostPath volume >

[jira] [Comment Edited] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860598#comment-16860598 ] Dongjoon Hyun edited comment on SPARK-27499 at 6/11/19 7:42 AM: No, I

[jira] [Updated] (SPARK-24175) improve the Spark 2.4 migration guide

[ https://issues.apache.org/jira/browse/SPARK-24175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24175: -- Component/s: Documentation > improve the Spark 2.4 migration guide >

[jira] [Updated] (SPARK-24125) Add quoting rules to SQL guide

[ https://issues.apache.org/jira/browse/SPARK-24125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24125: -- Component/s: (was: SQL) Documentation > Add quoting rules to SQL guide >

[jira] [Updated] (SPARK-24125) Add quoting rules to SQL guide

[ https://issues.apache.org/jira/browse/SPARK-24125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24125: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Add quoting rules to SQL

[jira] [Updated] (SPARK-24584) More efficient storage of executor pod state

[ https://issues.apache.org/jira/browse/SPARK-24584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24584: -- Summary: More efficient storage of executor pod state (was: [K8s] More efficient storage of

[jira] [Updated] (SPARK-24254) Eagerly evaluate some subqueries over LocalRelation

[ https://issues.apache.org/jira/browse/SPARK-24254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24254: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Eagerly evaluate some

[jira] [Updated] (SPARK-24584) [K8s] More efficient storage of executor pod state

[ https://issues.apache.org/jira/browse/SPARK-24584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24584: -- Affects Version/s: (was: 2.4.0) 3.0.0 > [K8s] More efficient

[jira] [Updated] (SPARK-25050) Handle more than two types in avro union types when writing avro files

[ https://issues.apache.org/jira/browse/SPARK-25050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25050: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Handle more than two types

[jira] [Resolved] (SPARK-24831) Spark Executor class create new serializer each time

[ https://issues.apache.org/jira/browse/SPARK-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-24831. --- Resolution: Incomplete Sorry, but JIRA is not for questions. > Spark Executor class create

[jira] [Updated] (SPARK-25396) Read array of JSON objects via an Iterator

[ https://issues.apache.org/jira/browse/SPARK-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25396: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Read array of JSON objects

[jira] [Updated] (SPARK-27013) Consider adding support for external encoders when resolving org.apache.spark.sql.catalyst.encoders.ExpressionEncoder's apply method

[ https://issues.apache.org/jira/browse/SPARK-27013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27013: -- Component/s: (was: Optimizer) SQL > Consider adding support for external

[jira] [Updated] (SPARK-26948) vertex and edge rowkey upgrade and support multiple types?

[ https://issues.apache.org/jira/browse/SPARK-26948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26948: -- Affects Version/s: (was: 2.4.0) 3.0.0 > vertex and edge rowkey

[jira] [Updated] (SPARK-27013) Consider adding support for external encoders when resolving org.apache.spark.sql.catalyst.encoders.ExpressionEncoder's apply method

[ https://issues.apache.org/jira/browse/SPARK-27013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27013: -- Affects Version/s: (was: 2.4.0) 3.0.0 > Consider adding support

  1   2   >