[
https://issues.apache.org/jira/browse/SPARK-28898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919210#comment-16919210
]
Hyukjin Kwon commented on SPARK-28898:
--
BTW, I wonder if there's a good way to automatically
[
https://issues.apache.org/jira/browse/SPARK-28898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919209#comment-16919209
]
Hyukjin Kwon commented on SPARK-28898:
--
I think we don't need to list entire list up but might be
[
https://issues.apache.org/jira/browse/SPARK-28900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919203#comment-16919203
]
Hyukjin Kwon edited comment on SPARK-28900 at 8/30/19 5:28 AM:
---
Just quick
[
https://issues.apache.org/jira/browse/SPARK-28900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919203#comment-16919203
]
Hyukjin Kwon commented on SPARK-28900:
--
Just quick FYI, {{spark-master-test-maven-hadoop-3.2]}
[
https://issues.apache.org/jira/browse/SPARK-28809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivu Sondur updated SPARK-28809:
-
Comment: was deleted
(was: [~dkbiswal]
i am not able to find the "SHOW TABLE" command in spark
[
https://issues.apache.org/jira/browse/SPARK-28906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919190#comment-16919190
]
Hyukjin Kwon commented on SPARK-28906:
--
cc [~kiszk] FYI
> `bin/spark-submit --version` shows
[
https://issues.apache.org/jira/browse/SPARK-28916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919188#comment-16919188
]
Hyukjin Kwon commented on SPARK-28916:
--
Reproducer:
{code}
val df = spark.range(100).selectExpr((0
[
https://issues.apache.org/jira/browse/SPARK-28910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-28910:
-
Description:
When {{hive.metastore.schema.verification=true}},
[
https://issues.apache.org/jira/browse/SPARK-28910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-28910:
-
Component/s: (was: Spark Core)
SQL
> Prevent schema verification when
[
https://issues.apache.org/jira/browse/SPARK-28861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919183#comment-16919183
]
Hyukjin Kwon commented on SPARK-28861:
--
ping [~Shrikhande]
> Jetty property handling:
[
https://issues.apache.org/jira/browse/SPARK-28912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919182#comment-16919182
]
Hyukjin Kwon commented on SPARK-28912:
--
It might be much easier for other poeple write a test and
[
https://issues.apache.org/jira/browse/SPARK-28924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919181#comment-16919181
]
Sharanabasappa G Keriwaddi commented on SPARK-28924:
[~abhishek.akg] , Thanks. I
[
https://issues.apache.org/jira/browse/SPARK-28899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919180#comment-16919180
]
Hyukjin Kwon commented on SPARK-28899:
--
Fixed in https://github.com/apache/spark/pull/25610
>
ABHISHEK KUMAR GUPTA created SPARK-28924:
Summary: spark.jobGroup.id and spark.job.description is missing in
Document
Key: SPARK-28924
URL: https://issues.apache.org/jira/browse/SPARK-28924
[
https://issues.apache.org/jira/browse/SPARK-28918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919173#comment-16919173
]
Hyukjin Kwon commented on SPARK-28918:
--
BTW {{from_utc_timestamp}} is deprecated as of Spark 3.0.
[
https://issues.apache.org/jira/browse/SPARK-28918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-28918.
--
Resolution: Not A Problem
> from_utc_timestamp function is mistakenly considering DST for
[
https://issues.apache.org/jira/browse/SPARK-28916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-28916:
-
Target Version/s: (was: 2.3.1, 2.4.3)
> Generated SpecificSafeProjection.apply method grows
Xianyin Xin created SPARK-28923:
---
Summary: Deduplicate the codes 'multipartIdentifier' and
'identifierSeq'
Key: SPARK-28923
URL: https://issues.apache.org/jira/browse/SPARK-28923
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-28920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-28920.
---
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 25625
[
https://issues.apache.org/jira/browse/SPARK-28920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-28920:
-
Assignee: Liang-Chi Hsieh
> Set up java version for github workflow
>
[
https://issues.apache.org/jira/browse/SPARK-28919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-28919.
---
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 25624
[
https://issues.apache.org/jira/browse/SPARK-28919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-28919:
-
Assignee: Dongjoon Hyun
> Add more profiles for JDK8/11 build test
>
[
https://issues.apache.org/jira/browse/SPARK-28922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-28922:
-
Assignee: Gabor Somogyi
> Safe Kafka parameter redaction
>
[
https://issues.apache.org/jira/browse/SPARK-28922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-28922.
---
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 25621
[
https://issues.apache.org/jira/browse/SPARK-28922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-28922:
--
Reporter: Gabor Somogyi (was: Dongjoon Hyun)
> Safe Kafka parameter redaction
>
Dongjoon Hyun created SPARK-28922:
-
Summary: Safe Kafka parameter redaction
Key: SPARK-28922
URL: https://issues.apache.org/jira/browse/SPARK-28922
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-28921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Schweigert updated SPARK-28921:
Description:
Spark jobs are failing on latest versions of Kubernetes when jobs attempt to
Paul Schweigert created SPARK-28921:
---
Summary: Spark jobs failing on latest versions of Kubernetes
(1.15.3, 1.14.6, 1,13.10)
Key: SPARK-28921
URL: https://issues.apache.org/jira/browse/SPARK-28921
[
https://issues.apache.org/jira/browse/SPARK-28843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-28843.
--
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 25545
[
https://issues.apache.org/jira/browse/SPARK-28843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-28843:
Assignee: Ryan Blue
> Set OMP_NUM_THREADS to executor cores reduce Python memory
Liang-Chi Hsieh created SPARK-28920:
---
Summary: Set up java version for github workflow
Key: SPARK-28920
URL: https://issues.apache.org/jira/browse/SPARK-28920
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-28918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919045#comment-16919045
]
Shivu Sondur commented on SPARK-28918:
--
[~hissashirocha]
Brazil DST removed on Sunday, 17 February
[
https://issues.apache.org/jira/browse/SPARK-28919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-28919:
--
Description:
>From Jenkins, we build with JDK8 and test with JDK8/11.
We are testing JDK11
Dongjoon Hyun created SPARK-28919:
-
Summary: Add more profiles for JDK8/11 build test
Key: SPARK-28919
URL: https://issues.apache.org/jira/browse/SPARK-28919
Project: Spark
Issue Type:
Luiz Hissashi da Rocha created SPARK-28918:
--
Summary: from_utc_timestamp function is mistakenly considering DST
for Brazil in 2019
Key: SPARK-28918
URL: https://issues.apache.org/jira/browse/SPARK-28918
[
https://issues.apache.org/jira/browse/SPARK-28899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue resolved SPARK-28899.
---
Fix Version/s: 3.0.0
Resolution: Fixed
> merge the testing in-memory v2 catalogs from
[
https://issues.apache.org/jira/browse/SPARK-28760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin resolved SPARK-28760.
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 25477
[
https://issues.apache.org/jira/browse/SPARK-28760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin reassigned SPARK-28760:
--
Assignee: Gabor Somogyi
> Add end-to-end Kafka delegation token test
>
[
https://issues.apache.org/jira/browse/SPARK-25874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918813#comment-16918813
]
Marcelo Vanzin commented on SPARK-25874:
I'm going to close this one; the only remaining task is
[
https://issues.apache.org/jira/browse/SPARK-25874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin resolved SPARK-25874.
Fix Version/s: 3.0.0
Resolution: Done
> Simplify abstractions in the K8S backend
>
[
https://issues.apache.org/jira/browse/SPARK-27611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-27611.
---
Fix Version/s: 3.0.0
Resolution: Fixed
Resolved by
[
https://issues.apache.org/jira/browse/SPARK-28807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-28807.
-
Fix Version/s: 3.0.0
Assignee: Dilip Biswal
Resolution: Fixed
> Document SHOW DATABASES
Imran Rashid created SPARK-28917:
Summary: Jobs can hang because of race of RDD.dependencies
Key: SPARK-28917
URL: https://issues.apache.org/jira/browse/SPARK-28917
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-28786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-28786.
-
Fix Version/s: 3.0.0
Assignee: Huaxin Gao
Resolution: Fixed
> Document INSERT statement
[
https://issues.apache.org/jira/browse/SPARK-28917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918764#comment-16918764
]
Imran Rashid commented on SPARK-28917:
--
[~markhamstra] [~jiangxb1987] [~tgraves] [~Ngone51] would
MOBIN created SPARK-28916:
-
Summary: Generated SpecificSafeProjection.apply method grows
beyond 64 KB when use SparkSQL
Key: SPARK-28916
URL: https://issues.apache.org/jira/browse/SPARK-28916
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-28915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Li updated SPARK-28915:
-
Description:
WorkerOffer is a case class, so we don't need to use new when instantiating it,
which
Jiaqi Li created SPARK-28915:
Summary: The new keyword is not used when instantiating the
WorkerOffer.
Key: SPARK-28915
URL: https://issues.apache.org/jira/browse/SPARK-28915
Project: Spark
Nirek Sharma created SPARK-28914:
Summary: GMM Returning Clusters With Equal Probability
Key: SPARK-28914
URL: https://issues.apache.org/jira/browse/SPARK-28914
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-28913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Wang updated SPARK-28913:
---
Description:
The stack trace is below:
{quote}19/08/28 07:00:40 WARN Executor task launch worker
[
https://issues.apache.org/jira/browse/SPARK-28913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Wang updated SPARK-28913:
---
Summary: ArrayIndexOutOfBoundsException and Not-stable AUC metrics in ALS
for datasets with 12
[
https://issues.apache.org/jira/browse/SPARK-28913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Wang updated SPARK-28913:
---
Description:
The stack trace is below:
{quote}19/08/28 07:00:40 WARN Executor task launch worker
[
https://issues.apache.org/jira/browse/SPARK-28913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Wang updated SPARK-28913:
---
Description:
The stack trace is below:
{quote}19/08/28 07:00:40 WARN Executor task launch worker
[
https://issues.apache.org/jira/browse/SPARK-28913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Wang updated SPARK-28913:
---
Summary: ArrayIndexOutOfBoundsException in ALS for datasets with 12
billion instances (was:
[
https://issues.apache.org/jira/browse/SPARK-28913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Wang updated SPARK-28913:
---
Description:
The stack trace is below:
{quote}19/08/28 07:00:40 WARN Executor task launch worker
[
https://issues.apache.org/jira/browse/SPARK-28913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Wang updated SPARK-28913:
---
Summary: ArrayIndexOutOfBoundsException in ALS for Large datasets (was:
CLONE -
Qiang Wang created SPARK-28913:
--
Summary: CLONE - ArrayIndexOutOfBoundsException in ALS for Large
datasets
Key: SPARK-28913
URL: https://issues.apache.org/jira/browse/SPARK-28913
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-28913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Wang updated SPARK-28913:
---
Affects Version/s: (was: 1.2.0)
(was: 1.1.0)
Aleksandr Kashkirov created SPARK-28912:
---
Summary: MatchError exception in CheckpointWriteHandler
Key: SPARK-28912
URL: https://issues.apache.org/jira/browse/SPARK-28912
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-28911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wenxuanguan updated SPARK-28911:
Description:
Pattern of datasource options is Camel-Case, such as CheckpointLocation, and
only
wenxuanguan created SPARK-28911:
---
Summary: Unify Kafka source option pattern
Key: SPARK-28911
URL: https://issues.apache.org/jira/browse/SPARK-28911
Project: Spark
Issue Type: Improvement
Juliusz Sompolski created SPARK-28910:
-
Summary: Prevent schema verification when connecting to in memory
derby
Key: SPARK-28910
URL: https://issues.apache.org/jira/browse/SPARK-28910
Project:
ABHISHEK KUMAR GUPTA created SPARK-28909:
Summary: ANSI SQL: delete and update does not support in Spark
Key: SPARK-28909
URL: https://issues.apache.org/jira/browse/SPARK-28909
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918386#comment-16918386
]
Saisai Shao commented on SPARK-28340:
-
My simple concern is that there may be other places which
[
https://issues.apache.org/jira/browse/SPARK-28908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wenxuanguan updated SPARK-28908:
Affects Version/s: (was: 2.4.3)
3.0.0
> Structured Streaming Kafka
[
https://issues.apache.org/jira/browse/SPARK-28908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wenxuanguan updated SPARK-28908:
Attachment: Kafka Sink Exactly-Once Semantics Design Sketch.pdf
> Structured Streaming Kafka sink
[
https://issues.apache.org/jira/browse/SPARK-28908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wenxuanguan updated SPARK-28908:
Attachment: Kafka Sink Exactly-Once Semantics Design Sketch.pdf
> Structured Streaming Kafka sink
[
https://issues.apache.org/jira/browse/SPARK-28908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wenxuanguan updated SPARK-28908:
Attachment: (was: Kafka Sink Exactly-Once Semantics Design Sketch.pdf)
> Structured Streaming
wenxuanguan created SPARK-28908:
---
Summary: Structured Streaming Kafka sink support Exactly-Once
semantics
Key: SPARK-28908
URL: https://issues.apache.org/jira/browse/SPARK-28908
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-28908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wenxuanguan updated SPARK-28908:
Description: Since Apache Kafka supports transaction from 0.11.0.0, we can
implement Kafka sink
[
https://issues.apache.org/jira/browse/SPARK-28678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918371#comment-16918371
]
Hyukjin Kwon commented on SPARK-28678:
--
It doesn't have to assign someone. If you open a PR, then
Xianjin YE created SPARK-28907:
--
Summary: Review invalid usage of new Configuration()
Key: SPARK-28907
URL: https://issues.apache.org/jira/browse/SPARK-28907
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-28678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918341#comment-16918341
]
Ting Yang commented on SPARK-28678:
---
I'd like to work on this, can you assign it to me.
[
https://issues.apache.org/jira/browse/SPARK-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918306#comment-16918306
]
Saisai Shao commented on SPARK-28340:
-
We also saw a bunch of exceptions in our production
74 matches
Mail list logo
