[
https://issues.apache.org/jira/browse/SPARK-46590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-46590:
---
Attachment: problem.log
> Coalesce partiton assert error after skew join optimization
>
Jackey Lee created SPARK-46590:
--
Summary: Coalesce partiton assert error after skew join
optimization
Key: SPARK-46590
URL: https://issues.apache.org/jira/browse/SPARK-46590
Project: Spark
Jackey Lee created SPARK-46567:
--
Summary: Remove ThreadLocal for ReadAheadInputStream
Key: SPARK-46567
URL: https://issues.apache.org/jira/browse/SPARK-46567
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-38427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690971#comment-17690971
]
Jackey Lee commented on SPARK-38427:
[~LuciferYang]
> DataFilter pushed down with PartitionFilter
[ https://issues.apache.org/jira/browse/SPARK-38427 ]
Jackey Lee deleted comment on SPARK-38427:
was (Author: jackey lee):
[~LuciferYang]
> DataFilter pushed down with PartitionFilter for Orc
>
[
https://issues.apache.org/jira/browse/SPARK-38427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690969#comment-17690969
]
Jackey Lee commented on SPARK-38427:
[~LuciferYang]
> DataFilter pushed down with PartitionFilter
[
https://issues.apache.org/jira/browse/SPARK-40708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17623543#comment-17623543
]
Jackey Lee commented on SPARK-40708:
Maybe we can use `CommitProtocol`, which can return more
Jackey Lee created SPARK-40150:
--
Summary: Dynamically merge File Splits
Key: SPARK-40150
URL: https://issues.apache.org/jira/browse/SPARK-40150
Project: Spark
Issue Type: Improvement
Jackey Lee created SPARK-38713:
--
Summary: Change spark.sessionstate.conf.getConf/setConf operation
to spark.conf.get/set
Key: SPARK-38713
URL: https://issues.apache.org/jira/browse/SPARK-38713
Project:
[
https://issues.apache.org/jira/browse/SPARK-38521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-38521:
---
Affects Version/s: 3.4.0
(was: 3.3.0)
> Throw Exception if overwriting
[
https://issues.apache.org/jira/browse/SPARK-38521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-38521:
---
Fix Version/s: 3.4.0
(was: 3.3.0)
> Throw Exception if overwriting hive
[
https://issues.apache.org/jira/browse/SPARK-38427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-38427:
---
Affects Version/s: 3.4.0
(was: 3.3.0)
> DataFilter pushed down with
[
https://issues.apache.org/jira/browse/SPARK-38440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-38440:
---
Affects Version/s: 3.4.0
(was: 3.3.0)
> DataFilter pushed down with
[
https://issues.apache.org/jira/browse/SPARK-38433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-38433:
---
Affects Version/s: 3.4.0
(was: 3.3.0)
> Add Shell Code Style Check
[
https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-37933:
---
Affects Version/s: 3.4.0
(was: 3.3.0)
> Limit push down for parquet
[
https://issues.apache.org/jira/browse/SPARK-38041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-38041:
---
Affects Version/s: 3.4.0
(was: 3.3.0)
> DataFilter pushed down with
[
https://issues.apache.org/jira/browse/SPARK-37919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-37919:
---
Affects Version/s: 3.4.0
(was: 3.3.0)
> SQL UI shows accurate Metrics
[
https://issues.apache.org/jira/browse/SPARK-37831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-37831:
---
Affects Version/s: (was: 3.2.1)
> Add task partition id in metrics
>
Jackey Lee created SPARK-38521:
--
Summary: Throw Exception if overwriting hive partition table with
dynamic and staticPartitionOverwriteMode
Key: SPARK-38521
URL: https://issues.apache.org/jira/browse/SPARK-38521
Jackey Lee created SPARK-38440:
--
Summary: DataFilter pushed down with PartitionFilter fro Parquet
V1 Datasource
Key: SPARK-38440
URL: https://issues.apache.org/jira/browse/SPARK-38440
Project: Spark
Jackey Lee created SPARK-38433:
--
Summary: Add Shell Code Style Check Action
Key: SPARK-38433
URL: https://issues.apache.org/jira/browse/SPARK-38433
Project: Spark
Issue Type: Improvement
Jackey Lee created SPARK-38427:
--
Summary: DataFilter pushed down with PartitionFilter for Orc
Key: SPARK-38427
URL: https://issues.apache.org/jira/browse/SPARK-38427
Project: Spark
Issue Type:
Jackey Lee created SPARK-38419:
--
Summary: Remove tab character and trailing space in script
Key: SPARK-38419
URL: https://issues.apache.org/jira/browse/SPARK-38419
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-38041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-38041:
---
Description:
At present, the Filter is divided into DataFilter and PartitionFilter when it
is
[
https://issues.apache.org/jira/browse/SPARK-38041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17483527#comment-17483527
]
Jackey Lee commented on SPARK-38041:
Thanks for your attention, [~hyukjin.kwon]. I'm working on this
Jackey Lee created SPARK-38041:
--
Summary: DataFilter pushed down with PartitionFilter
Key: SPARK-38041
URL: https://issues.apache.org/jira/browse/SPARK-38041
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee reopened SPARK-37933:
The pre pr is only to optimize a part of the code, there will be other prs for
this jira in the
[ https://issues.apache.org/jira/browse/SPARK-37933 ]
Jackey Lee deleted comment on SPARK-37933:
was (Author: jackey lee):
[https://github.com/apache/spark/pull/35242]
This pr is only to optimize a part of the code, there will be other prs for
[
https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477838#comment-17477838
]
Jackey Lee commented on SPARK-37933:
[https://github.com/apache/spark/pull/35242]
This pr is only
Jackey Lee created SPARK-37933:
--
Summary: Limit push down for parquet datasource v2
Key: SPARK-37933
URL: https://issues.apache.org/jira/browse/SPARK-37933
Project: Spark
Issue Type:
Jackey Lee created SPARK-37920:
--
Summary: Change Code style for pom.xml
Key: SPARK-37920
URL: https://issues.apache.org/jira/browse/SPARK-37920
Project: Spark
Issue Type: Bug
Jackey Lee created SPARK-37919:
--
Summary: SQL UI shows accurate Metrics with stage retries
Key: SPARK-37919
URL: https://issues.apache.org/jira/browse/SPARK-37919
Project: Spark
Issue Type:
Jackey Lee created SPARK-37860:
--
Summary: [BUG] Revert: Fix taskid in the stage page task event
timeline
Key: SPARK-37860
URL: https://issues.apache.org/jira/browse/SPARK-37860
Project: Spark
Jackey Lee created SPARK-37831:
--
Summary: Add task partition id in metrics
Key: SPARK-37831
URL: https://issues.apache.org/jira/browse/SPARK-37831
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-32512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-32512:
---
Description: This Jira is trying to add basic partition command API,
`AlterTableAddPartitionExec`
Jackey Lee created SPARK-32512:
--
Summary: Add basic partition command for datasourcev2
Key: SPARK-32512
URL: https://issues.apache.org/jira/browse/SPARK-32512
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-31694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114467#comment-17114467
]
Jackey Lee commented on SPARK-31694:
Hi [~younggyuchun]
Thank you for your attention
Basically, a
[
https://issues.apache.org/jira/browse/SPARK-31640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105885#comment-17105885
]
Jackey Lee commented on SPARK-31640:
Hi [~younggyuchun] [~brkyvz], I have add this issue on Hive
[
https://issues.apache.org/jira/browse/SPARK-31694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-31694:
---
Summary: Add SupportsPartitions Catalog APIs on DataSourceV2 (was: Add
SupportsPartitions on
Jackey Lee created SPARK-31694:
--
Summary: Add SupportsPartitions on DataSourceV2
Key: SPARK-31694
URL: https://issues.apache.org/jira/browse/SPARK-31694
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-31438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-31438:
---
Description:
In Spark, we need do some hook after job cleaned, such as cleaning hive
external
[
https://issues.apache.org/jira/browse/SPARK-31438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-31438:
---
Description:
In Spark, we need do some hook, such as cleaning hive external temporary paths,
after
Jackey Lee created SPARK-31438:
--
Summary: Support JobCleaned Status in SparkListener
Key: SPARK-31438
URL: https://issues.apache.org/jira/browse/SPARK-31438
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-31346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-31346:
---
Description: In InsertIntoHiveTable and InsertIntoHiveDirCommand, we use
deleteExternalTmpPath to
Jackey Lee created SPARK-31346:
--
Summary: Add new configuration to verify temporary directory
cleaned
Key: SPARK-31346
URL: https://issues.apache.org/jira/browse/SPARK-31346
Project: Spark
Jackey Lee created SPARK-31241:
--
Summary: Support Hive on DataSourceV2
Key: SPARK-31241
URL: https://issues.apache.org/jira/browse/SPARK-31241
Project: Spark
Issue Type: Improvement
Jackey Lee created SPARK-31142:
--
Summary: Remove useless conf set in pyspark context
Key: SPARK-31142
URL: https://issues.apache.org/jira/browse/SPARK-31142
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-31100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-31100:
---
Description: We should check if the namespace exists while calling "use
namespace", and throw
Jackey Lee created SPARK-31100:
--
Summary: Detect namespace existence when setting namespace
Key: SPARK-31100
URL: https://issues.apache.org/jira/browse/SPARK-31100
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-30868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040868#comment-17040868
]
Jackey Lee commented on SPARK-30868:
Can't agree more. It is up to the user to choose how to handle
[
https://issues.apache.org/jira/browse/SPARK-30868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040822#comment-17040822
]
Jackey Lee commented on SPARK-30868:
So your opinion is to return normal results even if the
[
https://issues.apache.org/jira/browse/SPARK-30868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-30868:
---
Description:
At present, HiveClientImpl.runHive will not throw an exception when it runs
[
https://issues.apache.org/jira/browse/SPARK-30868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040557#comment-17040557
]
Jackey Lee commented on SPARK-30868:
[~Ankitraj] No, I think we should just throw Exception once
Jackey Lee created SPARK-30868:
--
Summary: Throw Exception if runHive(sql) failed
Key: SPARK-30868
URL: https://issues.apache.org/jira/browse/SPARK-30868
Project: Spark
Issue Type: Bug
Jackey Lee created SPARK-30514:
--
Summary: add ENV_PYSPARK_MAJOR_PYTHON_VERSION support for
JavaMainAppResource
Key: SPARK-30514
URL: https://issues.apache.org/jira/browse/SPARK-30514
Project: Spark
Jackey Lee created SPARK-30513:
--
Summary: Question about spark on k8s
Key: SPARK-30513
URL: https://issues.apache.org/jira/browse/SPARK-30513
Project: Spark
Issue Type: Question
Jackey Lee created SPARK-30036:
--
Summary: REPARTITION hint dose not work with order by
Key: SPARK-30036
URL: https://issues.apache.org/jira/browse/SPARK-30036
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-29771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-29771:
---
Comment: was deleted
(was: This patch is mainly used in the scenario where the executor started
[
https://issues.apache.org/jira/browse/SPARK-29771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-29771:
---
Description:
ExecutorPodsAllocator does not limit the number of executor errors or
deletions,
[
https://issues.apache.org/jira/browse/SPARK-29771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968933#comment-16968933
]
Jackey Lee edited comment on SPARK-29771 at 11/7/19 5:12 AM:
-
This patch is
[
https://issues.apache.org/jira/browse/SPARK-29771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968933#comment-16968933
]
Jackey Lee commented on SPARK-29771:
This patch is mainly used in the scenario where the executor
Jackey Lee created SPARK-29771:
--
Summary: Limit executor max failures before failing the application
Key: SPARK-29771
URL: https://issues.apache.org/jira/browse/SPARK-29771
Project: Spark
Issue
Jackey Lee created SPARK-29444:
--
Summary: Add configuration to support JacksonGenrator to keep
fields with null values
Key: SPARK-29444
URL: https://issues.apache.org/jira/browse/SPARK-29444
Project:
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804544#comment-16804544
]
Jackey Lee commented on SPARK-24630:
[~KevinZwx]
More discussions are in this doc.
[
https://issues.apache.org/jira/browse/SPARK-26912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-26912:
---
Summary: Allow setting permission for event_log (was: Allow set permission
for event_log)
> Allow
Jackey Lee created SPARK-26912:
--
Summary: Allow set permission for event_log
Key: SPARK-26912
URL: https://issues.apache.org/jira/browse/SPARK-26912
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730003#comment-16730003
]
Jackey Lee commented on SPARK-24630:
[~jackylk]
I have finished a detailed design doc, we can talk
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-24630:
---
Attachment: SQLStreaming SPIP V2.pdf
> SPIP: Support SQLStreaming in Spark
>
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-24630:
---
Attachment: (was: SQLStreaming SPIP.pdf)
> SPIP: Support SQLStreaming in Spark
>
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724011#comment-16724011
]
Jackey Lee commented on SPARK-24630:
[~jackylk] Sorry for the late reply.
I haven't considered
Jackey Lee created SPARK-26394:
--
Summary: Annotation error for Utils.timeStringAsMs
Key: SPARK-26394
URL: https://issues.apache.org/jira/browse/SPARK-26394
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-25937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676374#comment-16676374
]
Jackey Lee commented on SPARK-25937:
Not exactly the same thing.
The current implementation is to
[
https://issues.apache.org/jira/browse/SPARK-25937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674981#comment-16674981
]
Jackey Lee commented on SPARK-25937:
[~kabhwan]
Good point. It is exactly how I want to solve this
[
https://issues.apache.org/jira/browse/SPARK-25937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-25937:
---
Description:
Kafka Source & Sink is widely used in Spark and has the highest frequency
in
Jackey Lee created SPARK-25937:
--
Summary: Support user-defined schema in Kafka Source & Sink
Key: SPARK-25937
URL: https://issues.apache.org/jira/browse/SPARK-25937
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664529#comment-16664529
]
Jackey Lee commented on SPARK-24630:
[~shijinkui] [~kabhwan] [~uncleGen]
I have removed the stream
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651061#comment-16651061
]
Jackey Lee commented on SPARK-24630:
[~shijinkui]
Without stream keyword means SQLStreaming cannot
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641516#comment-16641516
]
Jackey Lee edited comment on SPARK-24630 at 10/8/18 8:36 AM:
-
[~kabhwan]
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641516#comment-16641516
]
Jackey Lee commented on SPARK-24630:
[~kabhwan]
The DDL in SQLStreaming has nothing change with
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641258#comment-16641258
]
Jackey Lee commented on SPARK-24630:
SQLStreaming is another interfaces for StructStreaming. Those,
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566954#comment-16566954
]
Jackey Lee commented on SPARK-24630:
[~uncleGen] Are you willing to assist in the code review? I can
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566782#comment-16566782
]
Jackey Lee edited comment on SPARK-24630 at 8/2/18 1:47 PM:
SQLStreaming is
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566782#comment-16566782
]
Jackey Lee commented on SPARK-24630:
SQLStreaming is actually based on StructStreaming, thus, most
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16561798#comment-16561798
]
Jackey Lee edited comment on SPARK-24630 at 7/30/18 11:38 AM:
--
For Stream
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16561798#comment-16561798
]
Jackey Lee commented on SPARK-24630:
For Stream Table DDL，we have a better way to deal with(such as
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554210#comment-16554210
]
Jackey Lee commented on SPARK-24630:
In SQLStreaming, we also support standard SQL as batch queries,
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526264#comment-16526264
]
Jackey Lee commented on SPARK-24630:
Main Goal:
* SQL API for StructStreaming
Benefits:
* Users,
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-24630:
---
Attachment: SQLStreaming SPIP.pdf
> SPIP: Support SQLStreaming in Spark
>
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackey Lee updated SPARK-24630:
---
Summary: SPIP: Support SQLStreaming in Spark (was: Support SQLStreaming in
Spark)
> SPIP: Support
Jackey Lee created SPARK-24630:
--
Summary: Support SQLStreaming in Spark
Key: SPARK-24630
URL: https://issues.apache.org/jira/browse/SPARK-24630
Project: Spark
Issue Type: Improvement
90 matches
Mail list logo
