dzcxzl created SPARK-48218:
--
Summary: TransportClientFactory.createClient may NPE cause
FetchFailedException
Key: SPARK-48218
URL: https://issues.apache.org/jira/browse/SPARK-48218
Project: Spark
dzcxzl created SPARK-48070:
--
Summary: Support AdaptiveQueryExecSuite to skip check results
Key: SPARK-48070
URL: https://issues.apache.org/jira/browse/SPARK-48070
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-48037:
---
Affects Version/s: 3.3.0
(was: 3.1.0)
(was: 3.0.1)
>
dzcxzl created SPARK-48037:
--
Summary: SortShuffleWriter lacks shuffle write related metrics
resulting in potentially inaccurate data
Key: SPARK-48037
URL: https://issues.apache.org/jira/browse/SPARK-48037
dzcxzl created SPARK-47799:
--
Summary: Preserve parameter information when using SBT package jar
Key: SPARK-47799
URL: https://issues.apache.org/jira/browse/SPARK-47799
Project: Spark
Issue Type:
dzcxzl created SPARK-47456:
--
Summary: Support ORC Brotli codec
Key: SPARK-47456
URL: https://issues.apache.org/jira/browse/SPARK-47456
Project: Spark
Issue Type: Improvement
Components:
[
https://issues.apache.org/jira/browse/SPARK-46943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-46943:
---
Description:
When we enable `spark.sql.join.preferSortMergeJoin=false`, we may get the
following error.
[
https://issues.apache.org/jira/browse/SPARK-46943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-46943:
---
Description:
When we enable `spark.sql.join.preferSortMergeJoin=false`, we may get the
following error.
dzcxzl created SPARK-46943:
--
Summary: Support for configuring ShuffledHashJoin plan size
Threshold
Key: SPARK-46943
URL: https://issues.apache.org/jira/browse/SPARK-46943
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-33458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769889#comment-17769889
]
dzcxzl commented on SPARK-33458:
After [HIVE-22900|https://issues.apache.org/jira/browse/HIVE-22900]
dzcxzl created SPARK-44650:
--
Summary: `spark.executor.defaultJavaOptions` Check illegal java
options
Key: SPARK-44650
URL: https://issues.apache.org/jira/browse/SPARK-44650
Project: Spark
Issue
dzcxzl created SPARK-44583:
--
Summary: `spark.*.io.connectionCreationTimeout` parameter
documentation
Key: SPARK-44583
URL: https://issues.apache.org/jira/browse/SPARK-44583
Project: Spark
Issue
dzcxzl created SPARK-44556:
--
Summary: Reuse `OrcTail` when enable vectorizedReader
Key: SPARK-44556
URL: https://issues.apache.org/jira/browse/SPARK-44556
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-44497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-44497:
---
Description: In SPARK-37831, the partition id is added in taskinfo, and the
task partition id cannot be
dzcxzl created SPARK-44497:
--
Summary: Show task partition id in Task table
Key: SPARK-44497
URL: https://issues.apache.org/jira/browse/SPARK-44497
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-44490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-44490:
---
Description: In
[SPARK-21809|https://issues.apache.org/jira/browse/SPARK-21809], we introduced
dzcxzl created SPARK-44490:
--
Summary: Remove TaskPagedTable in StagePage
Key: SPARK-44490
URL: https://issues.apache.org/jira/browse/SPARK-44490
Project: Spark
Issue Type: Improvement
dzcxzl created SPARK-44454:
--
Summary: HiveShim getTablesByType support fallback
Key: SPARK-44454
URL: https://issues.apache.org/jira/browse/SPARK-44454
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-44240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-44240:
---
Description:
{code:java}
set spark.sql.execution.topKSortFallbackThreshold=1;
SELECT min(id) FROM (
[
https://issues.apache.org/jira/browse/SPARK-44240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-44240:
---
Description:
{code:java}
set spark.sql.execution.topKSortFallbackThreshold=1;
SELECT min(id) FROM (
[
https://issues.apache.org/jira/browse/SPARK-44240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-44240:
---
Description:
{code:java}
set spark.sql.execution.topKSortFallbackThreshold=1;
SELECT min(id) FROM (
[
https://issues.apache.org/jira/browse/SPARK-44240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-44240:
---
Attachment: topKSortFallbackThresholdDesc.png
> Setting the topKSortFallbackThreshold value may lead to
[
https://issues.apache.org/jira/browse/SPARK-44240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-44240:
---
Description:
{code:java}
set spark.sql.execution.topKSortFallbackThreshold=1;
SELECT min(id) FROM (
[
https://issues.apache.org/jira/browse/SPARK-44240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-44240:
---
Description:
{code:java}
set spark.sql.execution.topKSortFallbackThreshold=1;
SELECT min(id) FROM (
[
https://issues.apache.org/jira/browse/SPARK-44240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-44240:
---
Attachment: topKSortFallbackThreshold.png
> Setting the topKSortFallbackThreshold value may lead to
[
https://issues.apache.org/jira/browse/SPARK-44240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-44240:
---
Description:
{code:java}
set spark.sql.execution.topKSortFallbackThreshold=1;
SELECT min(id) FROM (
dzcxzl created SPARK-44240:
--
Summary: Setting the topKSortFallbackThreshold value may lead to
inaccurate results
Key: SPARK-44240
URL: https://issues.apache.org/jira/browse/SPARK-44240
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-37605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl resolved SPARK-37605.
Resolution: Duplicate
> Support the configuration of the initial number of scan partitions when
>
[
https://issues.apache.org/jira/browse/SPARK-43301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-43301:
---
Summary: BlockStoreClient getHostLocalDirs RPC supports IOException retry
(was: BlockStoreClient
dzcxzl created SPARK-43301:
--
Summary: BlockStoreClient getHostLocalDirs RPC supports
IOexception retry
Key: SPARK-43301
URL: https://issues.apache.org/jira/browse/SPARK-43301
Project: Spark
Issue
dzcxzl created SPARK-42808:
--
Summary: Avoid getting availableProcessors every time in
MapOutputTrackerMaster#getStatistics
Key: SPARK-42808
URL: https://issues.apache.org/jira/browse/SPARK-42808
Project:
dzcxzl created SPARK-42807:
--
Summary: Apply custom log URL pattern for yarn-client AM log URL
in SHS
Key: SPARK-42807
URL: https://issues.apache.org/jira/browse/SPARK-42807
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-42366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-42366:
---
Summary: Log shuffle data corruption diagnose cause (was: Log output
shuffle data corruption diagnose
[
https://issues.apache.org/jira/browse/SPARK-42366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-42366:
---
Summary: Log output shuffle data corruption diagnose cause (was: Log
output shuffle data corruption
dzcxzl created SPARK-42366:
--
Summary: Log output shuffle data corruption diagnose causes
Key: SPARK-42366
URL: https://issues.apache.org/jira/browse/SPARK-42366
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-35744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654965#comment-17654965
]
dzcxzl commented on SPARK-35744:
This problem should be solved by upgrading avro 1.11.0 version
dzcxzl created SPARK-41003:
--
Summary: BHJ LeftAnti does not update numOutputRows when codegen
is disabled
Key: SPARK-41003
URL: https://issues.apache.org/jira/browse/SPARK-41003
Project: Spark
dzcxzl created SPARK-40987:
--
Summary: Avoid creating a directory when deleting a block, causing
DAGScheduler to not work
Key: SPARK-40987
URL: https://issues.apache.org/jira/browse/SPARK-40987
Project:
dzcxzl created SPARK-40312:
--
Summary: Add missing configuration documentation in Spark History
Server
Key: SPARK-40312
URL: https://issues.apache.org/jira/browse/SPARK-40312
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-39830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17569385#comment-17569385
]
dzcxzl commented on SPARK-39830:
cc @[~dongjoon]
> Reading ORC table that requires type promotion may
[
https://issues.apache.org/jira/browse/SPARK-39830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-39830:
---
Description:
We can add a UT to test the scenario after the ORC-1205 release.
bin/spark-shell
dzcxzl created SPARK-39830:
--
Summary: Reading ORC table that requires type promotion may throw
AIOOBE
Key: SPARK-39830
URL: https://issues.apache.org/jira/browse/SPARK-39830
Project: Spark
Issue
Title: Message Title
dzcxzl created an
[
https://issues.apache.org/jira/browse/SPARK-39355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-39355:
---
Summary: Single column uses quoted to construct UnresolvedAttribute (was:
Avoid UnresolvedAttribute.apply
[
https://issues.apache.org/jira/browse/SPARK-39415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl resolved SPARK-39415.
Resolution: Duplicate
> Local mode supports HadoopDelegationTokenManager
>
[
https://issues.apache.org/jira/browse/SPARK-39415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-39415:
---
Summary: Local mode supports HadoopDelegationTokenManager (was: Local mode
supports
dzcxzl created SPARK-39415:
--
Summary: Local mode supports delegationTokenManager
Key: SPARK-39415
URL: https://issues.apache.org/jira/browse/SPARK-39415
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-39382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-39382:
---
Summary: UI show the duration of the failed task when the executor lost
(was: UI show the duartion of the
[
https://issues.apache.org/jira/browse/SPARK-39387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-39387:
---
Description:
HIVE-25190: Fix many small allocations in BytesColumnVector
{code:java}
Caused by:
dzcxzl created SPARK-39387:
--
Summary: Upgrade hive-storage-api to 2.7.3
Key: SPARK-39387
URL: https://issues.apache.org/jira/browse/SPARK-39387
Project: Spark
Issue Type: Improvement
dzcxzl created SPARK-39382:
--
Summary: UI show the duartion of the failed task when the executor
lost
Key: SPARK-39382
URL: https://issues.apache.org/jira/browse/SPARK-39382
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-39381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-39381:
---
Description: Now vectorized columar orc writer batch size is default 1024.
(was: Now vectorized columar
dzcxzl created SPARK-39381:
--
Summary: Make vectorized orc columar writer batch size configurable
Key: SPARK-39381
URL: https://issues.apache.org/jira/browse/SPARK-39381
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-39355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-39355:
---
Summary: Avoid UnresolvedAttribute.apply throwing ParseException (was:
UnresolvedAttribute should only use
dzcxzl created SPARK-39355:
--
Summary: UnresolvedAttribute should only use CatalystSqlParser if
name contains dot
Key: SPARK-39355
URL: https://issues.apache.org/jira/browse/SPARK-39355
Project: Spark
dzcxzl created SPARK-38979:
--
Summary: Improve error log readability in
OrcUtils.requestedColumnIds
Key: SPARK-38979
URL: https://issues.apache.org/jira/browse/SPARK-38979
Project: Spark
Issue
dzcxzl created SPARK-38951:
--
Summary: Aggregate aliases override field names in
ResolveAggregateFunctions
Key: SPARK-38951
URL: https://issues.apache.org/jira/browse/SPARK-38951
Project: Spark
dzcxzl created SPARK-38936:
--
Summary: Script transform feed thread should have name
Key: SPARK-38936
URL: https://issues.apache.org/jira/browse/SPARK-38936
Project: Spark
Issue Type: Improvement
dzcxzl created SPARK-37605:
--
Summary: Support the configuration of the initial number of scan
partitions when executing a take on a query
Key: SPARK-37605
URL: https://issues.apache.org/jira/browse/SPARK-37605
[
https://issues.apache.org/jira/browse/SPARK-37561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-37561:
---
Attachment: getDelegationToken_load_functions.png
> Avoid loading all functions when obtaining hive's
[
https://issues.apache.org/jira/browse/SPARK-37561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-37561:
---
Description:
At present, when obtaining the delegationToken of hive, all functions will be
loaded.
This is
dzcxzl created SPARK-37561:
--
Summary: Avoid loading all functions when obtaining hive's
DelegationToken
Key: SPARK-37561
URL: https://issues.apache.org/jira/browse/SPARK-37561
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-36799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-36799:
---
Summary: Pass queryExecution name in CLI (was: Pass queryExecution name in
CLI when only select query)
>
[
https://issues.apache.org/jira/browse/SPARK-37217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-37217:
---
Summary: The number of dynamic partitions should early check when writing
to external tables (was: Dynamic
dzcxzl created SPARK-37217:
--
Summary: Dynamic partitions should fail quickly when writing to
external tables to prevent data deletion
Key: SPARK-37217
URL: https://issues.apache.org/jira/browse/SPARK-37217
dzcxzl created SPARK-36799:
--
Summary: Pass queryExecution name in CLI when only select query
Key: SPARK-36799
URL: https://issues.apache.org/jira/browse/SPARK-36799
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-36616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407128#comment-17407128
]
dzcxzl commented on SPARK-36616:
You can use the JdbcConnectionProvider interface provided by
[
https://issues.apache.org/jira/browse/SPARK-36550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-36550:
---
Description:
Now when UDF reflection fails, InvocationTargetException is thrown, but it is
not a specific
dzcxzl created SPARK-36550:
--
Summary: Propagation cause when UDF reflection fails
Key: SPARK-36550
URL: https://issues.apache.org/jira/browse/SPARK-36550
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-36451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-36451:
---
Description:
Because SPARK-35863 Upgrade Ivy to 2.5.0, it supports skip searching the source
and doc pom,
dzcxzl created SPARK-36451:
--
Summary: Ivy skips looking for source and doc pom
Key: SPARK-36451
URL: https://issues.apache.org/jira/browse/SPARK-36451
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-35437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-35437:
---
Summary: Use expressions to filter Hive partitions at client side (was:
Hive partition filtering client
dzcxzl created SPARK-36390:
--
Summary: Replace SessionState.close with SessionState.detachSession
Key: SPARK-36390
URL: https://issues.apache.org/jira/browse/SPARK-36390
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-32467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368005#comment-17368005
]
dzcxzl edited comment on SPARK-32467 at 7/6/21, 1:16 PM:
-
YARN-3239. WebAppProxy
[
https://issues.apache.org/jira/browse/SPARK-34632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373410#comment-17373410
]
dzcxzl commented on SPARK-34632:
You can use the default Authenticator to get the username through ugi.
dzcxzl created SPARK-35913:
--
Summary: Create hive permanent function with owner name
Key: SPARK-35913
URL: https://issues.apache.org/jira/browse/SPARK-35913
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-32467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368005#comment-17368005
]
dzcxzl commented on SPARK-32467:
YARN-3239. WebAppProxy does not support a final tracking url which has
dzcxzl created SPARK-35437:
--
Summary: Hive partition filtering client optimization
Key: SPARK-35437
URL: https://issues.apache.org/jira/browse/SPARK-35437
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265724#comment-17265724
]
dzcxzl edited comment on SPARK-33790 at 1/15/21, 4:28 PM:
--
[
https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265724#comment-17265724
]
dzcxzl edited comment on SPARK-33790 at 1/15/21, 4:27 PM:
--
Thread stack when
[
https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265724#comment-17265724
]
dzcxzl edited comment on SPARK-33790 at 1/15/21, 4:26 PM:
--
Thread stack when
[
https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265724#comment-17265724
]
dzcxzl edited comment on SPARK-33790 at 1/15/21, 4:25 PM:
--
Thread stack when
[
https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265812#comment-17265812
]
dzcxzl commented on SPARK-33790:
ok, I opened a JIRA [SPARK-34125
[
https://issues.apache.org/jira/browse/SPARK-34125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-34125:
---
Description:
2.x version of history server
EventLoggingListener.codecMap is of type mutable.HashMap, which
[
https://issues.apache.org/jira/browse/SPARK-34125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-34125:
---
Description:
2.x version of history server
EventLoggingListener.codecMap is of type mutable.HashMap, which
[
https://issues.apache.org/jira/browse/SPARK-34125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-34125:
---
Description:
2.x version of history server
EventLoggingListener.codecMap is of type mutable.HashMap, which
[
https://issues.apache.org/jira/browse/SPARK-34125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-34125:
---
Attachment: jstack.png
> Make EventLoggingListener.codecMap thread-safe
>
dzcxzl created SPARK-34125:
--
Summary: Make EventLoggingListener.codecMap thread-safe
Key: SPARK-34125
URL: https://issues.apache.org/jira/browse/SPARK-34125
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-34125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-34125:
---
Attachment: top.png
> Make EventLoggingListener.codecMap thread-safe
>
[
https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265724#comment-17265724
]
dzcxzl commented on SPARK-33790:
Thread stack when not working
[
https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265691#comment-17265691
]
dzcxzl commented on SPARK-33790:
This is indeed a performance regression problem.
The following is my
dzcxzl created SPARK-33900:
--
Summary: Show shuffle read size / records correctly when only
remotebytesread is available
Key: SPARK-33900
URL: https://issues.apache.org/jira/browse/SPARK-33900
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-33790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-33790:
---
Description:
FsHistoryProvider#checkForLogs already has FileStatus when constructing
dzcxzl created SPARK-33790:
--
Summary: Reduce the rpc call of getFileStatus in
SingleFileEventLogFileReader
Key: SPARK-33790
URL: https://issues.apache.org/jira/browse/SPARK-33790
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-33753:
---
Description:
HadoopRDD uses soft-reference map to cache jobconf (rdd_id -> jobconf).
When the number of
[
https://issues.apache.org/jira/browse/SPARK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-33753:
---
Description:
HadoopRDD uses soft-reference map to cache jobconf (rdd_id -> jobconf).
When the number of
[
https://issues.apache.org/jira/browse/SPARK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-33753:
---
Attachment: jobconf.png
> Reduce the memory footprint and gc of the cache (hadoopJobMetadata)
>
[
https://issues.apache.org/jira/browse/SPARK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-33753:
---
Description:
HadoopRDD uses soft-reference map to cache jobconf (rdd_id -> jobconf).
When the number of
[
https://issues.apache.org/jira/browse/SPARK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-33753:
---
Attachment: fix_visual_gc.png
fix_job_finish_time.png
fix_gcutil.png
>
[
https://issues.apache.org/jira/browse/SPARK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dzcxzl updated SPARK-33753:
---
Attachment: current_gcutil.png
> Reduce the memory footprint and gc of the cache (hadoopJobMetadata)
>
1 - 100 of 154 matches
Mail list logo
