[jira] [Commented] (FLINK-20762) Remove unused checkpointStorage path after job finished

2020-12-24 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254710#comment-17254710 ] Yun Tang commented on FLINK-20762: -- [~Jiangang] I think you could read some existing in

[jira] [Comment Edited] (FLINK-20762) Remove unused checkpointStorage path after job finished

2020-12-24 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254710#comment-17254710 ] Yun Tang edited comment on FLINK-20762 at 12/25/20, 2:46 AM: -

[jira] [Commented] (FLINK-20653) SharedState should be registered after PendingCheckpoint storeCheckpoint

2020-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254377#comment-17254377 ] Yun Tang commented on FLINK-20653: -- BTW, [~Vow] FLINK-11937 is not merged yet, did you

[jira] [Reopened] (FLINK-20653) SharedState should be registered after PendingCheckpoint storeCheckpoint

2020-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reopened FLINK-20653: -- > SharedState should be registered after PendingCheckpoint storeCheckpoint > -

[jira] [Closed] (FLINK-20653) SharedState should be registered after PendingCheckpoint storeCheckpoint

2020-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-20653. Resolution: Not A Bug > SharedState should be registered after PendingCheckpoint storeCheckpoint > ---

[jira] [Commented] (FLINK-20653) SharedState should be registered after PendingCheckpoint storeCheckpoint

2020-12-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253970#comment-17253970 ] Yun Tang commented on FLINK-20653: -- Though the code of checkpoint store changed much re

[jira] [Updated] (FLINK-20675) Asynchronous checkpoint failure would not fail the job anymore

2020-12-22 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-20675: - Fix Version/s: (was: 1.12.1) 1.12.2 > Asynchronous checkpoint failure would not f

[jira] [Comment Edited] (FLINK-20653) SharedState should be registered after PendingCheckpoint storeCheckpoint

2020-12-22 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253859#comment-17253859 ] Yun Tang edited comment on FLINK-20653 at 12/23/20, 3:15 AM: -

[jira] [Commented] (FLINK-20653) SharedState should be registered after PendingCheckpoint storeCheckpoint

2020-12-22 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253859#comment-17253859 ] Yun Tang commented on FLINK-20653: -- If checkpoint {{chk-23}} registers successfully but

[jira] [Commented] (FLINK-20684) Add compression type option in RocksDBConfigurableOptions

2020-12-21 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17252832#comment-17252832 ] Yun Tang commented on FLINK-20684: -- Before we decide to add this option in \{{RocksDBCo

[jira] [Updated] (FLINK-20675) Asynchronous checkpoint failure would not fail the job anymore

2020-12-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-20675: - Description: After FLINK-12364, no mater how many times of asynchronous part of checkpoint on task fail

[jira] [Commented] (FLINK-20675) Asynchronous checkpoint failure would not fail the job anymore

2020-12-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17251834#comment-17251834 ] Yun Tang commented on FLINK-20675: -- cc [~pnowojski] > Asynchronous checkpoint failure

[jira] [Created] (FLINK-20675) Asynchronous checkpoint failure would not fail the job anymore

2020-12-18 Thread Yun Tang (Jira)
Yun Tang created FLINK-20675: Summary: Asynchronous checkpoint failure would not fail the job anymore Key: FLINK-20675 URL: https://issues.apache.org/jira/browse/FLINK-20675 Project: Flink Issue

[jira] [Commented] (FLINK-20672) CheckpointAborted RPC failure can fail JM

2020-12-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17251694#comment-17251694 ] Yun Tang commented on FLINK-20672: -- I think you're right and this might happen in extra

[jira] [Commented] (FLINK-20672) CheckpointAborted RPC failure can fail JM

2020-12-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17251633#comment-17251633 ] Yun Tang commented on FLINK-20672: -- Is this also true for notification checkpoint compl

[jira] [Resolved] (FLINK-19013) Log start/end of state restoration

2020-12-17 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-19013. -- Fix Version/s: 1.12.1 Resolution: Fixed > Log start/end of state restoration >

[jira] [Commented] (FLINK-19013) Log start/end of state restoration

2020-12-17 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17251083#comment-17251083 ] Yun Tang commented on FLINK-19013: -- Merged master: f4a0f78ac153efa5995028f90df2438ddd6

[jira] [Commented] (FLINK-20653) SharedState should be registered after PendingCheckpoint storeCheckpoint

2020-12-17 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17251060#comment-17251060 ] Yun Tang commented on FLINK-20653: -- [~Vow] What do you mean " the deleted incremental s

[jira] [Assigned] (FLINK-19013) Log start/end of state restoration

2020-12-15 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-19013: Assignee: Yun Tang > Log start/end of state restoration > -- > >

[jira] [Resolved] (FLINK-20209) Add missing checkpoint configuration to Flink UI

2020-12-14 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-20209. -- Resolution: Fixed > Add missing checkpoint configuration to Flink UI > ---

[jira] [Updated] (FLINK-20209) Add missing checkpoint configuration to Flink UI

2020-12-14 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-20209: - Fix Version/s: 1.12.1 1.13.0 > Add missing checkpoint configuration to Flink UI > ---

[jira] [Commented] (FLINK-20209) Add missing checkpoint configuration to Flink UI

2020-12-14 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248951#comment-17248951 ] Yun Tang commented on FLINK-20209: -- Merged master: 59809fc3afc226530741e736504f032c1775

[jira] [Commented] (FLINK-20581) java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.util.JavaGcCleanerWrapper

2020-12-13 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248723#comment-17248723 ] Yun Tang commented on FLINK-20581: -- [~Lonie] Have you ever checked whether class {{org

[jira] [Commented] (FLINK-19462) Checkpoint statistics for unfinished task snapshots

2020-12-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247728#comment-17247728 ] Yun Tang commented on FLINK-19462: -- [~roman_khachatryan] I cannot view your documentati

[jira] [Resolved] (FLINK-20554) The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly displayed on the Overview page of the UI

2020-12-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-20554. -- Resolution: Fixed > The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly > di

[jira] [Commented] (FLINK-20554) The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly displayed on the Overview page of the UI

2020-12-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247708#comment-17247708 ] Yun Tang commented on FLINK-20554: -- Merged master: 91e81eee9b3096580f6ff830b6b6401cdfd5

[jira] [Commented] (FLINK-19911) add read buffer for input stream

2020-12-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247209#comment-17247209 ] Yun Tang commented on FLINK-19911: -- Since previous PR has been closed after more guys i

[jira] [Commented] (FLINK-20376) Error in restoring checkpoint/savepoint when Flink is upgraded from 1.9 to 1.11.2

2020-12-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247143#comment-17247143 ] Yun Tang commented on FLINK-20376: -- [~Partha Mishra] Could you share the {{_metadata}}

[jira] [Updated] (FLINK-20554) The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly displayed on the Overview page of the UI

2020-12-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-20554: - Fix Version/s: 1.12.1 1.11.3 > The Checkpointed Data Size of the Latest Completed Che

[jira] [Commented] (FLINK-20554) The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly displayed on the Overview page of the UI

2020-12-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247073#comment-17247073 ] Yun Tang commented on FLINK-20554: -- I think you're right and I have assigned this ticke

[jira] [Assigned] (FLINK-20554) The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly displayed on the Overview page of the UI

2020-12-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-20554: Assignee: ming li (was: mehdi amini) > The Checkpointed Data Size of the Latest Completed Checkp

[jira] [Assigned] (FLINK-20554) The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly displayed on the Overview page of the UI

2020-12-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-20554: Assignee: mehdi amini > The Checkpointed Data Size of the Latest Completed Checkpoint is incorrec

[jira] [Commented] (FLINK-19710) Fix performance regression to rebase FRocksDB with higher version RocksDB

2020-12-06 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244945#comment-17244945 ] Yun Tang commented on FLINK-19710: -- Yes, I also heard voice to support ARM platform. As

[jira] [Commented] (FLINK-20480) Methods from CheckpointedFunction interface in FlinkKafkaConsumerBase are set to final

2020-12-06 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244929#comment-17244929 ] Yun Tang commented on FLINK-20480: -- I think setting {{CheckpointedFunction}} interfaces

[jira] [Commented] (FLINK-19710) Fix performance regression to rebase FRocksDB with higher version RocksDB

2020-12-06 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244922#comment-17244922 ] Yun Tang commented on FLINK-19710: -- [~jherico] Thanks for your care and sorry for not u

[jira] [Created] (FLINK-20420) ES6 ElasticsearchSinkITCase failed due to no output for 900 seconds

2020-11-30 Thread Yun Tang (Jira)
Yun Tang created FLINK-20420: Summary: ES6 ElasticsearchSinkITCase failed due to no output for 900 seconds Key: FLINK-20420 URL: https://issues.apache.org/jira/browse/FLINK-20420 Project: Flink

[jira] [Commented] (FLINK-20376) Error in restoring checkpoint/savepoint when Flink is upgraded from 1.9 to 1.11.2

2020-11-26 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17239490#comment-17239490 ] Yun Tang commented on FLINK-20376: -- I think the reported message has given enough hint:

[jira] [Resolved] (FLINK-19998) Invalid Links in docs

2020-11-25 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-19998. -- Resolution: Fixed > Invalid Links in docs > - > > Key: FLINK-19998

[jira] [Commented] (FLINK-19998) Invalid Links in docs

2020-11-25 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17239033#comment-17239033 ] Yun Tang commented on FLINK-19998: -- merged in master c008907d2a629449c8d0ad9725d13b0604

[jira] [Commented] (FLINK-20298) Replace usage of {{site.baseurl}} in flink documentation

2020-11-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237799#comment-17237799 ] Yun Tang commented on FLINK-20298: -- What's the relationship with FLINK-19998? > Replac

[jira] [Closed] (FLINK-19293) RocksDB last_checkpoint.state_size grows endlessly until savepoint/restore

2020-11-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-19293. Resolution: Information Provided > RocksDB last_checkpoint.state_size grows endlessly until savepoint/rest

[jira] [Commented] (FLINK-19293) RocksDB last_checkpoint.state_size grows endlessly until savepoint/restore

2020-11-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237411#comment-17237411 ] Yun Tang commented on FLINK-19293: -- Glad to know you have figured out the root cause, a

[jira] [Commented] (FLINK-19125) Avoid memory fragmentation when running flink docker image

2020-11-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237356#comment-17237356 ] Yun Tang commented on FLINK-19125: -- [~NicoK] I think we could add documentation at  [ht

[jira] [Created] (FLINK-20288) Correct documentation about savepoint self-contained

2020-11-22 Thread Yun Tang (Jira)
Yun Tang created FLINK-20288: Summary: Correct documentation about savepoint self-contained Key: FLINK-20288 URL: https://issues.apache.org/jira/browse/FLINK-20288 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-20287) Add documentation of how to switch memory allocator in Flink docker image

2020-11-22 Thread Yun Tang (Jira)
Yun Tang created FLINK-20287: Summary: Add documentation of how to switch memory allocator in Flink docker image Key: FLINK-20287 URL: https://issues.apache.org/jira/browse/FLINK-20287 Project: Flink

[jira] [Commented] (FLINK-15507) Activate local recovery for RocksDB backends by default

2020-11-22 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237093#comment-17237093 ] Yun Tang commented on FLINK-15507: -- Current local recovery mechanism might not be so me

[jira] [Commented] (FLINK-17170) Cannot stop streaming job with savepoint which uses kinesis consumer

2020-11-22 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237081#comment-17237081 ] Yun Tang commented on FLINK-17170: -- I think my previous suggestion of "Check whether th

[jira] [Commented] (FLINK-19125) Avoid memory fragmentation when running flink docker image

2020-11-22 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237064#comment-17237064 ] Yun Tang commented on FLINK-19125: -- [~liyu], thanks for your reminds, already updated t

[jira] [Updated] (FLINK-19125) Avoid memory fragmentation when running flink docker image

2020-11-22 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-19125: - Release Note: Adopt Jemalloc as default memory allocator in official docker image's docker-entrypoint.sh

[jira] [Commented] (FLINK-18712) Flink RocksDB statebackend memory leak issue

2020-11-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234512#comment-17234512 ] Yun Tang commented on FLINK-18712: -- Sorry for late reply, already close this ticket and

[jira] [Closed] (FLINK-18712) Flink RocksDB statebackend memory leak issue

2020-11-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-18712. Resolution: Information Provided > Flink RocksDB statebackend memory leak issue > ---

[jira] [Assigned] (FLINK-20209) Add missing checkpoint configuration to Flink UI

2020-11-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-20209: Assignee: Peidian Li > Add missing checkpoint configuration to Flink UI > ---

[jira] [Commented] (FLINK-20209) Add missing checkpoint configuration to Flink UI

2020-11-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234435#comment-17234435 ] Yun Tang commented on FLINK-20209: -- The information of 'unalignedCheckpoints' has been

[jira] [Updated] (FLINK-10954) Hardlink from files of previous local stored state might cross devices

2020-11-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-10954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-10954: - Fix Version/s: (was: 1.12.0) 1.13.0 > Hardlink from files of previous local store

[jira] [Commented] (FLINK-18004) Update checkpoint UI related pictures in documentation

2020-11-06 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17227506#comment-17227506 ] Yun Tang commented on FLINK-18004: -- Merged in master with 707d824cac77f41fa8ee9072668bc

[jira] [Resolved] (FLINK-18004) Update checkpoint UI related pictures in documentation

2020-11-06 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-18004. -- Assignee: Yun Tang Resolution: Fixed > Update checkpoint UI related pictures in documentation >

[jira] [Commented] (FLINK-19911) Read checkpoint stream with buffer to speedup restore

2020-11-04 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17226479#comment-17226479 ] Yun Tang commented on FLINK-19911: -- [~fanrui] Sure, already assigned this ticket to you

[jira] [Assigned] (FLINK-19911) Read checkpoint stream with buffer to speedup restore

2020-11-04 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-19911: Assignee: fanrui > Read checkpoint stream with buffer to speedup restore > --

[jira] [Commented] (FLINK-13598) frocksdb doesn't have arm release

2020-10-28 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-13598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17222061#comment-17222061 ] Yun Tang commented on FLINK-13598: -- [~wangxiyuan] we're planing to upgrade RocksDB vers

[jira] [Resolved] (FLINK-19594) SubTasks start index don't unified and may confuse users

2020-10-27 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-19594. -- Resolution: Fixed > SubTasks start index don't unified and may confuse users > ---

[jira] [Commented] (FLINK-19594) SubTasks start index don't unified and may confuse users

2020-10-27 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221192#comment-17221192 ] Yun Tang commented on FLINK-19594: -- Merged in master branch with commit id 9b1ebde13da7

[jira] [Resolved] (FLINK-19553) The format of checkpoint Completion Time and Failure Time should be changed from HH:mm:ss to yyyy-MM-dd HH:mm:ss

2020-10-26 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang resolved FLINK-19553. -- Resolution: Fixed > The format of checkpoint Completion Time and Failure Time should be changed > fro

[jira] [Commented] (FLINK-19553) The format of checkpoint Completion Time and Failure Time should be changed from HH:mm:ss to yyyy-MM-dd HH:mm:ss

2020-10-26 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221098#comment-17221098 ] Yun Tang commented on FLINK-19553: -- Merged into master with commit 8e6b04a103771f4f5065

[jira] [Assigned] (FLINK-19594) SubTasks start index don't unified and may confuse users

2020-10-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-19594: Assignee: zlzhang0122 (was: Zoltan Ivanfi) > SubTasks start index don't unified and may confuse

[jira] [Assigned] (FLINK-19594) SubTasks start index don't unified and may confuse users

2020-10-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-19594: Assignee: Zoltan Ivanfi > SubTasks start index don't unified and may confuse users >

[jira] [Commented] (FLINK-19619) Test failed in Azure For EmulatedPubSubSourceTest

2020-10-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216576#comment-17216576 ] Yun Tang commented on FLINK-19619: -- Another instance: https://dev.azure.com/apache-fli

[jira] [Commented] (FLINK-14482) Bump up rocksdb version

2020-10-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216572#comment-17216572 ] Yun Tang commented on FLINK-14482: -- Thanks for [~liyu] 's reminder, I have created the

[jira] [Updated] (FLINK-19710) Avoid performance regression introduced by thread-local keyword of FRocksDB

2020-10-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-19710: - Component/s: Runtime / State Backends > Avoid performance regression introduced by thread-local keyword

[jira] [Assigned] (FLINK-6984) Do not recover CompletedCheckpointStore on every restore

2020-10-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-6984: --- Assignee: Jiayi Liao (was: Yun Tang) > Do not recover CompletedCheckpointStore on every restore > -

[jira] [Created] (FLINK-19710) Avoid performance regression introduced by thread-local keyword of FRocksDB

2020-10-19 Thread Yun Tang (Jira)
Yun Tang created FLINK-19710: Summary: Avoid performance regression introduced by thread-local keyword of FRocksDB Key: FLINK-19710 URL: https://issues.apache.org/jira/browse/FLINK-19710 Project: Flink

[jira] [Commented] (FLINK-19596) Do not recover CompletedCheckpointStore on each failover

2020-10-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216530#comment-17216530 ] Yun Tang commented on FLINK-19596: -- [~wind_ljy] this issue duplicates with FLINK-6984 a

[jira] [Closed] (FLINK-18373) Drop performance unit tests in RocksDB state backend module

2020-10-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-18373. Resolution: Fixed > Drop performance unit tests in RocksDB state backend module >

[jira] [Commented] (FLINK-18373) Drop performance unit tests in RocksDB state backend module

2020-10-18 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216440#comment-17216440 ] Yun Tang commented on FLINK-18373: -- Merged in master a91e3c7e58bbe8d86e8e7ad997609dc2d

[jira] [Commented] (FLINK-18712) Flink RocksDB statebackend memory leak issue

2020-10-16 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17215313#comment-17215313 ] Yun Tang commented on FLINK-18712: -- [~Caesar] Jemalloc could help resolve memory fragme

[jira] [Commented] (FLINK-19642) RocksDBListStatePerformanceTest.testRocksDbListStateAPIs is instable

2020-10-14 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213908#comment-17213908 ] Yun Tang commented on FLINK-19642: -- [~dian.fu] we have already agree to drop those test

[jira] [Commented] (FLINK-19594) SubTasks start index don't unified and may confuse users

2020-10-13 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213040#comment-17213040 ] Yun Tang commented on FLINK-19594: -- [~chesnay] As I wrote in previous comment, the inco

[jira] [Updated] (FLINK-19553) The format of checkpoint Completion Time and Failure Time should be changed from HH:mm:ss to yyyy-MM-dd HH:mm:ss

2020-10-13 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-19553: - Fix Version/s: 1.12.0 > The format of checkpoint Completion Time and Failure Time should be changed > f

[jira] [Commented] (FLINK-19553) The format of checkpoint Completion Time and Failure Time should be changed from HH:mm:ss to yyyy-MM-dd HH:mm:ss

2020-10-13 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212897#comment-17212897 ] Yun Tang commented on FLINK-19553: -- [~liufangliang] Already assign to you, please go ah

[jira] [Assigned] (FLINK-19553) The format of checkpoint Completion Time and Failure Time should be changed from HH:mm:ss to yyyy-MM-dd HH:mm:ss

2020-10-13 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-19553: Assignee: liufangliang > The format of checkpoint Completion Time and Failure Time should be chan

[jira] [Assigned] (FLINK-19125) Avoid memory fragmentation when running flink docker image

2020-10-13 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-19125: Assignee: Yun Tang > Avoid memory fragmentation when running flink docker image > ---

[jira] [Commented] (FLINK-19594) SubTasks start index don't unified and may confuse users

2020-10-12 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212823#comment-17212823 ] Yun Tang commented on FLINK-19594: -- This is caused by [job-checkpoints-subtask.compone

[jira] [Commented] (FLINK-19481) Add support for a flink native GCS FileSystem

2020-10-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211631#comment-17211631 ] Yun Tang commented on FLINK-19481: -- [~jgrier] Sounds reasonable to me. [~rmetzger] wha

[jira] [Commented] (FLINK-19553) The format of checkpoint Completion Time and Failure Time should be changed from HH:mm:ss to yyyy-MM-dd HH:mm:ss

2020-10-10 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211627#comment-17211627 ] Yun Tang commented on FLINK-19553: -- I think this deserves to fix, [~liufangliang] would

[jira] [Commented] (FLINK-19481) Add support for a flink native GCS FileSystem

2020-10-09 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211228#comment-17211228 ] Yun Tang commented on FLINK-19481: -- If so, why we still must need to support native GCS

[jira] [Commented] (FLINK-19481) Add support for a flink native GCS FileSystem

2020-10-03 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1720#comment-1720 ] Yun Tang commented on FLINK-19481: -- [~baugarten], from my point of view, we could still

[jira] [Commented] (FLINK-19382) Introducing ReplayableSourceStateBackend

2020-09-25 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17201968#comment-17201968 ] Yun Tang commented on FLINK-19382: -- Thanks for bringing this discussion. If you ever re

[jira] [Commented] (FLINK-18263) Allow external checkpoints to be persisted even when the job is in "Finished" state.

2020-09-25 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17201960#comment-17201960 ] Yun Tang commented on FLINK-18263: -- If so, [~markcho] would you consider to continue mo

[jira] [Commented] (FLINK-19283) Allow subclasses to override/extend FlinkKafkaConsumerBase checkpoint methods

2020-09-17 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198091#comment-17198091 ] Yun Tang commented on FLINK-19283: -- [~adriank], I am not familiar with the background o

[jira] [Commented] (FLINK-18263) Allow external checkpoints to be persisted even when the job is in "Finished" state.

2020-09-15 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195961#comment-17195961 ] Yun Tang commented on FLINK-18263: -- After state processor API introduced and Flink-1494

[jira] [Commented] (FLINK-18335) NotifyCheckpointAbortedITCase.testNotifyCheckpointAborted time outs

2020-09-07 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191940#comment-17191940 ] Yun Tang commented on FLINK-18335: -- [~rmetzger], I am preparing a PR to implement what

[jira] [Commented] (FLINK-19125) Avoid memory fragmentation when running flink docker image

2020-09-07 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191938#comment-17191938 ] Yun Tang commented on FLINK-19125: -- [~xiaotao183], current Flink's image is based on op

[jira] [Commented] (FLINK-19125) Avoid memory fragmentation when running flink docker image

2020-09-06 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191433#comment-17191433 ] Yun Tang commented on FLINK-19125: -- [~fly_in_gis], as discussed offline, jemalloc and J

[jira] [Created] (FLINK-19125) Avoid memory fragmentation when running flink docker image

2020-09-02 Thread Yun Tang (Jira)
Yun Tang created FLINK-19125: Summary: Avoid memory fragmentation when running flink docker image Key: FLINK-19125 URL: https://issues.apache.org/jira/browse/FLINK-19125 Project: Flink Issue Type

[jira] [Assigned] (FLINK-18712) Flink RocksDB statebackend memory leak issue

2020-09-02 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-18712: Assignee: Yun Tang > Flink RocksDB statebackend memory leak issue >

[jira] [Commented] (FLINK-18712) Flink RocksDB statebackend memory leak issue

2020-09-02 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17189138#comment-17189138 ] Yun Tang commented on FLINK-18712: -- I use [k8s session cluster|https://ci.apache.org/p

[jira] [Commented] (FLINK-18712) Flink RocksDB statebackend memory leak issue

2020-08-28 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17186516#comment-17186516 ] Yun Tang commented on FLINK-18712: -- Some updates: I have contacted with [~lio_sy] offli

[jira] [Commented] (FLINK-19060) Checkpoint not triggered when use broadcast stream

2020-08-27 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185734#comment-17185734 ] Yun Tang commented on FLINK-19060: -- Did you check the logs of jobmanager to see why no

[jira] [Commented] (FLINK-13598) frocksdb doesn't have arm release

2020-08-19 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-13598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17180733#comment-17180733 ] Yun Tang commented on FLINK-13598: -- [~wangxiyuan], first of all, we should bump the ver

[jira] [Commented] (FLINK-7289) Memory allocation of RocksDB can be problematic in container environments

2020-08-07 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-7289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173320#comment-17173320 ] Yun Tang commented on FLINK-7289: - [~itsharshonly], you could call [DBOptions#setUseDire

[jira] [Commented] (FLINK-18712) Flink RocksDB statebackend memory leak issue

2020-07-26 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17165194#comment-17165194 ] Yun Tang commented on FLINK-18712: -- [~lio_sy], do you have a simple job to reproduce th

<    9   10   11   12   13   14   15   16   17   18   >