[
https://issues.apache.org/jira/browse/FLINK-21543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17294236#comment-17294236
]
Yun Tang commented on FLINK-21543:
----------------------------------
[~zhoujira86] First of all, do not use any third-party unofficial rocksDB jar
package. We did not ever promise that could work well.
Secondly, as FIFO compaction could cause data lost quietly, we have never tried
it in Flink and lack such experiences you mentioned here.
> when using FIFO compaction, I found sst being deleted on the first checkpoint
> -----------------------------------------------------------------------------
>
> Key: FLINK-21543
> URL: https://issues.apache.org/jira/browse/FLINK-21543
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Reporter: xiaogang zhou
> Priority: Major
>
> 2021/03/01-18:51:01.202049 7f59042fc700 (Original Log Time
> 2021/03/01-18:51:01.200883) [/compaction/compaction_picker_fifo.cc:107]
> [_timer_state/processing_user-timers] FIFO compaction: picking file 1710 with
> creation time 0 for deletion
>
> the configuration is like
> currentOptions.setCompactionStyle(getCompactionStyle());
> currentOptions.setLevel0FileNumCompactionTrigger(8);
> //
> currentOptions.setMaxTableFilesSizeFIFO(MemorySize.parse("2gb").getBytes());
> CompactionOptionsFIFO compactionOptionsFIFO = new CompactionOptionsFIFO();
>
> compactionOptionsFIFO.setMaxTableFilesSize(MemorySize.parse("8gb").getBytes());
> compactionOptionsFIFO.setAllowCompaction(true);
>
> the rocksdb version is
> <dependency>
> <groupId>io.github.myasuka</groupId>
> <artifactId>frocksdbjni</artifactId>
> <version>6.10.2-ververica-3.0</version>
> </dependency>
>
> I think the problem is caused by tableproperty is lost by snapshot. Can any
> one suggest how i can skip this problem?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
