[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050702#comment-17050702 ] Tsz-wo Sze commented on RATIS-815: -- Thanks [~ljain] for committing the patch. I just have found that

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049463#comment-17049463 ] Lokesh Jain commented on RATIS-815: --- [~adoroszlai] Thanks for reporting the issue! [~szetszwo] Thanks

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049462#comment-17049462 ] Lokesh Jain commented on RATIS-815: --- [~szetszwo] Thanks for updating the patch! The changes look good to

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049445#comment-17049445 ] Hadoop QA commented on RATIS-815: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049382#comment-17049382 ] Tsz-wo Sze commented on RATIS-815: -- r815_20200302.patch: addresses [~ljain]'s comments. > Log entry

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049380#comment-17049380 ] Tsz-wo Sze commented on RATIS-815: -- > ... Regarding the latest change oustanding and remaining are very

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049337#comment-17049337 ] Lokesh Jain commented on RATIS-815: --- [~szetszwo] Thanks for updating the patch! The changes look good to

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048130#comment-17048130 ] Hadoop QA commented on RATIS-815: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048124#comment-17048124 ] Hadoop QA commented on RATIS-815: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048111#comment-17048111 ] Tsz-wo Sze commented on RATIS-815: -- r815_20200228.patch: addresses [~ljain]'s comments. > Log entry

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048105#comment-17048105 ] Tsz-wo Sze commented on RATIS-815: -- > The log worker should never write over the max size. ... Indeed,

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043444#comment-17043444 ] Lokesh Jain commented on RATIS-815: --- [~szetszwo] Thanks for working on this! The changes look good to

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043184#comment-17043184 ] Hadoop QA commented on RATIS-815: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042202#comment-17042202 ] Tsz-wo Sze commented on RATIS-815: -- [~adoroszlai], thanks a lot for testing it. [~ljain], could you

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041636#comment-17041636 ] Attila Doroszlai commented on RATIS-815: Thanks [~szetszwo] for the patch. Tested it with the

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041422#comment-17041422 ] Tsz-wo Sze commented on RATIS-815: -- r815_20200220.patch: fixes some bugs and simplifies code. > Log

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041244#comment-17041244 ] Tsz-wo Sze commented on RATIS-815: -- This is a bug in the code if the log entry size > preallocate size.

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041239#comment-17041239 ] Tsz-wo Sze commented on RATIS-815: -- I see. Could you test also setting the preallocate size to the same

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041224#comment-17041224 ] Lokesh Jain commented on RATIS-815: --- > Do you mean that "fc.force(false)" does not work? Yes, it did

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041216#comment-17041216 ] Tsz-wo Sze commented on RATIS-815: -- Thanks a lot for the update. > 1. Adding a sync in preallocate

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041148#comment-17041148 ] Lokesh Jain commented on RATIS-815: --- Attempts. # Adding a sync in preallocate method does not help with

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040462#comment-17040462 ] Tsz-wo Sze commented on RATIS-815: -- {quote} Other operations, in particular those that take an explicit

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040452#comment-17040452 ] Tsz-wo Sze commented on RATIS-815: -- In ozone, we have the log segment size 4MB and preallocate size 1MB.

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040415#comment-17040415 ] Tsz-wo Sze commented on RATIS-815: -- Discussed with [~ljain], it looks like that there is a race condition

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039847#comment-17039847 ] Lokesh Jain commented on RATIS-815: --- [~szetszwo] Many log segments have a corruption. There are around

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039272#comment-17039272 ] Tsz-wo Sze commented on RATIS-815: -- [~adoroszlai], thanks for testing it with RATIS-767 reverted. I

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17038174#comment-17038174 ] Attila Doroszlai commented on RATIS-815: Tested with RATIS-767 reverted: same problem happens. >

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037385#comment-17037385 ] Tsz-wo Sze commented on RATIS-815: -- [~adoroszlai], thanks for reporting the bug. Is it easy to

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036965#comment-17036965 ] Lokesh Jain commented on RATIS-815: --- Analysis so far. # On avoiding checksum exception the parsing of

[jira] [Commented] (RATIS-815) Log entry corrupted with 0 checksum

[ https://issues.apache.org/jira/browse/RATIS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036961#comment-17036961 ] Lokesh Jain commented on RATIS-815: --- Attached logs and dumps collected by [~adoroszlai] on his cluster.