> You have a replicated filesystem, brick1 and brick2.
> Brick 2 goes down and you edit a 4k file, appending data to it.
> That change, and the fact that there is a pending change, is stored on brick1.
> Brick2 returns to service.
> Your app wants to append to the file again. It calls stat on the file. Brick2
> answers first stating that the file is 4k long. Your app seeks to 4k and
> writes. Now the data you wrote before is gone.
>
> This is one of the processes by which stale stat data can cause data loss.
> That's why each lookup() (which precedes the stat) causes a self-heal check
> and why it's a problem that hasn't been resolved in the last two years.
>
> I don't know the answer. I know that they want this problem to be solved, but
> right now the best solution is hardware. The lower the latency, the less of a
> problem you'll have.
Well I'd assume that the brick that comes online has to check everything from
the other online bricks before it's authoritative in answering any client
calls. This way if a brick comes up and sees another brick(s) in its replica
blocks online the assumption should be that the data can be bad on this brick.
Therefore until a complete self heal is performed the brick should be
considered bad for this information. The next step from this is how to
guarantee that a brick actually returns to healthy state in a busy filesystem.
The basic way would be that any new writes of files are written to all bricks
(including the bad) and declared good on the healing brick and as a background
process all files on the brick are hashed and checked against the bricks that
were live before. In a reasonable environment this should complete in a
reasonable amount of time and at worst means that you'll be running at a
reduced performance while this sync is happening, but it would guarantee that
you
don't have data loss unless you lose all your previously online bricks in
which case you're anyway in disaster recovery where this semi-live brick can
help in recovering files from the time it last went down or better.
Mario Kadastik, PhD
Researcher
---
"Physics is like sex, sure it may have practical reasons, but that's not why
we do it"
-- Richard P. Feynman
_______________________________________________
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users
