On Tue, Oct 31, 2023 at 04:05:46PM -0300, Fabiano Rosas wrote: > Daniel P. Berrangé <berra...@redhat.com> writes: > > > On Tue, Oct 31, 2023 at 12:52:41PM -0300, Fabiano Rosas wrote: > >> Daniel P. Berrangé <berra...@redhat.com> writes: > >> > > >> > I guess I'm not seeing the problem still. A single FD is passed across > >> > from libvirt, but QEMU is free to turn that into *many* FDs for its > >> > internal use, using dup() and then setting O_DIRECT on as many/few of > >> > the dup()d FDs as its wants to. > >> > >> The problem is that duplicated FDs share the file status flags. If we > >> set O_DIRECT on the multifd channels and the main thread happens to do > >> an unaligned write with qemu_file_put* then the filesystem will fail > >> that write. > > > > Doh, I had forgotten that sharing. > > > > Do we have any synchronization between multifd channels and the main > > thread ? eg does the main thread wait for RAM sending completion > > before carrying on writing other non-RAM data ? > > We do have, but the issue with that approach is that there are no rules > for adding data into the stream. Anyone could add a qemu_put_* call > right in the middle of the section for whatever reason. > > That is almost a separate matter due to our current compatibility model > being based on capabilities rather than resilience of the stream > format. So extraneous data in the stream always causes the migration to > break. > > But with the O_DIRECT situation we'd be adding another aspect to > this. Not only changing the code requires syncing capabilities (as it > does today), but it would also require knowing which parts of the stream > can be interrupted by new data and which cannot. > > So while it would probably work, it's also a little fragile. If QEMU > were given 2 FDs or given access to the file, then only the multifd > channels would get O_DIRECT and they are guaranteed to not have > extraneous unaligned data showing up.
So the problem with add-fd is that when requesting a FD, the monitor code masks flags with O_ACCMODE. What if we extended it such that the monitor masked with O_ACCMODE | O_DIRECT. That would let us pass 1 plain FD and one O_DIRECT fd, and be able to ask for each separately by setting O_DIRECT or not. Existing users of add-fd are not likely to be affected since none of them will be using O_DIRECT. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
