On Thu, Aug 25, 2022 at 10:42:41AM +1000, Alistair Popple wrote: > > Peter Xu <pet...@redhat.com> writes: > > > On Wed, Aug 24, 2022 at 04:25:44PM -0400, Peter Xu wrote: > >> On Wed, Aug 24, 2022 at 11:56:25AM +1000, Alistair Popple wrote: > >> > >> Still I don't know whether there'll be any side effect of having > >> > >> stall tlbs > >> > >> in !present ptes because I'm not familiar enough with the private dev > >> > >> swap > >> > >> migration code. But I think having them will be safe, even if > >> > >> redundant. > >> > > >> > What side-effect were you thinking of? I don't see any issue with not > >> > TLB flushing stale device-private TLBs prior to the migration because > >> > they're not accessible anyway and shouldn't be in any TLB. > >> > >> Sorry to be misleading, I never meant we must add them. As I said it's > >> just that I don't know the code well so I don't know whether it's safe to > >> not have it. > >> > >> IIUC it's about whether having stall system-ram stall tlb in other > >> processor would matter or not here. E.g. some none pte that this code > >> collected (boosted both "cpages" and "npages" for a none pte) could have > >> stall tlb in other cores that makes the page writable there. > > > > For this one, let me give a more detailed example. > > Thanks, I would have been completely lost about what you were talking > about without this :-) > > > It's about whether below could happen: > > > > thread 1 thread 2 thread 3 > > -------- -------- -------- > > write to page P (data=P1) > > (cached TLB writable) > > zap_pte_range() > > pgtable lock > > clear pte for page P > > pgtable unlock > > ... > > migrate_vma_collect > > pte none, npages++, > > cpages++ > > allocate device page > > copy data (with P1) > > map pte as device > > swap > > write to page P again > > (data updated from P1->P2) > > flush tlb > > > > Then at last from processor side P should have data P2 but actually from > > device memory it's P1. Data corrupt. > > In the above scenario migrate_vma_collect_pmd() will observe pte_none. > This will mark the src_pfn[] array as needing a new zero page which will > be installed by migrate_vma_pages()->migrate_vma_insert_page(). > > So there is no data to be copied hence there can't be any data > corruption. Remember these are private anonymous pages, so any > zap_pte_range() indicates the data is no longer needed (eg. > MADV_DONTNEED).
My bad to have provided an example but invalid. :) So if the trylock in the function is the only way to migrate this page, then I agree stall tlb is fine. > > >> > >> When I said I'm not familiar with the code, it's majorly about one thing I > >> never figured out myself, in that migrate_vma_collect_pmd() has this > >> optimization to trylock on the page, collect if it succeeded: > >> > >> /* > >> * Optimize for the common case where page is only mapped once > >> * in one process. If we can lock the page, then we can safely > >> * set up a special migration page table entry now. > >> */ > >> if (trylock_page(page)) { > >> ... > >> } else { > >> put_page(page); > >> mpfn = 0; > >> } > >> > >> But it's kind of against a pure "optimization" in that if trylock failed, > >> we'll clear the mpfn so the src[i] will be zero at last. Then will we > >> directly give up on this page, or will we try to lock_page() again > >> somewhere? > > That comment is out dated. We used to try locking the page again but > that was removed by ab09243aa95a ("mm/migrate.c: remove > MIGRATE_PFN_LOCKED"). See > https://lkml.kernel.org/r/20211025041608.289017-1-apop...@nvidia.com > > Will post a clean-up for it. That'll help, thanks. -- Peter Xu