On 10/19/26 15:47, Thomas Hellström wrote:
> On Mon, 2026-10-19 at 10:30 -0300, Jason Gunthorpe wrote:
>> On Mon, Oct 19, 2026 at 02:43:29PM +0200, Thomas Hellström wrote:
>>> On Mon, 2026-10-19 at 10:35 +0200, Christian König wrote:
>>>> On 10/16/26 22:26, Karl Mehltretter wrote:
>>>>> On Tue, Feb 17, 2026 at 08:11:34PM -0400, Jason Gunthorpe
>>>>> wrote:
>>>>>> + match_args.attach = attach;
>>>>>> + if (dmabuf->ops->match_mapping) {
>>>>>> + ret = dmabuf->ops-
>>>>>>> match_mapping(&match_args);
>>>>>> + }
>>>>>> +
>>>>>> attach->dmabuf = dmabuf;
>>>>>> attach->importer_ops = importer_ops;
>>>>>> attach->importer_priv = importer_priv;
>>>>>
>>>>> I may misunderstand the intended contract. match_mapping() runs
>>>>> while the
>>>>> attachment is still zeroed and without dmabuf->resv held.
>>>>>
>>>>> Patch 11's amdgpu callback needs importer_ops and
>>>>> importer_priv, so
>>>>> its XGMI
>>>>> match cannot work here. Patch 12's VFIO callback asserts
>>>>> dmabuf-
>>>>>> resv and
>>>>> reads priv->vdev, which cleanup changes under that lock.
>>>>>
>>>>> Should the core initialize the attachment and hold dmabuf->resv
>>>>> for
>>>>> match_mapping(), or should callbacks avoid attachment and lock-
>>>>> protected
>>>>> state?
>>>>
>>>> Right. This completely misplaced.
>>>>
>>>> The matching can't be done at attachment time, but rather needs
>>>> to be
>>>> done before creating a mapping while holding the resv lock so
>>>> that
>>>> the matching decision can't change.
>>>
>>> So question is what exactly should the contract be?
>>> Looking at the pcie_p2p negotiation today it looks somewhat like.
>>>
>>> 1) Importer and exporter agree at attach time what mappings are
>>> allowed
>>> pending placement conditions and other attachers.
Not quite. They check if the P2P is in theory possible, but they don't make any
decisions on the final placement.
>>> 2) At map-time the mapping is negotiated taking all constraints
>>> into
>>> account.
>>> 3) The negotiated mapping is valid until an invalidate_mappings()
>>> callback is received.
>>>
>>> Following this would allow a re-implementation of pcie_p2p as a
>>> mapping
>>> type and would allow a controlled link-severing to send an
>>> invalidate_mappings() notifier and the mappings to be re-
>>> negotiated.
>>>
>>> Importers requiring an early hint as to what the actual mapping
>>> would
>>> look like would need to be prepared to re-consider at map-time,
>>> with a
>>> reasonable assumption that the reconsideration and re-mapping would
>>> happen seldom enough.
>>
>> The idea was the supported mapping types are fixed properties of the
>> importer and exporter.
That doesn't work quite like this.
>> Like iommufd is not suddenly going to start understanding ualink
>> after
>> an invalidation.
>>
>> Further, I had the (perhaps naive) expectation that the interconnect
>> option was not something that would change during mapping. I did not
>> think you would loose your PCIe option if ualink becomes available,
>> or
>> vice versa. Though on deeper reflection I suppose someone could build
>> HW so you might loose your ualink option if the dmabuf is moved to
>> system memory while PCIe would still work.
>
> True, but there are other complications. Let's say at map time there is
> another importer that don't support anything but system memory dma-
> bufs. Then both ualink and PCIe p2p could not be used, and we'd fall
> back to system. Or a map() move to VRAM from system for a fast
> interconnect failed for some reason. (Somebody might have pinned it in
> system).
Yes, exactly that.
> But agreed it sounds to me like the interconnect capabilities should be
> fixed at attach time.
We can check preconditions, e.g. if PCI P2P would work with this root
complex/isolation setup etc...
>>
>> Making the decision right before map seems like a reasonable
>> enhancement, but I can no longer remember off hand if that would
>> cause
>> complications :)
>
> It was that mellanox ATS thing, right? That's the only case I heard of
> that needed the mapping type decision (or at least a strong hint)
> early.
Yeah I agree that this is the only one we have heard of, but that doesn't mean
we shouldn't at least try to support it somehow.
I mean what we can do is to narrow down the possible placements on attach so
that mellanox ATS can at least rule out that P2P is possible early and
eventually stop wasting time setting that up.
It can still be that P2P or whatever interconnect fails later on, we have very
nice "the fast interconnect has a failure use the slow backup" use cases as
well, so importers really won't get away without handling that even if it's
slow.
Regards,
Christian.
>
> /Thomas
>
>
>>
>> Jason
