On Thu, Aug 6, 2026 at 4:52 AM Christian König <[email protected]> wrote:
>
> Oh, well history repeats itself.
>
> On 8/6/26 06:30, Pravin Kumar Ravi wrote:
> > Hi all,
> > This RFC is intended to start a design discussion around Synx, a SoC-wide
> > synchronization framework used to coordinate fences between Linux-hosted
> > clients and non-Linux execution environments such as remote processors and
> > firmware.
> >
> > The intent of Synx is not to replace dma-fence for existing in-kernel
> > GPU/display style synchronization. Rather, provide a global synchronization
> > mechanism spanning multiple subsystems and processors, while still
> > integrating with existing Linux primitives such as dma-fence and sync_file
> > where appropriate. The solution has shown significant power and performance
> > benefits in the last few generations of Qualcomm mobile and XR chipsets,
> > and is gathering more use cases. Also, the solution is SoC-agnostic, so
> > other vendors can adopt it in the future if interested.
> >
> >
> > *Motivation*
> > Many multimedia and accelerator pipelines on modern SoCs are no longer
> > strictly host-centric. Work may be produced, consumed, waited on, or
> > signaled by multiple execution environments, including Linux drivers,
> > userspace components, DSPs, ISPs, NPUs, video firmware, camera firmware, or
> > other remote processors.
> >
> > dma-fence is a good fit when the fence state, callback list, and lifetime
> > are owned and progressed by Linux kernel participants. However, for
> > cross-processor use cases, a synchronization object must satisfy properties
> > that dma-fence does not currently provide:
> >
> > 1.
> > *Cross-subsystem visibility*: A fence created by one subsystem must be
> > importable, waitable, and signalable by other subsystems, including
> > non-Linux processors. struct dma_fence is local Linux kernel state and is
> > not directly visible to remote processors or firmware.
> >
> >
> > 2.
> > *Point-to-point remote signaling*: One subsystem should be able to
> > signal another subsystem without interrupting the Linux host CPU every time
> > in the signaling path. Since dma-fence maintains all waiters in the Linux,
> > remote-to-remote synchronization has to unnecessarily interrupt or involve
> > the Linux host.
> >
> >
> > 3.
> > *Distributed lifetime management*: References may be held by Linux and
> > non-Linux participants. Any subsystem must have authority to create and
> > clean up fences. This requires lifetime state to be tracked in a
> > shared/global table, rather than being owned solely by a local Linux
> > object. dma-fence lifetime is centered around Linux references, which does
> > not naturally model references held and released independently by remote
> > processors.
> >
> >
> > 4.
> > *Race-free global synchronization*: Registering a waiter and observing
> > signal state must be atomic across participating subsystems to avoid missed
> > wakeups/signals. Atomicity between remote waiter registration and remote
> > signaling needs a protocol-level guarantee, not just local callback
> > handling.
> >
> >
> > 5.
> > *Subsystem restart recovery*: If a remote processor or firmware
> > subsystem restarts, references held by the crashed subsystem must be
> > released, and waiters that depend on that subsystem must be completed or
> > failed in a well-defined way. Subsystem restart cleanup is outside the
> > dma-fence model.
> >
> >
> > 6.
> > *Global composition*: It should be possible to compose fences that were
> > created by different subsystems or processors into a single aggregate
> > synchronization object. dma_fence_array can compose Linux dma_fence
> > objects, but it does not by itself solve the composition of globally
> > visible synchronization handles created by different processors.
> >
> >
> > 7.
> > *Transport abstraction*: Remote signaling should be independent of the
> > underlying transport, for example rpmsg, mailbox, GLink or another SoC
> > transport.
> >
> >
> > 8.
> > *Linux integration*: Linux clients should be able to interoperate with
> > dma-fence and sync_file where that is the right ABI or in-kernel interface.
> >
> >
> >
> > *Proposed model*
> > Synx introduces a globally unique synchronization handle. The handle
> > indexes an entry in a global synchronization table that is accessible to
> > all participating subsystems. Each entry tracks: a global handle ID,
> > current fence state, the set of waiting and subscribed cores or subsystems
> > to allow point-to-point remote signaling, a distributed reference count,
> > and parent/child relationships for composed fences.
> >
> > The design is organized around four components:
> >
> > 1.
> > *Handle and session management*: Any subsystem, Linux or remote, may
> > create or import a Synx handle and hold a reference counted against the
> > global table. Handles are destroyed by the subsystem which releases the
> > last reference, regardless of which subsystem it is.
> >
> >
> > 2.
> > *Transport abstraction*: Remote wait and signal messages are delivered
> > through a pluggable transport layer. The synchronization core is not tied
> > to any specific backend (rpmsg, mailbox, GLink, etc.), enabling
> > point-to-point remote signaling without involving the Linux host CPU when
> > both producer and consumer are outside Linux.
> >
> >
> > 3.
> > *Linux interoperability*: A Synx handle may be associated with a
> > dma_fence or exported as a sync_file, allowing Linux kernel drivers and
> > userspace to interact with Synx objects through existing interfaces.
> >
> >
> > 4.
> > *Fence composition*: Aggregate fences are modeled as parent/child
> > relationships between global handles. A composed fence can span handles
> > created by different subsystems or processors, and its completion state is
> > derived from the states of its children in the global table.
> >
> >
> > 5.
> > *Subsystem restart recovery*: When a remote processor restarts, the
> > global table is walked to drop all references held by that subsystem and to
> > complete or fail any affected waiters according to the configured recovery
> > policy.
> >
> >
> >
> > *Possible initial patch structure*
> > If the overall direction is acceptable, an initial RFC series could be
> > split as follows:
> >
> > *
> > Documentation describing the cross-subsystem synchronization problem
> > and the Synx object model.
> > *
> > Code to manage Synx sessions and handle lifetime. And synchronization
> > using wait and signal
> > *
> > dma_fence and sync_file interoperability layer.
> > *
> > Fence composition support.
> > *
> > Transport abstraction for remote wait/signal messages.
> > *
> > SSR/restart cleanup hooks for remote processors.
> > *
> > A minimal Qualcomm SoC integration backend and example client.
> >
> >
> > At this stage, feedback on the model and layering would be more useful than
> > detailed code review. In particular, we would appreciate guidance on
> > whether this should be pursued as a Qualcomm SoC-specific driver first, or
> > whether the cross-processor synchronization pieces are generic enough to
> > justify a common framework.
>
> Well to make a longer story short what you describe here was the original
> dma_fence proposal from >10 years ago. We fortunately rejected it already
> back then.
>
> By now I think most people in the Linux community who have worked on
> dma_fences agree that stuff like that is an extremely bad idea.
>
> First of all some of the stuff you suggest here go strictly against
> documented dma_fence restrictions, especially no point to point remote
> signaling! dma_fences have a huge interaction with Linux core memory
> management which makes stuff like that impossible.
>
>
> What you can do is to do this in userspace with DMA-buf and user space
> fences, signaling points etc... Take a look at how user space waits in XE
> work, how eventfd works and ROCm events work for an overview and maybe some
> starting ideas.
>
> We could certainly build a framework around that, especially eventfd sounds
> like a common ground to most drivers already, but please stay away from
> dma_fences when you want to do something like that.
>
So I'm just starting to think a bit about this, since it will be
potentially relevant when we implement fwsched for adreno..
Do we really need anything more than some sort of dma_fence signaling
extension? Ie. signaling on the CPU remains exactly the same as
today, anything waiting on the CPU would see no change. But if both
exporter and importer support the same extension, that can be used to
extract extra information to pass to fw.
The fw might observe the fence getting signaled with lower latency,
but that's fine, there isn't really any guarantee (other than timeline
ordering) about the order that multiple waiters observe that the fence
is signaled. We've just pushed one of the waiters down to firmware,
that is all. Ie. something like
rcu_read_lock();
if (fence->ops->signaling_ext &&
do_I_understand_this(fence->ops->signaling_ext)) {
fence->ops->get_magic_signaling_foo(fence, &magic);
... pass magic to fw ...
} else {
... same as before ...
}
rcu_read_unlock();
If either the importer or exporter don't understand the same signaling
extension, everything works as before. For ex, external eGPU/dGPU
sharing with say the on-chip ISP. But if both do support it, the
signaling extension provides a way to extract whatever magic thing is
needed to do the wait in fw.
BR,
-R
