On Sun, 29 Mar 2026, Nicolas Pitre wrote:
> On Sun, 29 Mar 2026, Aura Kelloniemi wrote:
>
> > If we want BRLTTY support in kmscon, we will likely have to initiate that
> > ourselves: contact the developers and work out how best to integrate it. I
> > think this is important, because the Linux console is unlikely to gain the
> > features many sighted users now expect: high‑quality TrueType fonts, full
> > RGB
> > color, richer mouse support, working scrollback, clipboard, etc. These are
> > precisely the kinds of features that motivate projects like kmscon.
>
> Agreed. And as a matter of fact it appears to me that you are a very
> good candidate to establish that contact. Your extensive understanding
> of the landscape is notable and you could make a difference by
> coordinating communication between all parties involved without
> necessarily being the one doing all the coding work.
Well... given the glutton for punishment that I am, I went ahead and
looked at what BRLTTY+kmscon would entail. That resulted in the attached
design document. The actual code implementing it would be quite simple
and probably smaller than the document itself.
Comments welcome.
Nicolas
# Design: kmscon Screen Access for BRLTTY
## Status
Draft â 2026-03-30
## Authors
Nicolas Pitre <[email protected]> (BRLTTY community), with design
assistance from Claude (Anthropic)
## Summary
This document proposes an architecture for making kmscon â a userspace
replacement for the Linux kernel virtual terminal (VT) â accessible to
BRLTTY, the Linux braille display daemon. The design introduces a shared
memory segment for efficient screen content access and a Unix domain
socket for control operations and event signaling.
## Contents
- Background: kmscon, BRLTTY, and the problem being solved
- Design goals: efficiency, robustness, extensibility
- Architecture overview: two-channel design (shm + socket)
- Detailed design: shm format, TLV protocol, cell layout, screen export
renderer, socket messages, keyboard input, session management,
multiple clients, area highlighting
- Security: access control via socket permissions, peer credentials,
anonymous shm
- Alternatives considered: five approaches evaluated and discarded
- Implementation roadmap: phased plan for both projects
- Open questions: items requiring further discussion
## Background
### kmscon
kmscon is a userspace terminal emulator that renders directly to
DRM/KMS display hardware, replacing the kernel's built-in virtual
console. It uses libtsm (Terminal-emulator State Machine) for all
terminal emulation and state management. kmscon supports multiple
sessions (analogous to kernel VTs), multiple display outputs, and
multiple font/rendering backends.
- Repository: https://github.com/kmscon/kmscon
- Build system: meson
- Key dependency: libtsm (https://github.com/kmscon/libtsm)
### BRLTTY
BRLTTY is a background daemon that provides braille display access to
the Linux console. It supports dozens of braille display hardware models
and includes multiple screen drivers for different environments (Linux
VT, tmux, AT-SPI2, Android, etc.).
- Website: http://brltty.app/
- Maintainer: Dave Mielke
### The problem
BRLTTY's Linux screen driver depends entirely on kernel VT interfaces:
| Interface | Purpose |
|-----------|---------|
| `/dev/vcsa[N]` | Read screen content (characters + VGA attributes), cursor position, dimensions |
| `/dev/vcsu[N]` | Read screen content as 32-bit Unicode codepoints (preferred over vcsa when available) |
| `/dev/tty0` + `VT_GETSTATE` | Query active VT number |
| `KDGETMODE` | Determine text vs. graphics mode |
| `TIOCSTI` | Inject keystrokes into the terminal |
| `TIOCL_SETSEL` | Text selection |
| `TIOCL_GETBRACKETEDPASTE` | Query bracketed paste mode |
| `GIO_UNIMAP` | Font-to-Unicode mapping (fallback when `/dev/vcsu` is not available) |
`/dev/vcsu` is the primary text interface on modern kernels. It provides
full Unicode codepoints (one `uint32_t` per cell) without requiring
font-to-Unicode reverse mapping via `GIO_UNIMAP`. However, it does not
carry attributes â BRLTTY reads those from `/dev/vcsa` alongside.
Double-width characters are represented by placing a zero-width space
(U+200B) in the continuation cell.
When kmscon replaces the kernel VT, **all of these interfaces
disappear**. BRLTTY's Linux screen driver becomes non-functional.
A new integration path is needed.
## Design goals
1. **Efficiency**: BRLTTY should only read the cells it needs (the
braille window region), not the entire screen on every update.
2. **Event-driven**: BRLTTY should sleep when the screen is idle and
wake only when content changes. No busy-polling.
3. **Robustness**: If BRLTTY crashes or disconnects, kmscon must
continue to function normally with no user-visible impact.
4. **Extensibility**: The protocol must allow new metadata fields and
capabilities to be added without breaking existing implementations.
5. **Minimal conversion overhead**: Data formats should be close to
what each side uses natively to minimize per-cell transformation
costs.
6. **No artificial limits**: The design must not assume maximum screen
dimensions. Configurations such as 3840Ã2160 at 8Ã8 font (480Ã270
cells) are realistic.
## Architecture overview
```
âââââââââââââââââââââââââââââââ ââââââââââââââââââââââââââââââ
â kmscon â â BRLTTY â
â â â â
â ââââââââââââââââââââââââââ â â âââââââââââââââââââââââââ â
â â tsm_screen (libtsm) â â â â Screen Driver "kmscon"â â
â â terminal state machine â â â â (new) â â
â ââââââââââââ¬ââââââââââââââ â â âââââââââ¬ââââââââââââââââ â
â â â â â â
â ââââââââââââ¼ââââââââââââââ â shm â âââââââââ¼ââââââââââââââââ â
â â Screen export renderer â âââââââââºâ â shm reader â â
â â (new) â â mmap â â (read-only) â â
â ââââââââââââ¬ââââââââââââââ â â âââââââââââââââââââââââââ â
â â â â â
â ââââââââââââ¼ââââââââââââââ â socket â âââââââââââââââââââââââââ â
â â Control socket ââââ¼ââââââââºâ â Control client â â
â â (new) â â â â â â
â ââââââââââââââââââââââââââ â â âââââââââââââââââââââââââ â
â â â â
â ââââââââââââââââââââââââââ â â â
â â uterm_input â â â â
â â (keyboard via evdev) ââââ¼âââââââââ¼ââ⺠key events offered â
â ââââââââââââââââââââââââââ â â to BRLTTY first â
âââââââââââââââââââââââââââââââ ââââââââââââââââââââââââââââââ
```
The design has two communication channels:
- **Shared memory**: Screen cell data (read-only for BRLTTY). This is
the hot path â BRLTTY reads only the cells under its braille window
directly from mapped memory with no IPC round-trip.
- **Unix domain socket**: Control operations, event notifications, and
keyboard interaction. This carries low-volume, low-frequency messages.
## Detailed design
### Shared memory segment
#### Lifecycle
kmscon creates an anonymous shared memory segment (`memfd_create()`) for
each terminal session. The segment is sized to hold a fixed preamble, a
variable-length TLV header, and the cell array for the current screen
dimensions. The fd is passed to BRLTTY over the Unix socket at
connection time.
When the screen is resized, kmscon creates a **new** shm segment with
the new dimensions, populates it, and passes the new file descriptor to
BRLTTY over the socket. BRLTTY maps the new segment, then unmaps and
closes the old one. This avoids race conditions during resize â the old
segment remains valid until BRLTTY releases it.
This safety relies on standard file descriptor semantics: the underlying
memory is not freed until all file descriptors and mappings are closed.
The sequence is:
1. kmscon creates and populates the new shm segment.
2. kmscon sends the new segment's fd to BRLTTY over the socket.
3. kmscon closes its fd to the old segment.
4. BRLTTY receives the new fd and maps the new segment.
5. BRLTTY unmaps and closes the old segment â its memory is now freed.
#### Preamble (fixed format)
The first bytes of the shm segment are a fixed-size preamble:
```c
struct kmscon_shm_preamble {
uint32_t magic; /* Identifies this as a kmscon shm segment */
uint16_t version; /* Preamble/TLV framing version */
uint16_t header_size; /* Total header size (preamble + TLV area) */
};
```
The `version` field only changes if the preamble or TLV framing format
itself changes. Adding new TLV fields does not require a version bump.
#### Consistency model
No locking is used on the shm segment. If BRLTTY reads cells while
kmscon is mid-write, the braille display may show a momentary
inconsistency. This is resolved by the socket notification mechanism:
kmscon sends a `Screen updated` notification after every write. If a
write occurs while BRLTTY is reading, the notification ensures BRLTTY
will re-read promptly.
This is the same model the Linux kernel uses for `/dev/vcs*`. The VT
layer clears a poll event flag when a read starts. If the screen changes
during the read, the flag is set again, causing `poll()` to return
immediately and trigger a re-read. No locking is involved â eventual
consistency is sufficient for display output.
The socket notification serves the same role as the kernel's poll event
flag:
1. BRLTTY blocks in `poll()` on the socket fd.
2. kmscon updates shm and sends a `Screen updated` notification.
3. BRLTTY wakes, drains the notification, and reads its braille window
from shm.
4. If kmscon wrote to shm during step 3, it will have sent another
notification â BRLTTY's next `poll()` returns immediately.
5. If nothing changed during the read, BRLTTY blocks until the next
notification.
#### TLV format
Both the shm header and socket messages use the same TLV (type-length-
value) framing and share a unified type numbering space. Each entry is:
```c
struct kmscon_tlv_entry {
uint16_t type; /* Field type identifier */
uint16_t length; /* Value length in bytes */
/* uint8_t value[length], padded to 4-byte alignment */
};
```
The `length` field contains the actual data size. The TLV walker is
responsible for rounding up to the next 4-byte boundary to find the
start of the next entry:
```
next_entry = current_value_ptr + ((length + 3) & ~3)
```
In the shm header, type `0x0000` is a sentinel marking the end of the
TLV area. On the socket, each message carries one or more TLV entries.
Each side skips types it does not recognize. kmscon can add new fields
without breaking older BRLTTY drivers. BRLTTY can look for a field and
gracefully handle its absence.
#### Type number ranges
| Range | Usage |
|-------|-------|
| `0x0001â0x00FF` | Shared data types â used in both the shm header (as a full state snapshot) and socket messages (as incremental updates). Parsed with the same code path. |
| `0x0100â0x01FF` | kmscon â BRLTTY control messages â socket only. |
| `0x0200â0x02FF` | BRLTTY â kmscon control messages â socket only. |
This unification means that, for example, a cursor position change can
be communicated as a single TLV entry (type `0x0002`) on the socket
rather than requiring a full shm header rewrite and a separate "screen
updated" notification. BRLTTY can apply the delta directly or re-read
from shm â whichever is more appropriate for the specific field.
#### Shared data types (0x0001â0x00FF)
| Type | Name | Size | Value format |
|------|------|------|-------------|
| `0x0001` | Screen dimensions | 4 | `uint16_t cols, uint16_t rows` |
| `0x0002` | Cursor position | 4 | `uint16_t col, uint16_t row` |
| `0x0003` | Cursor attributes | 4 | `uint32_t flags` (see below) |
| `0x0004` | Mouse pointer position | 4 | `uint16_t col, uint16_t row` |
| `0x0005` | Active session number | 2 | `uint16_t number` |
| `0x0006` | Cell array descriptor | 8 | `uint32_t offset, uint16_t cell_stride, uint16_t format_version` |
Cursor attributes flags:
| Bit | Attribute |
|-----|-----------|
| 0 | Visible |
| 1 | Blinking |
| 2â3 | Shape: 0 = default, 1 = block, 2 = underline, 3 = bar |
| 4 | Bracketed paste mode active |
| 5â31 | Reserved |
Bracketed paste mode is included in cursor attributes because it
describes how input behaves at the cursor â when active, injected text
must be wrapped in bracket escape sequences. BRLTTY checks this flag
before pasting clipboard content.
Currently libtsm only tracks cursor visibility (`TSM_SCREEN_HIDE_CURSOR`).
Cursor shape (DECSCUSR) and blink are not yet implemented in libtsm but
are standard terminal features likely to be added in the future. The
flags field accommodates these from the start.
The cell array descriptor tells BRLTTY where the cell data begins in
the shm segment and how large each cell is. If the cell format grows
in the future (e.g., to add hyperlink IDs or underline style), the
stride increases but older readers can safely ignore trailing bytes per
cell by relying on the stride value.
#### Cell format
Each cell in the array represents one screen position:
```c
struct kmscon_shm_cell {
uint32_t codepoint; /* Unicode codepoint (UCS-4) */
uint16_t flags; /* Width + attribute flags */
uint8_t fr, fg, fb; /* Foreground color (RGB) */
uint8_t br, bg, bb; /* Background color (RGB) */
};
```
12 bytes per cell, naturally aligned.
The `flags` field encodes both character width and text attributes:
| Bits | Attribute |
|------|-----------|
| 0 | Single-width |
| 1 | Double-width |
| 2 | Bold |
| 3 | Italic |
| 4 | Underline |
| 5 | Blink |
| 6 | Inverse (resolved â informational only) |
| 7 | Protect |
| 8â15 | Reserved |
Character width is obtained by masking the low two bits: `flags & 0x03`.
A normal character has bit 0 set (width = 1). A double-width character
has bit 1 set (width = 2). A continuation cell following a double-width
character has neither bit set (width = 0).
libtsm stores double-width characters (e.g., CJK ideographs) as a
primary cell with width 2 followed by a continuation cell with width 0.
The shm format preserves this representation directly. BRLTTY can use
the width for correct braille display alignment and cursor routing
without needing to inspect character values or peek at adjacent cells.
This is an improvement over the kernel VT's `/dev/vcsu` interface,
which represents double-width characters by placing a zero-width space
(U+200B) in the continuation cell â requiring the reader to recognize
the sentinel character value.
The RGB values are **always final resolved colors**. When inverse mode
is active, kmscon swaps foreground and background before writing to shm.
Color codes from libtsm's palette are resolved to RGB at write time.
BRLTTY receives ready-to-use colors with no palette lookup needed.
##### Mapping to BRLTTY's ScreenCharacter
The conversion from `kmscon_shm_cell` to BRLTTY's `ScreenCharacter` is
nearly trivial:
| shm cell field | ScreenCharacter field |
|---|---|
| `codepoint` | `text` (wchar_t) |
| `fr, fg, fb` | `color.foreground` (RGBColor) |
| `br, bg, bb` | `color.background` (RGBColor) |
| `flags & BOLD` | `color.isBold` |
| `flags & ITALIC` | `color.isItalic` |
| `flags & UNDERLINE` | `color.hasUnderline` |
| `flags & BLINK` | `color.isBlinking` |
| `flags & 0x03` | character width |
| (always) | `color.usingRGB = 1` |
This conversion is performed only for the cells within BRLTTY's current
braille window â typically 40 or 80 cells â not the entire screen.
Unlike other BRLTTY screen drivers (Linux VT, tmux) which must maintain
a shadow copy of the screen content in the driver's own memory, the
kmscon driver does not need one. The shm segment is persistently mapped
and always reflects the current screen state, so BRLTTY can read any
cell at any time without having to cache or refresh a local copy.
##### Mapping from libtsm's tsm_screen_attr
kmscon's screen export renderer receives per-cell data via the
`tsm_screen_draw()` callback. Writing to shm is similarly direct:
| draw callback parameter | shm cell field |
|---|---|
| `ch` (UCS-4 array, first element) | `codepoint` |
| `width` | `flags` bits 0â1 (1 << (width - 1) for width > 0, else 0) |
| `attr.fr, attr.fg, attr.fb` | `fr, fg, fb` |
| `attr.br, attr.bg, attr.bb` | `br, bg, bb` |
| `attr.bold, attr.italic, ...` | `flags` bits 2+ |
libtsm's `to_rgb()` function resolves palette color codes to RGB values
before the draw callback fires, so the RGB fields are always populated
regardless of whether the original color was specified as a palette
index, a 256-color code, or a true-color RGB value.
### Screen export renderer
kmscon's rendering architecture already supports multiple renderers per
terminal session. Each display gets its own renderer (bbulk for software
rendering, gltex for OpenGL), and `redraw_all()` iterates through all
of them, calling `tsm_screen_draw()` once per renderer.
The screen export renderer is a new renderer that participates in this
same cycle. Instead of drawing to a framebuffer, it writes cell data to
the shared memory segment. It uses libtsm's age tracking to skip cells
that have not changed since the last pass â the same optimization the
visual renderers use.
After updating the shm segment, the export renderer sends a `Screen
updated` notification on the control socket to wake BRLTTY.
### Unix domain socket
The control socket carries all operations that are not read-only screen
content access.
#### Connection and discovery
kmscon listens on a well-known Unix domain socket path (e.g.,
`/run/kmscon/session-$N.sock`). BRLTTY's kmscon screen driver connects
on startup.
Discovery follows the same pattern as BRLTTY's tmux driver: sensible
auto-discovery with optional explicit override via driver parameters.
The kmscon screen driver accepts:
- `socket` â explicit socket path, overriding auto-discovery.
- `session` â connect to a specific session number.
When no parameters are given, the driver looks for a kmscon socket
under the default path (`/run/kmscon/`) and connects to the active
session. This mirrors the tmux driver's behavior, where tmux
auto-discovers a session when no explicit session name is provided.
#### Message framing
Socket messages use the same TLV framing as the shm header. Each
message consists of one or more TLV entries. This means the same
parsing code handles both shm header fields and socket messages.
A socket message carrying a shared data type (0x0001â0x00FF) is an
incremental update â it carries the same entry that appears in the
shm header, with the current value. For example, a cursor move sends
a single type `0x0002` entry with the new position. BRLTTY can apply
it directly without re-reading the shm header.
#### Control types: kmscon â BRLTTY (0x0100â0x01FF)
| Type | Name | Size | Purpose |
|------|------|------|---------|
| `0x0100` | Screen updated | 0+ | Cell content in shm has changed. Optional payload: dirty rows bitmask. |
| `0x0101` | Resize | 4 | Screen resized. Payload: `uint16_t cols, uint16_t rows`. Shm fd passed out-of-band via `SCM_RIGHTS`. |
| `0x0102` | Session switched | 6 | Active session changed. Payload: `uint16_t session_number, uint16_t cols, uint16_t rows`. Shm fd passed out-of-band via `SCM_RIGHTS`. |
| `0x0103` | Key event offer | 7 | A key event for BRLTTY to accept. Payload: `uint16_t keycode, uint8_t value, uint32_t modifiers`. Same format as key event injection (type `0x0200`). |
#### Control types: BRLTTY â kmscon (0x0200â0x02FF)
| Type | Name | Size | Purpose |
|------|------|------|---------|
| `0x0200` | Key event injection | 8 | Inject a key event. Payload: `uint16_t keycode, uint8_t value, uint8_t padding, uint32_t modifiers`. Uses the Linux `input_event` keycode space. `value`: 0 = release, 1 = press, 2 = repeat. |
| `0x0201` | Character injection | 4 | Inject a Unicode character. Payload: `uint32_t codepoint` (UCS-4). Written directly to the PTY as UTF-8. Used for braille input, clipboard paste, etc. |
| `0x0202` | Key event accepted | 0 | BRLTTY claims the offered key event. |
| `0x0203` | Enable key offers | 0 | BRLTTY requests kmscon to start offering key events. |
| `0x0204` | Disable key offers | 0 | BRLTTY requests kmscon to stop offering key events. |
| `0x0205` | Highlight region | 8 | Set visual highlight. Payload: `uint16_t left, uint16_t top, uint16_t right, uint16_t bottom`. |
| `0x0206` | Unhighlight | 0 | Clear visual highlight. |
| `0x0207` | Mouse click | 5 | Simulate a mouse click. Payload: `uint16_t col, uint16_t row, uint8_t button`. |
| `0x0208` | Switch session | 2 | Request switch to session number. Payload: `uint16_t number`. |
Note: cursor routing is not a kmscon operation. Unlike a mouse click
which is a single event that the application interprets, cursor routing
is an iterative process. BRLTTY implements it as a state machine that
sends arrow key presses (via key injection, type `0x0200`) and monitors
the cursor position response (via type `0x0002` in shm or socket
updates). The routing algorithm handles overshooting, vertical scrolling
detection, timeouts, and multi-phase adjustment (vertical then
horizontal) entirely within BRLTTY. No dedicated cursor routing message
is needed.
#### Key events and character injection
The protocol distinguishes two forms of input injection:
- **Key event injection** (type `0x0200`): Carries a Linux keycode,
press/release/repeat state, and modifier flags. The keycode and value
fields use the same encoding as the Linux input subsystem (`struct
input_event` from `<linux/input.h>`), ensuring key events can flow
between kmscon, BRLTTY, and the kernel input layer without
translation. kmscon processes injected key events through xkbcommon
with the full keymap context (dead keys, compose sequences, keyboard
layout), just as it would for a physical keypress. This is used for
keyboard event bounceback and cursor routing (arrow keys).
- **Character injection** (type `0x0201`): Carries a Unicode codepoint
(UCS-4). kmscon encodes it as UTF-8 and writes it directly to the PTY
master, bypassing xkbcommon. This is used for braille keyboard input,
clipboard paste, and any other source that produces text rather than
key events.
This separation preserves the full fidelity of key events throughout
the pipeline. A bounced-back key sequence replays exactly as if the
keys were pressed on the physical keyboard. Character data goes straight
to the application without being reinterpreted through a keymap.
Bracketed paste wrapping, if needed, is handled by BRLTTY before
sending character injection messages, informed by the bracketed paste
mode flag in the cursor attributes (type `0x0003`, bit 4).
### Keyboard input
#### Ownership model
kmscon owns the keyboard. It opens evdev devices directly and processes
key events through libxkbcommon. This is critical for robustness: if
BRLTTY crashes, keyboard input continues to work with no interruption
or reconfiguration.
#### First-refusal protocol
Key event offering is opt-in. By default, kmscon processes all keyboard
input itself. A client sends `Enable key offers` (type `0x0203`) when it
has keyboard bindings that need interception, and `Disable key offers`
(type `0x0204`) when it no longer needs them. If no keyboard bindings
are defined, the client never enables key offers and there is zero
overhead on the keyboard path.
When multiple clients have key offers enabled, they form a chain.
kmscon offers key events to the first client in the chain. That client
either consumes the key (binding matched) or injects it back (via type
`0x0200`). Injected keys from the first client are offered to the
second client, and so on. The last client's injections reach the
terminal emulator. If a client disconnects, it is removed from the
chain.
For a single client, the flow is:
1. kmscon reads a key event from evdev.
2. kmscon sends a `Key event offer` (type `0x0103`) to the client with
the linux keycode, press/release/repeat state, and modifier flags.
3. The client responds with `Key event accepted` (type `0x0202`).
4. The client either consumes the key or injects it back via type
`0x0200`, at which point kmscon feeds it to the terminal emulator.
From kmscon's perspective, every offered key is accepted. The client
takes full ownership of the key and decides what to do with it.
The `Key event accepted` reply also serves as a liveness check. If a
client does not acknowledge a key offer within a reasonable timeout,
kmscon assumes the client is hung and removes it from the key offer
chain. This prevents a malfunctioning client from silently sitting on
all keyboard input. Subsequent key events are offered to the next
client in the chain, or processed directly by the terminal emulator
if no clients remain.
#### Key combination handling
BRLTTY implements its own key bindings, some of which are multi-key
combinations. When BRLTTY sees the beginning of a potential binding, it
accepts each key as it arrives. If subsequent keys complete the binding,
BRLTTY acts on it and the keys are consumed. If the combination does
not match any binding, BRLTTY sends the accumulated keys back to kmscon
as individual key event injection messages (type `0x0200`). kmscon
processes these through xkbcommon exactly as if they came from the
physical keyboard.
This mirrors BRLTTY's existing behavior with `EVIOCGRAB` and uinput,
where it grabs the keyboard, buffers potential binding keys, and
re-injects unmatched sequences through a virtual input device. The
latency characteristics are comparable.
### Session management
kmscon sessions map directly to BRLTTY's virtual terminal concept.
Each session has its own:
- `tsm_screen` instance (terminal state)
- shm segment (screen content)
- Active/inactive state
BRLTTY tracks which session is active via the `Active session number`
TLV field and the `Session switched` socket notification. When a session
switch occurs, kmscon sends the new session's shm fd over the socket.
BRLTTY's `currentVirtualTerminal()` returns the active session number,
and `switchVirtualTerminal()` sends a `Switch session` request.
### Multiple clients
kmscon supports multiple simultaneous accessibility client connections.
This is a common scenario during BRLTTY development and testing, where
two BRLTTY instances run with different braille displays against the
same console.
- **Shared memory**: Read-only for all clients. Multiple readers can
mmap the same segment concurrently with no conflict.
- **Socket notifications**: kmscon sends screen update and session
notifications to all connected clients.
- **Key injection and character injection**: Any client can inject
input. kmscon processes injections from all clients identically.
- **Key offers**: Key events are cascaded through all clients that have
enabled key offers, forming a chain. kmscon offers key events to the
first client. Key injections from the first client are offered to the
second client (instead of going directly to the terminal). Key
injections from the second client are offered to the third, and so
on. The last client in the chain injects into the terminal emulator.
If no clients have key offers enabled, key events go directly to the
terminal emulator. Clients that have not enabled key offers are
skipped in the chain.
- **Highlight regions**: Each client may set its own highlight region.
kmscon renders all active highlights simultaneously, allowing a
sighted helper to see where each braille display is positioned on
screen.
### Area highlighting
BRLTTY can visually mark the region under the braille window on screen,
allowing a sighted helper to see where the braille user is reading.
BRLTTY sends a `Highlight region` message with the bounding rectangle.
kmscon applies inverse or distinct color attributes to those cells
during visual rendering, similar to how libtsm already renders
selections via `attr.inverse` in the draw callback.
`Unhighlight` clears the region.
## Security
The shm segment exposes the full terminal content â including passwords
and sensitive data as they are displayed â and the control socket allows
keystroke injection. Access control is critical.
### Socket permissions
kmscon creates the Unix domain socket with restrictive ownership and
permissions. The socket is owned by root and a designated group (e.g.,
`brl`), with mode `0660`. Only processes running as root or in the `brl`
group can connect. The kernel enforces supplementary group membership on
`connect()`, so BRLTTY does not need to run as root â membership in the
`brl` group is sufficient.
This follows the same model BRLTTY already uses for access to
`/dev/input/*` devices and braille USB hardware, where udev rules
assign group ownership. BRLTTY runs as a system service (started at
boot, before any user logs in) with the appropriate group membership
configured via its systemd unit (`SupplementaryGroups=brl`).
### Peer credential verification
When a client connects, kmscon can retrieve the peer credentials via
`SO_PEERCRED` (`getsockopt` with `SOL_SOCKET`), which returns the
peer's uid, gid, and pid as reported by the kernel. Note that this only
provides the primary gid, not supplementary groups. It is therefore
useful for logging and uid-level policy, but the socket filesystem
permissions are the primary gate for group-based access control.
### Anonymous shared memory
The shm segments are created using `memfd_create()` rather than
`shm_open()`. This produces anonymous memory with no filesystem name â
the segment cannot be discovered or opened by other processes by name.
The only way to access it is to receive the file descriptor over the
authenticated socket connection.
This provides defense in depth: even if a process somehow learns that
a shm segment exists, it cannot open it without the fd. The fd is only
sent to clients that have passed the socket permission checks.
### Why not seat-based access control
systemd-logind provides seat-based access for devices like audio and
video hardware â granting access to the active session on a seat. While
kmscon is inherently seat-aware, this model does not apply to BRLTTY:
BRLTTY is a system service that must be running before any user session
exists (to enable login via braille display). It does not belong to any
logind session and cannot rely on seat-based grants.
Group-based access is the appropriate mechanism for system services that
need persistent access to console resources regardless of session state.
### Summary of access control layers
1. **Socket filesystem permissions**: group-based access control on the
socket path, enforced by the kernel including supplementary groups.
2. **Peer credentials**: kmscon can verify the connecting process
identity for logging and additional policy.
3. **Anonymous shm**: no filesystem name, fd only available to
authenticated clients.
## Alternatives considered
### Alternative 1: Emulate /dev/vcsa from kmscon
kmscon could expose a virtual `/dev/vcsa*` device (via FUSE or a
character device) that mirrors the `tsm_screen` state. BRLTTY's
existing Linux screen driver would work without modification.
**Why we discarded this:**
- The vcsa format is limited â originally 8-bit characters plus VGA
attributes. The unicode device (`/dev/vcsu*`) provides 32-bit
codepoints but no attributes; BRLTTY must read both devices.
- vcsa only provides screen content. BRLTTY also needs VT ioctls
(`VT_GETSTATE`, `KDGETMODE`, etc.) which go through `/dev/tty*`.
Emulating this ioctl surface is large and fragile.
- BRLTTY's Linux driver polls vcsa with `POLLPRI`, which would need
to be emulated.
- It forces a modern Unicode-native terminal into a legacy VGA model.
- Any vcsa reader would still receive the full screen on every read.
### Alternative 2: Socket-only protocol (no shared memory)
Inspired by BRLTTY's tmux driver, which communicates entirely through
pipes. kmscon could expose a control-mode protocol and BRLTTY would
request screen content over the socket.
**Why we discarded this:**
- The tmux driver performs a full `capture-pane` on every update,
transferring the entire screen content even when only one character
changed and even when the braille display shows a different region.
- For a primary console, this inefficiency is significant.
- Shared memory allows BRLTTY to read only the cells it needs (the
braille window) with zero IPC overhead on the read path.
- We kept the socket for control operations and notifications, where
its request/response model is appropriate.
### Alternative 3: BRLTTY owns the keyboard
BRLTTY grabs the keyboard via `EVIOCGRAB` as it does today, and
forwards unhandled keys to kmscon over the control socket.
**Why we discarded this:**
- If BRLTTY crashes, kmscon loses all keyboard input. The terminal
becomes unusable until BRLTTY is restarted. This is a critical
robustness failure for a system console.
- Similarly, BRLTTY's uinput-based re-injection creates a dependency:
kmscon would read from a virtual device that disappears when BRLTTY
exits.
- The chosen design (kmscon owns the keyboard, offers keys to BRLTTY)
degrades gracefully: BRLTTY disconnection has zero impact on terminal
usability.
### Alternative 4: Fixed maximum shm allocation
Allocate shared memory for the largest plausible screen (e.g., 256Ã256)
to avoid remapping on resize.
**Why we discarded this:**
- Real-world configurations exceed any reasonable "maximum". Users run
3840Ã2160 displays with 8Ã8 fonts, yielding 480Ã270 cells.
Future displays and smaller fonts could push this further.
- The fd-passing approach for resize (create new segment, pass fd over
socket) is clean and imposes no artificial limits.
### Alternative 5: Shared library for terminal emulators and accessibility tools
Implement the screen export and access protocol as a shared library
that any terminal emulator could link against (writer side) and any
accessibility tool could link against (reader side).
**Why we discarded this:**
- Each terminal emulator has its own internal screen state
representation. kmscon uses libtsm with a draw callback. Other
emulators (foot, alacritty, kitty) have completely different
architectures. A shared library would either impose a common internal
format (invasive, unlikely to be accepted upstream) or provide an
adapter layer that each emulator populates â which is essentially the
shm cell format we already define, with extra ceremony.
- The writer side is a few hundred lines of straightforward code:
create shm, write cells, send socket notifications. The reader side
is similarly simple: mmap, parse TLV, read cells. Wrapping this in a
library adds a dependency, a versioning burden, and an API surface to
maintain for very little code reuse.
- A shared library introduces practical problems: packaging
dependencies across distributions, ABI stability commitments, version
skew between the library and its consumers, and a governance question
about who maintains it.
- The real interoperability contract is the **protocol** â the shm
format, TLV types, and socket message definitions documented in this
specification. Any terminal emulator can implement the writer side
and any accessibility tool can implement the reader side by following
the spec, without depending on a shared library.
- Reference implementations or helper code can be provided as copyable
source files or header-only utilities without the constraints of a
shared library.
## Implementation roadmap
### Phase 1: kmscon side
1. **Screen export renderer**: New source file in `src/` implementing
the shm writer as a `tsm_screen_draw()` consumer. Hooks into the
existing `redraw_all()` cycle.
2. **Shared memory management**: Create/destroy anonymous shm segments
(`memfd_create`) per session. Handle resize by creating new segments
and passing fds over the socket.
3. **Control socket listener**: Unix domain socket accepting BRLTTY
connections. Handle key injection, session switching, highlight
requests.
4. **Keyboard first-refusal**: When BRLTTY enables key offers, offer
key events before processing. Fall back to normal processing on
disconnect.
5. **Build integration**: New meson option (e.g., `screen_export` or
`accessibility`) to enable/disable the feature and its dependencies.
### Phase 2: BRLTTY side
1. **New screen driver**: `Drivers/Screen/Kmscon/screen.c` implementing
the `BaseScreen` interface.
2. **shm reader**: Map the shm segment read-only. Implement
`readCharacters()` by reading directly from the shm cell array.
No shadow screen copy needed.
3. **Socket client**: Connect to kmscon's control socket. Register for
async I/O notifications. Implement key event and character injection,
session switching, highlight, and keyboard event handling.
4. **Driver registration**: Add to BRLTTY's screen driver list and
build system.
### Phase 3: Refinements
- Dirty region hints in update notifications
- Negotiation of optional capabilities at connection time
## Open questions
1. **Protocol versioning and capability negotiation**: Should BRLTTY
and kmscon exchange supported message types at connection time?
2. **libtsm changes**: Adding a direct cell-read API to libtsm (instead
of relying solely on the draw callback) would simplify the export
renderer. Is the libtsm maintainer open to this?
4. **Upstream acceptance**: Both projects need to agree on the protocol.
A shared header file defining the shm and socket message formats
would ensure consistency.
_______________________________________________
This message was sent via the BRLTTY mailing list.
To post a message, send an e-mail to: [email protected]
For general information, go to: http://brltty.app/mailman/listinfo/brltty