On Sun, 29 Mar 2026, Nicolas Pitre wrote:

> On Sun, 29 Mar 2026, Aura Kelloniemi wrote:
> 
> > If we want BRLTTY support in kmscon, we will likely have to initiate that
> > ourselves: contact the developers and work out how best to integrate it. I
> > think this is important, because the Linux console is unlikely to gain the
> > features many sighted users now expect: high‑quality TrueType fonts, full 
> > RGB
> > color, richer mouse support, working scrollback, clipboard, etc. These are
> > precisely the kinds of features that motivate projects like kmscon.
> 
> Agreed. And as a matter of fact it appears to me that you are a very 
> good candidate to establish that contact. Your extensive understanding 
> of the landscape is notable and you could make a difference by 
> coordinating communication between all parties involved without 
> necessarily being the one doing all the coding work.

Well... given the glutton for punishment that I am, I went ahead and 
looked at what BRLTTY+kmscon would entail. That resulted in the attached 
design document. The actual code implementing it would be quite simple 
and probably smaller than the document itself.

Comments welcome.


Nicolas
# Design: kmscon Screen Access for BRLTTY

## Status

Draft — 2026-03-30

## Authors

Nicolas Pitre <[email protected]> (BRLTTY community), with design
assistance from Claude (Anthropic)

## Summary

This document proposes an architecture for making kmscon — a userspace
replacement for the Linux kernel virtual terminal (VT) — accessible to
BRLTTY, the Linux braille display daemon. The design introduces a shared
memory segment for efficient screen content access and a Unix domain
socket for control operations and event signaling.

## Contents

- Background: kmscon, BRLTTY, and the problem being solved
- Design goals: efficiency, robustness, extensibility
- Architecture overview: two-channel design (shm + socket)
- Detailed design: shm format, TLV protocol, cell layout, screen export
  renderer, socket messages, keyboard input, session management,
  multiple clients, area highlighting
- Security: access control via socket permissions, peer credentials,
  anonymous shm
- Alternatives considered: five approaches evaluated and discarded
- Implementation roadmap: phased plan for both projects
- Open questions: items requiring further discussion

## Background

### kmscon

kmscon is a userspace terminal emulator that renders directly to
DRM/KMS display hardware, replacing the kernel's built-in virtual
console. It uses libtsm (Terminal-emulator State Machine) for all
terminal emulation and state management. kmscon supports multiple
sessions (analogous to kernel VTs), multiple display outputs, and
multiple font/rendering backends.

- Repository: https://github.com/kmscon/kmscon
- Build system: meson
- Key dependency: libtsm (https://github.com/kmscon/libtsm)

### BRLTTY

BRLTTY is a background daemon that provides braille display access to
the Linux console. It supports dozens of braille display hardware models
and includes multiple screen drivers for different environments (Linux
VT, tmux, AT-SPI2, Android, etc.).

- Website: http://brltty.app/
- Maintainer: Dave Mielke

### The problem

BRLTTY's Linux screen driver depends entirely on kernel VT interfaces:

| Interface | Purpose |
|-----------|---------|
| `/dev/vcsa[N]` | Read screen content (characters + VGA attributes), cursor position, dimensions |
| `/dev/vcsu[N]` | Read screen content as 32-bit Unicode codepoints (preferred over vcsa when available) |
| `/dev/tty0` + `VT_GETSTATE` | Query active VT number |
| `KDGETMODE` | Determine text vs. graphics mode |
| `TIOCSTI` | Inject keystrokes into the terminal |
| `TIOCL_SETSEL` | Text selection |
| `TIOCL_GETBRACKETEDPASTE` | Query bracketed paste mode |
| `GIO_UNIMAP` | Font-to-Unicode mapping (fallback when `/dev/vcsu` is not available) |

`/dev/vcsu` is the primary text interface on modern kernels. It provides
full Unicode codepoints (one `uint32_t` per cell) without requiring
font-to-Unicode reverse mapping via `GIO_UNIMAP`. However, it does not
carry attributes — BRLTTY reads those from `/dev/vcsa` alongside.
Double-width characters are represented by placing a zero-width space
(U+200B) in the continuation cell.

When kmscon replaces the kernel VT, **all of these interfaces
disappear**. BRLTTY's Linux screen driver becomes non-functional.
A new integration path is needed.

## Design goals

1. **Efficiency**: BRLTTY should only read the cells it needs (the
   braille window region), not the entire screen on every update.

2. **Event-driven**: BRLTTY should sleep when the screen is idle and
   wake only when content changes. No busy-polling.

3. **Robustness**: If BRLTTY crashes or disconnects, kmscon must
   continue to function normally with no user-visible impact.

4. **Extensibility**: The protocol must allow new metadata fields and
   capabilities to be added without breaking existing implementations.

5. **Minimal conversion overhead**: Data formats should be close to
   what each side uses natively to minimize per-cell transformation
   costs.

6. **No artificial limits**: The design must not assume maximum screen
   dimensions. Configurations such as 3840×2160 at 8×8 font (480×270
   cells) are realistic.

## Architecture overview

```
┌─────────────────────────────┐        ┌────────────────────────────┐
│           kmscon             │        │           BRLTTY            │
│                              │        │                             │
│  ┌────────────────────────┐  │        │  ┌───────────────────────┐  │
│  │ tsm_screen (libtsm)    │  │        │  │ Screen Driver "kmscon"│  │
│  │ terminal state machine  │  │        │  │ (new)                 │  │
│  └──────────┬─────────────┘  │        │  └───────┬───────────────┘  │
│             │                │        │          │                  │
│  ┌──────────▼─────────────┐  │  shm   │  ┌───────▼───────────────┐  │
│  │ Screen export renderer │  ├───────►│  │ shm reader            │  │
│  │ (new)                  │  │ mmap   │  │ (read-only)           │  │
│  └──────────┬─────────────┘  │        │  └───────────────────────┘  │
│             │                │        │                             │
│  ┌──────────▼─────────────┐  │ socket │  ┌───────────────────────┐  │
│  │ Control socket         │◄─┼───────►│  │ Control client        │  │
│  │ (new)                  │  │        │  │                       │  │
│  └────────────────────────┘  │        │  └───────────────────────┘  │
│                              │        │                             │
│  ┌────────────────────────┐  │        │                             │
│  │ uterm_input            │  │        │                             │
│  │ (keyboard via evdev)   │──┼────────┼──► key events offered      │
│  └────────────────────────┘  │        │    to BRLTTY first         │
└─────────────────────────────┘        └────────────────────────────┘
```

The design has two communication channels:

- **Shared memory**: Screen cell data (read-only for BRLTTY). This is
  the hot path — BRLTTY reads only the cells under its braille window
  directly from mapped memory with no IPC round-trip.

- **Unix domain socket**: Control operations, event notifications, and
  keyboard interaction. This carries low-volume, low-frequency messages.

## Detailed design

### Shared memory segment

#### Lifecycle

kmscon creates an anonymous shared memory segment (`memfd_create()`) for
each terminal session. The segment is sized to hold a fixed preamble, a
variable-length TLV header, and the cell array for the current screen
dimensions. The fd is passed to BRLTTY over the Unix socket at
connection time.

When the screen is resized, kmscon creates a **new** shm segment with
the new dimensions, populates it, and passes the new file descriptor to
BRLTTY over the socket. BRLTTY maps the new segment, then unmaps and
closes the old one. This avoids race conditions during resize — the old
segment remains valid until BRLTTY releases it.

This safety relies on standard file descriptor semantics: the underlying
memory is not freed until all file descriptors and mappings are closed.
The sequence is:

1. kmscon creates and populates the new shm segment.
2. kmscon sends the new segment's fd to BRLTTY over the socket.
3. kmscon closes its fd to the old segment.
4. BRLTTY receives the new fd and maps the new segment.
5. BRLTTY unmaps and closes the old segment — its memory is now freed.

#### Preamble (fixed format)

The first bytes of the shm segment are a fixed-size preamble:

```c
struct kmscon_shm_preamble {
    uint32_t magic;          /* Identifies this as a kmscon shm segment */
    uint16_t version;        /* Preamble/TLV framing version            */
    uint16_t header_size;    /* Total header size (preamble + TLV area) */
};
```

The `version` field only changes if the preamble or TLV framing format
itself changes. Adding new TLV fields does not require a version bump.

#### Consistency model

No locking is used on the shm segment. If BRLTTY reads cells while
kmscon is mid-write, the braille display may show a momentary
inconsistency. This is resolved by the socket notification mechanism:
kmscon sends a `Screen updated` notification after every write. If a
write occurs while BRLTTY is reading, the notification ensures BRLTTY
will re-read promptly.

This is the same model the Linux kernel uses for `/dev/vcs*`. The VT
layer clears a poll event flag when a read starts. If the screen changes
during the read, the flag is set again, causing `poll()` to return
immediately and trigger a re-read. No locking is involved — eventual
consistency is sufficient for display output.

The socket notification serves the same role as the kernel's poll event
flag:

1. BRLTTY blocks in `poll()` on the socket fd.
2. kmscon updates shm and sends a `Screen updated` notification.
3. BRLTTY wakes, drains the notification, and reads its braille window
   from shm.
4. If kmscon wrote to shm during step 3, it will have sent another
   notification — BRLTTY's next `poll()` returns immediately.
5. If nothing changed during the read, BRLTTY blocks until the next
   notification.

#### TLV format

Both the shm header and socket messages use the same TLV (type-length-
value) framing and share a unified type numbering space. Each entry is:

```c
struct kmscon_tlv_entry {
    uint16_t type;     /* Field type identifier         */
    uint16_t length;   /* Value length in bytes          */
    /* uint8_t value[length], padded to 4-byte alignment */
};
```

The `length` field contains the actual data size. The TLV walker is
responsible for rounding up to the next 4-byte boundary to find the
start of the next entry:

```
next_entry = current_value_ptr + ((length + 3) & ~3)
```

In the shm header, type `0x0000` is a sentinel marking the end of the
TLV area. On the socket, each message carries one or more TLV entries.

Each side skips types it does not recognize. kmscon can add new fields
without breaking older BRLTTY drivers. BRLTTY can look for a field and
gracefully handle its absence.

#### Type number ranges

| Range | Usage |
|-------|-------|
| `0x0001–0x00FF` | Shared data types — used in both the shm header (as a full state snapshot) and socket messages (as incremental updates). Parsed with the same code path. |
| `0x0100–0x01FF` | kmscon → BRLTTY control messages — socket only. |
| `0x0200–0x02FF` | BRLTTY → kmscon control messages — socket only. |

This unification means that, for example, a cursor position change can
be communicated as a single TLV entry (type `0x0002`) on the socket
rather than requiring a full shm header rewrite and a separate "screen
updated" notification. BRLTTY can apply the delta directly or re-read
from shm — whichever is more appropriate for the specific field.

#### Shared data types (0x0001–0x00FF)

| Type | Name | Size | Value format |
|------|------|------|-------------|
| `0x0001` | Screen dimensions | 4 | `uint16_t cols, uint16_t rows` |
| `0x0002` | Cursor position | 4 | `uint16_t col, uint16_t row` |
| `0x0003` | Cursor attributes | 4 | `uint32_t flags` (see below) |
| `0x0004` | Mouse pointer position | 4 | `uint16_t col, uint16_t row` |
| `0x0005` | Active session number | 2 | `uint16_t number` |
| `0x0006` | Cell array descriptor | 8 | `uint32_t offset, uint16_t cell_stride, uint16_t format_version` |

Cursor attributes flags:

| Bit | Attribute |
|-----|-----------|
| 0 | Visible |
| 1 | Blinking |
| 2–3 | Shape: 0 = default, 1 = block, 2 = underline, 3 = bar |
| 4 | Bracketed paste mode active |
| 5–31 | Reserved |

Bracketed paste mode is included in cursor attributes because it
describes how input behaves at the cursor — when active, injected text
must be wrapped in bracket escape sequences. BRLTTY checks this flag
before pasting clipboard content.

Currently libtsm only tracks cursor visibility (`TSM_SCREEN_HIDE_CURSOR`).
Cursor shape (DECSCUSR) and blink are not yet implemented in libtsm but
are standard terminal features likely to be added in the future. The
flags field accommodates these from the start.

The cell array descriptor tells BRLTTY where the cell data begins in
the shm segment and how large each cell is. If the cell format grows
in the future (e.g., to add hyperlink IDs or underline style), the
stride increases but older readers can safely ignore trailing bytes per
cell by relying on the stride value.

#### Cell format

Each cell in the array represents one screen position:

```c
struct kmscon_shm_cell {
    uint32_t codepoint;    /* Unicode codepoint (UCS-4)       */
    uint16_t flags;        /* Width + attribute flags          */
    uint8_t  fr, fg, fb;   /* Foreground color (RGB)           */
    uint8_t  br, bg, bb;   /* Background color (RGB)           */
};
```

12 bytes per cell, naturally aligned.

The `flags` field encodes both character width and text attributes:

| Bits | Attribute |
|------|-----------|
| 0 | Single-width |
| 1 | Double-width |
| 2 | Bold |
| 3 | Italic |
| 4 | Underline |
| 5 | Blink |
| 6 | Inverse (resolved — informational only) |
| 7 | Protect |
| 8–15 | Reserved |

Character width is obtained by masking the low two bits: `flags & 0x03`.
A normal character has bit 0 set (width = 1). A double-width character
has bit 1 set (width = 2). A continuation cell following a double-width
character has neither bit set (width = 0).

libtsm stores double-width characters (e.g., CJK ideographs) as a
primary cell with width 2 followed by a continuation cell with width 0.
The shm format preserves this representation directly. BRLTTY can use
the width for correct braille display alignment and cursor routing
without needing to inspect character values or peek at adjacent cells.

This is an improvement over the kernel VT's `/dev/vcsu` interface,
which represents double-width characters by placing a zero-width space
(U+200B) in the continuation cell — requiring the reader to recognize
the sentinel character value.

The RGB values are **always final resolved colors**. When inverse mode
is active, kmscon swaps foreground and background before writing to shm.
Color codes from libtsm's palette are resolved to RGB at write time.
BRLTTY receives ready-to-use colors with no palette lookup needed.

##### Mapping to BRLTTY's ScreenCharacter

The conversion from `kmscon_shm_cell` to BRLTTY's `ScreenCharacter` is
nearly trivial:

| shm cell field | ScreenCharacter field |
|---|---|
| `codepoint` | `text` (wchar_t) |
| `fr, fg, fb` | `color.foreground` (RGBColor) |
| `br, bg, bb` | `color.background` (RGBColor) |
| `flags & BOLD` | `color.isBold` |
| `flags & ITALIC` | `color.isItalic` |
| `flags & UNDERLINE` | `color.hasUnderline` |
| `flags & BLINK` | `color.isBlinking` |
| `flags & 0x03` | character width |
| (always) | `color.usingRGB = 1` |

This conversion is performed only for the cells within BRLTTY's current
braille window — typically 40 or 80 cells — not the entire screen.

Unlike other BRLTTY screen drivers (Linux VT, tmux) which must maintain
a shadow copy of the screen content in the driver's own memory, the
kmscon driver does not need one. The shm segment is persistently mapped
and always reflects the current screen state, so BRLTTY can read any
cell at any time without having to cache or refresh a local copy.

##### Mapping from libtsm's tsm_screen_attr

kmscon's screen export renderer receives per-cell data via the
`tsm_screen_draw()` callback. Writing to shm is similarly direct:

| draw callback parameter | shm cell field |
|---|---|
| `ch` (UCS-4 array, first element) | `codepoint` |
| `width` | `flags` bits 0–1 (1 << (width - 1) for width > 0, else 0) |
| `attr.fr, attr.fg, attr.fb` | `fr, fg, fb` |
| `attr.br, attr.bg, attr.bb` | `br, bg, bb` |
| `attr.bold, attr.italic, ...` | `flags` bits 2+ |

libtsm's `to_rgb()` function resolves palette color codes to RGB values
before the draw callback fires, so the RGB fields are always populated
regardless of whether the original color was specified as a palette
index, a 256-color code, or a true-color RGB value.

### Screen export renderer

kmscon's rendering architecture already supports multiple renderers per
terminal session. Each display gets its own renderer (bbulk for software
rendering, gltex for OpenGL), and `redraw_all()` iterates through all
of them, calling `tsm_screen_draw()` once per renderer.

The screen export renderer is a new renderer that participates in this
same cycle. Instead of drawing to a framebuffer, it writes cell data to
the shared memory segment. It uses libtsm's age tracking to skip cells
that have not changed since the last pass — the same optimization the
visual renderers use.

After updating the shm segment, the export renderer sends a `Screen
updated` notification on the control socket to wake BRLTTY.

### Unix domain socket

The control socket carries all operations that are not read-only screen
content access.

#### Connection and discovery

kmscon listens on a well-known Unix domain socket path (e.g.,
`/run/kmscon/session-$N.sock`). BRLTTY's kmscon screen driver connects
on startup.

Discovery follows the same pattern as BRLTTY's tmux driver: sensible
auto-discovery with optional explicit override via driver parameters.
The kmscon screen driver accepts:

- `socket` — explicit socket path, overriding auto-discovery.
- `session` — connect to a specific session number.

When no parameters are given, the driver looks for a kmscon socket
under the default path (`/run/kmscon/`) and connects to the active
session. This mirrors the tmux driver's behavior, where tmux
auto-discovers a session when no explicit session name is provided.

#### Message framing

Socket messages use the same TLV framing as the shm header. Each
message consists of one or more TLV entries. This means the same
parsing code handles both shm header fields and socket messages.

A socket message carrying a shared data type (0x0001–0x00FF) is an
incremental update — it carries the same entry that appears in the
shm header, with the current value. For example, a cursor move sends
a single type `0x0002` entry with the new position. BRLTTY can apply
it directly without re-reading the shm header.

#### Control types: kmscon → BRLTTY (0x0100–0x01FF)

| Type | Name | Size | Purpose |
|------|------|------|---------|
| `0x0100` | Screen updated | 0+ | Cell content in shm has changed. Optional payload: dirty rows bitmask. |
| `0x0101` | Resize | 4 | Screen resized. Payload: `uint16_t cols, uint16_t rows`. Shm fd passed out-of-band via `SCM_RIGHTS`. |
| `0x0102` | Session switched | 6 | Active session changed. Payload: `uint16_t session_number, uint16_t cols, uint16_t rows`. Shm fd passed out-of-band via `SCM_RIGHTS`. |
| `0x0103` | Key event offer | 7 | A key event for BRLTTY to accept. Payload: `uint16_t keycode, uint8_t value, uint32_t modifiers`. Same format as key event injection (type `0x0200`). |

#### Control types: BRLTTY → kmscon (0x0200–0x02FF)

| Type | Name | Size | Purpose |
|------|------|------|---------|
| `0x0200` | Key event injection | 8 | Inject a key event. Payload: `uint16_t keycode, uint8_t value, uint8_t padding, uint32_t modifiers`. Uses the Linux `input_event` keycode space. `value`: 0 = release, 1 = press, 2 = repeat. |
| `0x0201` | Character injection | 4 | Inject a Unicode character. Payload: `uint32_t codepoint` (UCS-4). Written directly to the PTY as UTF-8. Used for braille input, clipboard paste, etc. |
| `0x0202` | Key event accepted | 0 | BRLTTY claims the offered key event. |
| `0x0203` | Enable key offers | 0 | BRLTTY requests kmscon to start offering key events. |
| `0x0204` | Disable key offers | 0 | BRLTTY requests kmscon to stop offering key events. |
| `0x0205` | Highlight region | 8 | Set visual highlight. Payload: `uint16_t left, uint16_t top, uint16_t right, uint16_t bottom`. |
| `0x0206` | Unhighlight | 0 | Clear visual highlight. |
| `0x0207` | Mouse click | 5 | Simulate a mouse click. Payload: `uint16_t col, uint16_t row, uint8_t button`. |
| `0x0208` | Switch session | 2 | Request switch to session number. Payload: `uint16_t number`. |

Note: cursor routing is not a kmscon operation. Unlike a mouse click
which is a single event that the application interprets, cursor routing
is an iterative process. BRLTTY implements it as a state machine that
sends arrow key presses (via key injection, type `0x0200`) and monitors
the cursor position response (via type `0x0002` in shm or socket
updates). The routing algorithm handles overshooting, vertical scrolling
detection, timeouts, and multi-phase adjustment (vertical then
horizontal) entirely within BRLTTY. No dedicated cursor routing message
is needed.

#### Key events and character injection

The protocol distinguishes two forms of input injection:

- **Key event injection** (type `0x0200`): Carries a Linux keycode,
  press/release/repeat state, and modifier flags. The keycode and value
  fields use the same encoding as the Linux input subsystem (`struct
  input_event` from `<linux/input.h>`), ensuring key events can flow
  between kmscon, BRLTTY, and the kernel input layer without
  translation. kmscon processes injected key events through xkbcommon
  with the full keymap context (dead keys, compose sequences, keyboard
  layout), just as it would for a physical keypress. This is used for
  keyboard event bounceback and cursor routing (arrow keys).

- **Character injection** (type `0x0201`): Carries a Unicode codepoint
  (UCS-4). kmscon encodes it as UTF-8 and writes it directly to the PTY
  master, bypassing xkbcommon. This is used for braille keyboard input,
  clipboard paste, and any other source that produces text rather than
  key events.

This separation preserves the full fidelity of key events throughout
the pipeline. A bounced-back key sequence replays exactly as if the
keys were pressed on the physical keyboard. Character data goes straight
to the application without being reinterpreted through a keymap.

Bracketed paste wrapping, if needed, is handled by BRLTTY before
sending character injection messages, informed by the bracketed paste
mode flag in the cursor attributes (type `0x0003`, bit 4).

### Keyboard input

#### Ownership model

kmscon owns the keyboard. It opens evdev devices directly and processes
key events through libxkbcommon. This is critical for robustness: if
BRLTTY crashes, keyboard input continues to work with no interruption
or reconfiguration.

#### First-refusal protocol

Key event offering is opt-in. By default, kmscon processes all keyboard
input itself. A client sends `Enable key offers` (type `0x0203`) when it
has keyboard bindings that need interception, and `Disable key offers`
(type `0x0204`) when it no longer needs them. If no keyboard bindings
are defined, the client never enables key offers and there is zero
overhead on the keyboard path.

When multiple clients have key offers enabled, they form a chain.
kmscon offers key events to the first client in the chain. That client
either consumes the key (binding matched) or injects it back (via type
`0x0200`). Injected keys from the first client are offered to the
second client, and so on. The last client's injections reach the
terminal emulator. If a client disconnects, it is removed from the
chain.

For a single client, the flow is:

1. kmscon reads a key event from evdev.
2. kmscon sends a `Key event offer` (type `0x0103`) to the client with
   the linux keycode, press/release/repeat state, and modifier flags.
3. The client responds with `Key event accepted` (type `0x0202`).
4. The client either consumes the key or injects it back via type
   `0x0200`, at which point kmscon feeds it to the terminal emulator.

From kmscon's perspective, every offered key is accepted. The client
takes full ownership of the key and decides what to do with it.

The `Key event accepted` reply also serves as a liveness check. If a
client does not acknowledge a key offer within a reasonable timeout,
kmscon assumes the client is hung and removes it from the key offer
chain. This prevents a malfunctioning client from silently sitting on
all keyboard input. Subsequent key events are offered to the next
client in the chain, or processed directly by the terminal emulator
if no clients remain.

#### Key combination handling

BRLTTY implements its own key bindings, some of which are multi-key
combinations. When BRLTTY sees the beginning of a potential binding, it
accepts each key as it arrives. If subsequent keys complete the binding,
BRLTTY acts on it and the keys are consumed. If the combination does
not match any binding, BRLTTY sends the accumulated keys back to kmscon
as individual key event injection messages (type `0x0200`). kmscon
processes these through xkbcommon exactly as if they came from the
physical keyboard.

This mirrors BRLTTY's existing behavior with `EVIOCGRAB` and uinput,
where it grabs the keyboard, buffers potential binding keys, and
re-injects unmatched sequences through a virtual input device. The
latency characteristics are comparable.

### Session management

kmscon sessions map directly to BRLTTY's virtual terminal concept.
Each session has its own:

- `tsm_screen` instance (terminal state)
- shm segment (screen content)
- Active/inactive state

BRLTTY tracks which session is active via the `Active session number`
TLV field and the `Session switched` socket notification. When a session
switch occurs, kmscon sends the new session's shm fd over the socket.
BRLTTY's `currentVirtualTerminal()` returns the active session number,
and `switchVirtualTerminal()` sends a `Switch session` request.

### Multiple clients

kmscon supports multiple simultaneous accessibility client connections.
This is a common scenario during BRLTTY development and testing, where
two BRLTTY instances run with different braille displays against the
same console.

- **Shared memory**: Read-only for all clients. Multiple readers can
  mmap the same segment concurrently with no conflict.
- **Socket notifications**: kmscon sends screen update and session
  notifications to all connected clients.
- **Key injection and character injection**: Any client can inject
  input. kmscon processes injections from all clients identically.
- **Key offers**: Key events are cascaded through all clients that have
  enabled key offers, forming a chain. kmscon offers key events to the
  first client. Key injections from the first client are offered to the
  second client (instead of going directly to the terminal). Key
  injections from the second client are offered to the third, and so
  on. The last client in the chain injects into the terminal emulator.
  If no clients have key offers enabled, key events go directly to the
  terminal emulator. Clients that have not enabled key offers are
  skipped in the chain.
- **Highlight regions**: Each client may set its own highlight region.
  kmscon renders all active highlights simultaneously, allowing a
  sighted helper to see where each braille display is positioned on
  screen.

### Area highlighting

BRLTTY can visually mark the region under the braille window on screen,
allowing a sighted helper to see where the braille user is reading.
BRLTTY sends a `Highlight region` message with the bounding rectangle.
kmscon applies inverse or distinct color attributes to those cells
during visual rendering, similar to how libtsm already renders
selections via `attr.inverse` in the draw callback.
`Unhighlight` clears the region.

## Security

The shm segment exposes the full terminal content — including passwords
and sensitive data as they are displayed — and the control socket allows
keystroke injection. Access control is critical.

### Socket permissions

kmscon creates the Unix domain socket with restrictive ownership and
permissions. The socket is owned by root and a designated group (e.g.,
`brl`), with mode `0660`. Only processes running as root or in the `brl`
group can connect. The kernel enforces supplementary group membership on
`connect()`, so BRLTTY does not need to run as root — membership in the
`brl` group is sufficient.

This follows the same model BRLTTY already uses for access to
`/dev/input/*` devices and braille USB hardware, where udev rules
assign group ownership. BRLTTY runs as a system service (started at
boot, before any user logs in) with the appropriate group membership
configured via its systemd unit (`SupplementaryGroups=brl`).

### Peer credential verification

When a client connects, kmscon can retrieve the peer credentials via
`SO_PEERCRED` (`getsockopt` with `SOL_SOCKET`), which returns the
peer's uid, gid, and pid as reported by the kernel. Note that this only
provides the primary gid, not supplementary groups. It is therefore
useful for logging and uid-level policy, but the socket filesystem
permissions are the primary gate for group-based access control.

### Anonymous shared memory

The shm segments are created using `memfd_create()` rather than
`shm_open()`. This produces anonymous memory with no filesystem name —
the segment cannot be discovered or opened by other processes by name.
The only way to access it is to receive the file descriptor over the
authenticated socket connection.

This provides defense in depth: even if a process somehow learns that
a shm segment exists, it cannot open it without the fd. The fd is only
sent to clients that have passed the socket permission checks.

### Why not seat-based access control

systemd-logind provides seat-based access for devices like audio and
video hardware — granting access to the active session on a seat. While
kmscon is inherently seat-aware, this model does not apply to BRLTTY:
BRLTTY is a system service that must be running before any user session
exists (to enable login via braille display). It does not belong to any
logind session and cannot rely on seat-based grants.

Group-based access is the appropriate mechanism for system services that
need persistent access to console resources regardless of session state.

### Summary of access control layers

1. **Socket filesystem permissions**: group-based access control on the
   socket path, enforced by the kernel including supplementary groups.
2. **Peer credentials**: kmscon can verify the connecting process
   identity for logging and additional policy.
3. **Anonymous shm**: no filesystem name, fd only available to
   authenticated clients.

## Alternatives considered

### Alternative 1: Emulate /dev/vcsa from kmscon

kmscon could expose a virtual `/dev/vcsa*` device (via FUSE or a
character device) that mirrors the `tsm_screen` state. BRLTTY's
existing Linux screen driver would work without modification.

**Why we discarded this:**

- The vcsa format is limited — originally 8-bit characters plus VGA
  attributes. The unicode device (`/dev/vcsu*`) provides 32-bit
  codepoints but no attributes; BRLTTY must read both devices.
- vcsa only provides screen content. BRLTTY also needs VT ioctls
  (`VT_GETSTATE`, `KDGETMODE`, etc.) which go through `/dev/tty*`.
  Emulating this ioctl surface is large and fragile.
- BRLTTY's Linux driver polls vcsa with `POLLPRI`, which would need
  to be emulated.
- It forces a modern Unicode-native terminal into a legacy VGA model.
- Any vcsa reader would still receive the full screen on every read.

### Alternative 2: Socket-only protocol (no shared memory)

Inspired by BRLTTY's tmux driver, which communicates entirely through
pipes. kmscon could expose a control-mode protocol and BRLTTY would
request screen content over the socket.

**Why we discarded this:**

- The tmux driver performs a full `capture-pane` on every update,
  transferring the entire screen content even when only one character
  changed and even when the braille display shows a different region.
- For a primary console, this inefficiency is significant.
- Shared memory allows BRLTTY to read only the cells it needs (the
  braille window) with zero IPC overhead on the read path.
- We kept the socket for control operations and notifications, where
  its request/response model is appropriate.

### Alternative 3: BRLTTY owns the keyboard

BRLTTY grabs the keyboard via `EVIOCGRAB` as it does today, and
forwards unhandled keys to kmscon over the control socket.

**Why we discarded this:**

- If BRLTTY crashes, kmscon loses all keyboard input. The terminal
  becomes unusable until BRLTTY is restarted. This is a critical
  robustness failure for a system console.
- Similarly, BRLTTY's uinput-based re-injection creates a dependency:
  kmscon would read from a virtual device that disappears when BRLTTY
  exits.
- The chosen design (kmscon owns the keyboard, offers keys to BRLTTY)
  degrades gracefully: BRLTTY disconnection has zero impact on terminal
  usability.

### Alternative 4: Fixed maximum shm allocation

Allocate shared memory for the largest plausible screen (e.g., 256×256)
to avoid remapping on resize.

**Why we discarded this:**

- Real-world configurations exceed any reasonable "maximum". Users run
  3840×2160 displays with 8×8 fonts, yielding 480×270 cells.
  Future displays and smaller fonts could push this further.
- The fd-passing approach for resize (create new segment, pass fd over
  socket) is clean and imposes no artificial limits.

### Alternative 5: Shared library for terminal emulators and accessibility tools

Implement the screen export and access protocol as a shared library
that any terminal emulator could link against (writer side) and any
accessibility tool could link against (reader side).

**Why we discarded this:**

- Each terminal emulator has its own internal screen state
  representation. kmscon uses libtsm with a draw callback. Other
  emulators (foot, alacritty, kitty) have completely different
  architectures. A shared library would either impose a common internal
  format (invasive, unlikely to be accepted upstream) or provide an
  adapter layer that each emulator populates — which is essentially the
  shm cell format we already define, with extra ceremony.
- The writer side is a few hundred lines of straightforward code:
  create shm, write cells, send socket notifications. The reader side
  is similarly simple: mmap, parse TLV, read cells. Wrapping this in a
  library adds a dependency, a versioning burden, and an API surface to
  maintain for very little code reuse.
- A shared library introduces practical problems: packaging
  dependencies across distributions, ABI stability commitments, version
  skew between the library and its consumers, and a governance question
  about who maintains it.
- The real interoperability contract is the **protocol** — the shm
  format, TLV types, and socket message definitions documented in this
  specification. Any terminal emulator can implement the writer side
  and any accessibility tool can implement the reader side by following
  the spec, without depending on a shared library.
- Reference implementations or helper code can be provided as copyable
  source files or header-only utilities without the constraints of a
  shared library.

## Implementation roadmap

### Phase 1: kmscon side

1. **Screen export renderer**: New source file in `src/` implementing
   the shm writer as a `tsm_screen_draw()` consumer. Hooks into the
   existing `redraw_all()` cycle.

2. **Shared memory management**: Create/destroy anonymous shm segments
   (`memfd_create`) per session. Handle resize by creating new segments
   and passing fds over the socket.

3. **Control socket listener**: Unix domain socket accepting BRLTTY
   connections. Handle key injection, session switching, highlight
   requests.

4. **Keyboard first-refusal**: When BRLTTY enables key offers, offer
   key events before processing. Fall back to normal processing on
   disconnect.

5. **Build integration**: New meson option (e.g., `screen_export` or
   `accessibility`) to enable/disable the feature and its dependencies.

### Phase 2: BRLTTY side

1. **New screen driver**: `Drivers/Screen/Kmscon/screen.c` implementing
   the `BaseScreen` interface.

2. **shm reader**: Map the shm segment read-only. Implement
   `readCharacters()` by reading directly from the shm cell array.
   No shadow screen copy needed.

3. **Socket client**: Connect to kmscon's control socket. Register for
   async I/O notifications. Implement key event and character injection,
   session switching, highlight, and keyboard event handling.

4. **Driver registration**: Add to BRLTTY's screen driver list and
   build system.

### Phase 3: Refinements

- Dirty region hints in update notifications
- Negotiation of optional capabilities at connection time

## Open questions

1. **Protocol versioning and capability negotiation**: Should BRLTTY
   and kmscon exchange supported message types at connection time?

2. **libtsm changes**: Adding a direct cell-read API to libtsm (instead
   of relying solely on the draw callback) would simplify the export
   renderer. Is the libtsm maintainer open to this?

4. **Upstream acceptance**: Both projects need to agree on the protocol.
   A shared header file defining the shm and socket message formats
   would ensure consistency.
_______________________________________________
This message was sent via the BRLTTY mailing list.
To post a message, send an e-mail to: [email protected]
For general information, go to: http://brltty.app/mailman/listinfo/brltty

Reply via email to