This is an automated email from the ASF dual-hosted git repository. xiaoxiang781216 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/nuttx.git
commit 2ecd84b7b42c8eea7775e95578ee8229bbd8ac3d Author: Kai Cheng <[email protected]> AuthorDate: Tue Sep 15 21:34:04 2026 +0800 pty: let PIPEIOC_SETSIZE and PIPEIOC_GETSIZE through The pipes a pseudo terminal is built from can only be sized through CONFIG_PSEUDOTERM_RXBUFSIZE and CONFIG_PSEUDOTERM_TXBUFSIZE, so a single consumer that needs more than the 256 byte default makes every PTY in the system pay for it - including the ones /dev/ptmx hands out at run time, which never asked. The buffers are allocated eagerly, so this is real memory: raising both to 2048 costs 3584 bytes per pair. The pipe driver has supported PIPEIOC_SETSIZE and PIPEIOC_GETSIZE for a while, but they cannot reach those pipes: the default branch of pty_ioctl() is #if 0'd out and returns -ENOTTY. Forward the two of them to the sink pipe, which is the rule FIONWRITE and FIONSPACE already follow, so a file sizes the pipe it writes into and the two directions stay independent - both ends of a pair ask for themselves. This lets a program whose framing needs a larger pipe size only its own device. A write larger than the pipe is split into that many partial writes, which a framed protocol carried over the PTY may not tolerate. Drop the stale note while here: it lists what the pipe driver supported before PIPEIOC_PEEK, PIPEIOC_SETSIZE and PIPEIOC_GETSIZE were added. Signed-off-by: Kai Cheng <[email protected]> --- drivers/serial/pty.c | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/drivers/serial/pty.c b/drivers/serial/pty.c index 85d5cfc2d3a..4d1b3fddfec 100644 --- a/drivers/serial/pty.c +++ b/drivers/serial/pty.c @@ -1009,6 +1009,24 @@ static int pty_ioctl(FAR struct file *filep, int cmd, unsigned long arg) } break; + /* Resize (PIPEIOC_SETSIZE) or query the size of (PIPEIOC_GETSIZE) the + * sink pipe, following the same rule as FIONWRITE and FIONSPACE above. + * A file writing more than the pipe holds is split into that many + * partial writes, which a protocol carried over the PTY may not + * tolerate, so let it size the pipe it writes into rather than making + * every PTY in the system pay for the largest consumer through + * CONFIG_PSEUDOTERM_RXBUFSIZE and CONFIG_PSEUDOTERM_TXBUFSIZE. Both + * ends of a pair have to ask for themselves, which is what keeps the + * two directions independent. + */ + + case PIPEIOC_SETSIZE: + case PIPEIOC_GETSIZE: + { + ret = file_ioctl(&dev->pd_sink, cmd, arg); + } + break; + case FIONBIO: { ret = file_ioctl(&dev->pd_src, cmd, arg); @@ -1090,10 +1108,10 @@ static int pty_ioctl(FAR struct file *filep, int cmd, unsigned long arg) /* Any unrecognized IOCTL commands will be passed to the contained * pipe driver. * - * REVISIT: We know for a fact that the pipe driver only supports - * FIONREAD, FIONWRITE, FIONSPACE and PIPEIOC_POLICY. The first two - * are handled above and PIPEIOC_POLICY should not be managed by - * applications -- it can break the PTY! + * REVISIT: The pipe driver supports FIONREAD, FIONWRITE, FIONSPACE, + * PIPEIOC_PEEK, PIPEIOC_SETSIZE, PIPEIOC_GETSIZE and PIPEIOC_POLICY. + * All but the last are handled above; PIPEIOC_POLICY should not be + * managed by applications -- it can break the PTY! */ default:
