Acfboy opened a new issue, #18566: URL: https://github.com/apache/nuttx/issues/18566
### Description *This document serves as my proposal to Apache NuttX for the 2026 Google Summer of Code.* **Author:** Wu Chenyang **Email:** [email protected] ## Motivation Over the past two decades, X11 has dominated the Linux graphics environment. A vast number of graphics libraries, UI libraries, and GUI applications have been written on top of X11. If we could allow some X11-based programs to run on NuttX, we'd be able to provide embedded devices with much more powerful functionality and a more convenient development approach. For instance, running the Dillo browser to display richer manuals; running BDReader for e-books, reducing the hardware cost of e-book readers, and so on. What's more, this idea is completely feasible—all the software mentioned above is based on FLTK, and NanoX, the X11-compatible project for embedded devices, can run FLTK just fine. Therefore, implementing this project would significantly enhance NuttX's functionality, scope of application, and user experience. ## Implementation Analysis ### Current Graphics System The current graphics system can be divided into three layers: 1. Low-level drivers 2. Simple wrappers in nxglib 3. Upper-level systems like nxtk/nxmu/nxwm The low-level drivers come in two flavors: framebuffer and lcd. Framebuffer is the standard Linux abstraction, mapping every pixel color on screen to a position in the file `/dev/fb0`. You can change each position's color through file operations, or use mmap to write colors more efficiently. The lcd driver provides interfaces via ioctl for writing a row or a rectangle. nxglib provides interfaces for drawing rectangles, moving rectangles, copying rectangles, and drawing trapezoids at various color depths. However, it doesn't involve rasterization of complex graphics—this layer is relatively simple. nxmu/nxtk, on the other hand, provides X11-like functionality: programs use the nxtk library, connect to the nx server through a message queue (instead of socket), register callbacks for window redraws and window changes, and can create windows and draw in them. But the interfaces differ significantly from X11, and it lacks important features like client framebuffer mapping and JPEG image loading. nxmu is built on top of nxglib. Above that sits nxwm as an application providing the desktop environment. ### NanoX NanoX consists of several layers. At the very top is the X11-compatible library for users, which handles communication between user programs and the nanox server. Since X11 was designed so the server and client don't have to be on the same machine, and since NuttX's message queue isn't as heavy as Linux's—whereas unix domain sockets require two pipes and creating system nodes, potentially incurring more overhead—I think we could change it to use mq for communication, just like nxtk does. The bottom layer handles framebuffer operations and mouse/keyboard hardware driver adaptation. Framebuffer is the standard approach that NuttX also supports, but in embedded environments there might be some devices unsuitable for fb, such as the lcd driver already in the code. I think a better approach here would be to build it on top of nxglib—just like the original nxmu does—while keeping NanoX's rendering engine, using nxglib only as a driver wrapper. For mouse and keyboard, both read directly from system calls, though we'd need to change `/dev/input/mice` to `/dev/mouse0` and replace the byte-reading in nanox with reading structures. Not much modification is needed. For touchscreen support, we'd need to replace the tslib dependency with NuttX's touchscreen driver support. nanox also provides nanowm as a desktop environment, where the start menu functionality needs adaptation to launch programs within NuttX. The other parts are implementations of various nanox interfaces, which we don't need to worry about. ### Summary of Required Modifications 1. Replace direct read/write of `/dev/fb0` with using the nxglib library. 2. Make small adjustments to mouse and keyboard information. 3. Replace socket in the nanox user library with mq. 4. porting other libs used in dillo and nanowm. Of course, there's also handling the build system so users can easily enable nanox support, testing nanox support with Dillo to verify completeness and reliability, and finally testing on real hardware. ## Proposed Timeline | Period | Task | Verification Goal | Notes | |--------|------|-------------------|-------| | Before May 25 (before coding officially starts) | Learn and understand the build system | Be able to add new configurations and components to the build system, trim nanox so it compiles in both flat and protect modes | NuttX's build configuration is quite complex; apart from the code, providing convenient configuration is easily overlooked but essential for actually porting nanox support. | | Before May 25 | Purchase ESP32-S3-8048S070 development board and screen | Run nxlines on it as a test. | Used for later hardware testing. If by then NuttX mainline still doesn't support the 7-inch version, I'm willing to contribute. | | May 25 to June 7 | Replace socket with mq in nanox user library | Write a test program for various drawing operations to test; at this stage, trim nanox and print drawing commands to terminal instead of hardware | | | June 8 to June 21 | Replace fb operations with nxglib | Write test programs using X11 interfaces to draw lines and windows on screen | | | June 21 to July 6 | Adapt keyboard, mouse, touchscreen, port FLTK | Write various pages based on FLTK, test the usability of the X11 port. | This is also the mid-term evaluation | | July 7 to July 25 | Port Dillo browser and nanowm, resolve dependency library porting | Open Dillo from the desktop start menu to display NuttX's Wiki page | Need to make nanovm's nxlaunch recognize and open NuttX applications through the configuration system | | July 26 to August 17 | Fix bugs, improve documentation, improve build system configuration support, optimize performance, complete the project | Complete project delivery | | ## Contributor Qualifications I have solid fundamentals in computer science, a strong foundation in algorithms and data structures, and practical software engineering experience. I won the Gold Medal at the ICPC Asia Regional Contest and the First Prize at the Northwestern China Regional of China's Operating System Kernel Implementation Competition. I actively contribute to open-source communities. I've deeply studied Apache DataFusion and fixed several issues I found. I've also implemented features for other open-source projects. I'm familiar with software development and collaboration standards. As a sophomore, I don't have research pressure or heavy coursework, and I have a strong interest in computer systems, giving me enough time and enthusiasm to complete this task. ## Anticipated Challenges - Embedded devices have limited memory; running nanox + nanowm + dillo might have performance issues that need optimization. - nxglib drawing differs from fb's direct memory mapping—for example, you can't directly read a pixel's color, which may require extra overhead for maintaining state. If this doesn't work, we may need to write an adaptation layer or configuration that operates fb when possible and handles lcd as best as possible when it can't. - Socket is a streaming protocol; changing to mq to send individual packets—how large should each packet be, where to truncate, and how to synchronize and reconstruct—might require experimentation. ## Bonus - Support TrueType fonts for better display quality on the 7-inch screen. - Try porting more X11-based applications. ### Verification - [x] I have verified before submitting the report. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
