Re: [RFC] generic CAN/CAN FD susbsytem for RTEMS from scratch (LinCAN inspired)
Dear RTEMS developers, we have made a progress with our CAN stack and virtual/CTU CAN FD controller tests using standard x86-64 QEMU. We can now provide scripts that build the stack and our applications on i386-pc686 target and run RTEMS in QEMU. The actual CAN stack can still be found in our university GitLab https://gitlab.fel.cvut.cz/otrees/rtems/rtems-canfd and following steps build and run RTEMS in QEMU: $ export PATH=$PATH:/opt/rtems/6/bin $ cd targets/i386_pc686 $ ./setup-host-socketcan $ ./qemu-i386-pc686-2x-ctu-pci-build $ ./qemu-i386-pc686-2x-ctu-pci-run Support for i386_pc686 BSP is located in /opt/rtems/6 in this case. You can use following script to compile the BSP with required configuration setting. $ ./i386-rtems-sys.cfg (you might need to change RTEMS_DIR variable based on your location of RTEMS source code directory) RTEMS terminal in QEMU should come up after executing run command. You can try applications for both virtual controller and CTU CAN FD controller. The controllers have to be initialized and register which is done by can_register -t [target] command (target is either virtual or ctucanfd). Registered devices are assigned to test applications with can_set_test_dev [dev0] [dev1] command, where dev0 and dev1 are paths to character devices. Then test applications are can_1w (one way) and can_2w (two way). For example for CTU CAN FD # this registers two CTU CAN FD controllers under dev/can0 and dev/can1 SHLL [/] # can_register -t ctucanfd # assign dev/can0 and dev/can1 to test applications SHLL [/] # can_set_test_dev /dev/can0 /dev/can1 # run test applications SHLL [/] # can_1w SHLL [/] # can_2w All those steps are also described in project README or you can use help app command in RTEMS terminal to get further description of commands and their arguments. We want to rewrite some other controllers for our CAN stack to provide further tests and widen the support (SJA1000 would probably be the first one on our list), but currently the priority is the full completion of the stack itself (error reports, all required IOCTLs, etc.). Best wishes, Michal Lenc ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [RFC] generic CAN/CAN FD susbsytem for RTEMS from scratch (LinCAN inspired)
Hello Gedare, thanks for feedback. On Tuesday 05 of March 2024 22:54:35 Gedare Bloom wrote: > On Thu, Feb 29, 2024 at 6:40 AM Pavel Pisa wrote: > > On Tuesday 27 of February 2024 22:27:43 Gedare Bloom wrote: > > > On Mon, Feb 12, 2024 at 8:03 AM Pavel Pisa wrote: > > > > > > > > https://gitlab.fel.cvut.cz/otrees/rtems/rtems-canfd > > > > > On a quick review, my recommendation would be to switch to using the > rtems_mutex mechanism of the self-contained objects, see > https://docs.rtems.org/branches/master/c-user/self_contained_objects.html > > I believe this will be a more suitable solution than the more general > interrupt lock. It should be relatively straightforward to implement > this replacement for both the qends and the fifo structures. Each may > embed its own rtems_mutex. I see you already use a > rtems_mutex_recursive to protect the can_bus structure. I would avoid > a design that allows recursive access if possible. I did not see if > you acquire the lock during interrupt service routines. > > Since you have the interrupt_lock mechanism implemented, may be it > would be a nice experiment to evaluate the performance vs the > self-contained rtems_mutex. I agree with switch to self-contained rtems_mutex_lock and probably we try even rtems_binary_semaphore_post for notification. In the fact, this was my question in RFC whether consider self-contained objects for synchronization. By switch of queues protection to mutexes, we lose option to fill it directly from interrupt and implement driver Rx and Tx in interrupts. This could be problem on very slow targets with really short Rx FIFO. But that is not case of CTU CAN FD and even SJA1000 has 128 bytes HW FIFO. The code locking by interrupt disable is for really short time, so it should not be a problem. But we have decided to use chip dedicated thread on RTEMS from start to allow its priority setup even somewhere in middle to allow if there are even some more timing critical tasks running. On Linux kernel, we have used interrupt disabling spinlock to allow write LinCAN Rx and Tx handles directly in interrupt service routine. But that locking mechanism automatically switches to priority aware RT mutexes on the fully preemptive kernel. Anyway, we switch to rtems_mutex_lock on RTEMS, it makes code even simpler and do timing before and after switch ho have idea about behavior on Zynq target (which falls into middle or lower performance range today so it is appropriate for assessment if overhead is significant). Best wishes, Pavel -- Pavel Pisa phone: +420 603531357 e-mail: p...@cmp.felk.cvut.cz Department of Control Engineering FEE CVUT Karlovo namesti 13, 121 35, Prague 2 university: http://control.fel.cvut.cz/ personal: http://cmp.felk.cvut.cz/~pisa social: https://social.kernel.org/ppisa projects: https://www.openhub.net/accounts/ppisa CAN related:http://canbus.pages.fel.cvut.cz/ RISC-V education: https://comparch.edu.cvut.cz/ Open Technologies Research Education and Exchange Services https://gitlab.fel.cvut.cz/otrees/org/-/wikis/home ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [RFC] generic CAN/CAN FD susbsytem for RTEMS from scratch (LinCAN inspired)
On Thu, Feb 29, 2024 at 6:40 AM Pavel Pisa wrote: > > Hello Gedare > > On Tuesday 27 of February 2024 22:27:43 Gedare Bloom wrote: > > On Mon, Feb 12, 2024 at 8:03 AM Pavel Pisa wrote: > > > Michal Lenc works on a new generic CAN/CAN FD subsystem for RTEMS under > > > my supervision. The project has reached a phase where we will be very > > > grateful for the review and pointing to required changes to start a > > > discussion of possible inclusion into the RTEMS mainline. The project is > > > currently being developed as a separate RTEMS application based on the > > > OMK build > > > > > > https://gitlab.fel.cvut.cz/otrees/rtems/rtems-canfd > > > > > > But long-term intention is to move > > > > > > > > > https://gitlab.fel.cvut.cz/otrees/rtems/rtems-canfd/-/tree/master/lib/can > > >drv > > > > > > directory into RTEMS cpukit/dev/can directory and corresponding include > > > files into cpukit/include/dev/can directory. > > > > Thanks. I will review this design and code, but it will be a few weeks > > before I can do that. The directory location is a suitable choice. > > Thanks. I consider critical to discuse if queues access protection > by rtems_interrupt_lock_acquire is appropriate etc. I.e., I see > rtems_semaphore_obtain problematic because it would exhaust > fixed configures resources easily. I am not sure, if there is > rtems core / "kernel" equivalent to contained objects as are used > for "userspace" pthread_mutex_lock etc... There are more such architectural > questions where feedback from you, Joel, Sebastian and or Chis > with deeper knowledge would help. This discussion should continue > in public list or as issues setup for RTEMS CAN FD subsystem. > On a quick review, my recommendation would be to switch to using the rtems_mutex mechanism of the self-contained objects, see https://docs.rtems.org/branches/master/c-user/self_contained_objects.html I believe this will be a more suitable solution than the more general interrupt lock. It should be relatively straightforward to implement this replacement for both the qends and the fifo structures. Each may embed its own rtems_mutex. I see you already use a rtems_mutex_recursive to protect the can_bus structure. I would avoid a design that allows recursive access if possible. I did not see if you acquire the lock during interrupt service routines. Since you have the interrupt_lock mechanism implemented, may be it would be a nice experiment to evaluate the performance vs the self-contained rtems_mutex. Gedare ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [RFC] generic CAN/CAN FD susbsytem for RTEMS from scratch (LinCAN inspired)
Hello Gedare On Tuesday 27 of February 2024 22:27:43 Gedare Bloom wrote: > On Mon, Feb 12, 2024 at 8:03 AM Pavel Pisa wrote: > > Michal Lenc works on a new generic CAN/CAN FD subsystem for RTEMS under > > my supervision. The project has reached a phase where we will be very > > grateful for the review and pointing to required changes to start a > > discussion of possible inclusion into the RTEMS mainline. The project is > > currently being developed as a separate RTEMS application based on the > > OMK build > > > > https://gitlab.fel.cvut.cz/otrees/rtems/rtems-canfd > > > > But long-term intention is to move > > > > > > https://gitlab.fel.cvut.cz/otrees/rtems/rtems-canfd/-/tree/master/lib/can > >drv > > > > directory into RTEMS cpukit/dev/can directory and corresponding include > > files into cpukit/include/dev/can directory. > > Thanks. I will review this design and code, but it will be a few weeks > before I can do that. The directory location is a suitable choice. Thanks. I consider critical to discuse if queues access protection by rtems_interrupt_lock_acquire is appropriate etc. I.e., I see rtems_semaphore_obtain problematic because it would exhaust fixed configures resources easily. I am not sure, if there is rtems core / "kernel" equivalent to contained objects as are used for "userspace" pthread_mutex_lock etc... There are more such architectural questions where feedback from you, Joel, Sebastian and or Chis with deeper knowledge would help. This discussion should continue in public list or as issues setup for RTEMS CAN FD subsystem. > > We have reached a state where the virtual interface has been tested on an > > MZ_APO Xilinx/AMD Zynq-based board and an LX_CPU/LX_RoCoN NXP > > LPC4088-based board. The communication with a complete CAN interface has > > been successfully tested on the MZ_APO Xilinx/AMD Zynq board with CTU CAN > > FD IP cores implemented in FPGA. > > > > Building on other RTEMS BSP should be possible by changing a single > > definition in the config.target file > > > > RTEMS_MAKEFILE_PATH=/opt/rtems/6/arm-rtems6/xilinx_zynq_zedboard > > When ported to cpukit, this would become part of the spec/build opts. For sure, but we want to have easy way to build code against different targets when it is still standalone. > > Michal Lenc will submit work as his Master's Thesis this May > > so I hope that Gedare would be willing to take the thesis > > reviewer role as we have already discussed. > > Indeed I will. I think this work is exciting and needed. Thanks for confirmation. There is limited time window when we can steer his work direction. When he finishes then I hope that he will be helpfull and I want to keep an eye on the project and maintain it as I have done 20 years with LinCAN and helped SocketCAN etc., but the resources for deeper redesign will be limited until some another studnet is found or me or some company has funded project which would be willing to invest even into infrastructural work. You have asked bout XCAN in an another thread, so I reply there even to inform the public. XCAN is AMD/XilinX controller integrated onto Zynq chips. I remember that there has been invested into the driver at German Aerospace Center. Carlo Brokering has worked on it based on the Prashanth S work on Beagle Bone in the frame of GSoC. But the work on infrastructure and FIFOs was not finished from any side even that I have provided our LinCAN sources etc. and later the result has be removed as broken from mainline. https://lists.rtems.org/pipermail/devel/2023-March/074504.html We have XCAN on our Zynq board routed to CTU CAN FD, so it can be tested but porting to new infrastructure hardly fits to Michal Lenc's thesis plan. Another interesting platform and controllers would be xilinx-zynqmp and xilinx-versal. These have advantage that CAN FD controller emulation is included in QEMU mainline. For our project testing for developers without HW, we plan to add support for PCI mapped CTU CAN FD on some x86 RTEMS BSP, because we have that support included in QEMU mainline. I have tried to provide even platform bus (AMBA) mapping for Xilinx Zynq as an experimental patch for QEMU https://github.com/ppisa/qemu/commit/2bcfd7b8dfd7d657ada2f8ec2b972f6469b33c94 Mmaped IO works but interrupt mapping is not working and I am not sure how it should be done correctly for dynamically plugable sysbus device yet. Best wishes, Pavel -- Pavel Pisa phone: +420 603531357 e-mail: p...@cmp.felk.cvut.cz Department of Control Engineering FEE CVUT Karlovo namesti 13, 121 35, Prague 2 university: http://control.fel.cvut.cz/ personal: http://cmp.felk.cvut.cz/~pisa social: https://social.kernel.org/ppisa projects: https://www.openhub.net/accounts/ppisa CAN related:http://canbus.pages.fel.cvut.cz/ RISC-V education: https://comparch.edu.cvut.cz/ Open Technologies Research Education and Exchange Services
Re: [RFC] generic CAN/CAN FD susbsytem for RTEMS from scratch (LinCAN inspired)
On Mon, Feb 12, 2024 at 8:03 AM Pavel Pisa wrote: > > Dear RTEMS community, the core developers team, > > Michal Lenc works on a new generic CAN/CAN FD subsystem for RTEMS under my > supervision. The project has reached a phase where we will be very grateful > for the review and pointing to required changes to start a discussion of > possible inclusion into the RTEMS mainline. The project is currently being > developed as a separate RTEMS application based on the OMK build > > https://gitlab.fel.cvut.cz/otrees/rtems/rtems-canfd > > But long-term intention is to move > > https://gitlab.fel.cvut.cz/otrees/rtems/rtems-canfd/-/tree/master/lib/candrv > > directory into RTEMS cpukit/dev/can directory and corresponding include files > into cpukit/include/dev/can directory. > Thanks. I will review this design and code, but it will be a few weeks before I can do that. The directory location is a suitable choice. > We have reached a state where the virtual interface has been tested on an > MZ_APO Xilinx/AMD Zynq-based board and an LX_CPU/LX_RoCoN NXP LPC4088-based > board. The communication with a complete CAN interface has been successfully > tested on the MZ_APO Xilinx/AMD Zynq board with CTU CAN FD IP cores > implemented in FPGA. > > Building on other RTEMS BSP should be possible by changing a single definition > in the config.target file > > RTEMS_MAKEFILE_PATH=/opt/rtems/6/arm-rtems6/xilinx_zynq_zedboard > When ported to cpukit, this would become part of the spec/build opts. > To test with the controller, we can provide FPGA bitstream, which should fit > Zed/MicroZed or other XilinX Zynq 7Z010 based boards, and thanks to the > source availability of CTU CAN FD core, it can be ported to other FPGA+SoC > platforms. We have QEMU CTU CAN FD PCI integration emulation so that > can be other target for continuous integration testing even on x86, but > PCI/PCIe mapping has to be added to the RTEMS driver code. > > The API to the driver is provided in the form of the POSIX character device. > Each CAN controller (chip) is registered as a node into the "/dev" namespace > by name (i.e., "can0", "can1", ... ). It can be opened multiple times, and > each instance of the open can be accessed from multiple threads. Written CAN > frames are posted into the input end of the queue (FIFO), which is connected > to the CAN controller (chip) or other frame processor on the output side. The > processor takes frames from FIFOs according to their priority class and > transmits them to the network. When the frame is successfully sent, it is > echoed back to all queues back to open file instances except the sending one > (that filtering is configurable). The filtering is implemented in > canque_filter_frame2edges() function. Received frames are filtered to all > queues to applications ends of the queues, which filter matches the CAN > identifier and frame type. > > The design is based on FIFOs designed closely to support needs to provide > service optimal to CAN frames delivery. On the controller side, > multiple slots, each with one CAN frame, can be taken from FIFO and kept > until the transmission is finished. Then, the frame from the slot is > distributed to inform even local clients about the frame that was sent. > The framework has a unique feature to allow pushback slots (frames) when some > later scheduled low-priority frame occupies the hardware Tx buffer, which is > urgently demanded for a higher priority pending frame to be sent. Then the > frame push back to the FIFO head is delivered for Tx processing again. > > We present (in the prepared article) how our design resolves bus arbitration > priority inversion, which is often a problem for generic CAN drivers and > subsystems, which allows access from more applications than is the fixed > Tx buffers count. With a well-behaved controller, we are able to keep > all available Tx hardware buffers filled by the right frames instantly. > When higher priority class frame is released by the application into > queues, the controller replaces the latest lower priority frames > with higher priority one and places it in the sequence after all > frames with the same priority class but before all with lower priority. > > We will present the idea in the prepared article at the > international CAN Conference in May > https://www.can-cia.org/icc/ > Thanks, I will read the article to see if I still have questions. > The design implementation choice is to use CAN-specific FIFOS queues. > The consistency is protected by rtems_interrupt_lock_acquire / > rtems_interrupt_lock_release, and thanks to reference counting > of users iterating with given queue from both its ends, the locks > are released during most of the operations and are held > for a really short time. We consider rtems_interrupt_lock > appropriate thanks to low overhead when we keep it > for limited code sequences. But switching to semaphores > or some POSIX mutexes is possible. But we also want to keep > code usable on
[RFC] generic CAN/CAN FD susbsytem for RTEMS from scratch (LinCAN inspired)
Dear RTEMS community, the core developers team, Michal Lenc works on a new generic CAN/CAN FD subsystem for RTEMS under my supervision. The project has reached a phase where we will be very grateful for the review and pointing to required changes to start a discussion of possible inclusion into the RTEMS mainline. The project is currently being developed as a separate RTEMS application based on the OMK build https://gitlab.fel.cvut.cz/otrees/rtems/rtems-canfd But long-term intention is to move https://gitlab.fel.cvut.cz/otrees/rtems/rtems-canfd/-/tree/master/lib/candrv directory into RTEMS cpukit/dev/can directory and corresponding include files into cpukit/include/dev/can directory. We have reached a state where the virtual interface has been tested on an MZ_APO Xilinx/AMD Zynq-based board and an LX_CPU/LX_RoCoN NXP LPC4088-based board. The communication with a complete CAN interface has been successfully tested on the MZ_APO Xilinx/AMD Zynq board with CTU CAN FD IP cores implemented in FPGA. Building on other RTEMS BSP should be possible by changing a single definition in the config.target file RTEMS_MAKEFILE_PATH=/opt/rtems/6/arm-rtems6/xilinx_zynq_zedboard To test with the controller, we can provide FPGA bitstream, which should fit Zed/MicroZed or other XilinX Zynq 7Z010 based boards, and thanks to the source availability of CTU CAN FD core, it can be ported to other FPGA+SoC platforms. We have QEMU CTU CAN FD PCI integration emulation so that can be other target for continuous integration testing even on x86, but PCI/PCIe mapping has to be added to the RTEMS driver code. The API to the driver is provided in the form of the POSIX character device. Each CAN controller (chip) is registered as a node into the "/dev" namespace by name (i.e., "can0", "can1", ... ). It can be opened multiple times, and each instance of the open can be accessed from multiple threads. Written CAN frames are posted into the input end of the queue (FIFO), which is connected to the CAN controller (chip) or other frame processor on the output side. The processor takes frames from FIFOs according to their priority class and transmits them to the network. When the frame is successfully sent, it is echoed back to all queues back to open file instances except the sending one (that filtering is configurable). The filtering is implemented in canque_filter_frame2edges() function. Received frames are filtered to all queues to applications ends of the queues, which filter matches the CAN identifier and frame type. The design is based on FIFOs designed closely to support needs to provide service optimal to CAN frames delivery. On the controller side, multiple slots, each with one CAN frame, can be taken from FIFO and kept until the transmission is finished. Then, the frame from the slot is distributed to inform even local clients about the frame that was sent. The framework has a unique feature to allow pushback slots (frames) when some later scheduled low-priority frame occupies the hardware Tx buffer, which is urgently demanded for a higher priority pending frame to be sent. Then the frame push back to the FIFO head is delivered for Tx processing again. We present (in the prepared article) how our design resolves bus arbitration priority inversion, which is often a problem for generic CAN drivers and subsystems, which allows access from more applications than is the fixed Tx buffers count. With a well-behaved controller, we are able to keep all available Tx hardware buffers filled by the right frames instantly. When higher priority class frame is released by the application into queues, the controller replaces the latest lower priority frames with higher priority one and places it in the sequence after all frames with the same priority class but before all with lower priority. We will present the idea in the prepared article at the international CAN Conference in May https://www.can-cia.org/icc/ The design implementation choice is to use CAN-specific FIFOS queues. The consistency is protected by rtems_interrupt_lock_acquire / rtems_interrupt_lock_release, and thanks to reference counting of users iterating with given queue from both its ends, the locks are released during most of the operations and are held for a really short time. We consider rtems_interrupt_lock appropriate thanks to low overhead when we keep it for limited code sequences. But switching to semaphores or some POSIX mutexes is possible. But we also want to keep code usable on RTEMS build even without POSIX API enabled, and classical API semaphores have significant size and execution time overhead. We have switched to BSD Tail Queues (TAILQ) to keep lists of the queues (FIFOs) coming to/from a given frame processor node because they are provided even by NewLib and make code portable #include The API is much harder to use than Linux struct list_head and supporting functions. My preferred alternative is my uLUt
