Re: [casper] Re: KATCP server configuration

2024-12-08 Thread Marc 
Hello

Not sure if this is useful:

There should be a "?client-list" command on the server telling you how many
connections are active.

tcpborphserver.h defines TBS_MAX_CLIENTS to 32, though I remember there
also being an earlier (or was it a  different ?) limit of 4 too - perhaps
some repositories have that value ?

regards

marc



On Sun, Dec 8, 2024 at 4:53 PM Ken Semanov  wrote:

> I need to know everything about what is happening in the PS side with the
> katcp server. Otherwise I cannot email the author of the katcp package to
> ask questions about it. He will need to know exactly what we are doing with
> CASPERized Linux. This is already going to be difficult
>  ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 
> ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌
>
> 
> CGBANNERINDICATORI need to know everything about what is happening in the
> PS side with the  katcp server. Otherwise I cannot email the author of the
> katcp package to ask questions about it.  He will need to know exactly what
> we are doing with CASPERized Linux.   This is already going to be difficult
> to communicate to him as it is.
>
> On Sunday, December 8, 2024 at 11:46:07 AM UTC-5 Ken Semanov wrote:
>
>> The Python version of the server side of KATCP is shown below.   (there
>> may be a C version somewhere)
>> .
>> https://github.com/ska-sa/katcp-python/blob/master/katcp/server.py
>> 
>>
>> This uses futures from concurrent.fut

Re: [casper] Re: KATCP server configuration

2024-12-08 Thread Kaj Wiik 

Hi Ken,

Yes, it is C. There is a version of KATCP server in Python too. In a 
nutshell the server is reading memory addresses of the BRAM areas from 
.dtbo and mmapping them to variables that are sent via KATCP.


Cheers,
Kaj


On 8.12.2024 23.07, Ken Semanov wrote:

Correction to earlier claim :  tcpborphserver is written in C.
The -b switch is specifying BOF files.  These .bin and .dtbo looks like 
board-specific bitstreams.  (not Beacon Object Files)


:~$ ps aux | grep tcp
root         497  1.4  0.0   5728  3032 ?        Ss   19:03   1:10 /bin/ 
tcpborphserver3 -b /lib/firmware
casper       872  0.0  0.0   5832   640 ttyPS0   S+   20:27   0:00 grep 
--color=auto tcp



=


:~$ ls -lathr /lib/firmware/
total 33M
drwxr-xr-x. 47 root root 4.0K Apr 27  2021 ..
-rwx--   1 root root 2.0K Jul 27  2021 250M_PL_125M_SYSREF_10M.txt
-rwx--   1 root root 2.0K Dec 24 15:48 extn250M_PL_125M_SYSREF_10M.txt
drwxr-xr-x.  2 root root 4.0K Dec 26 20:20 .
-rwx--   1 root root  33M May 25 19:56 tcpborphserver.bin
-rwx--   1 root root 2.3K May 25 19:56 tcpborphserver.dtbo


===

:~$ cat /etc/systemd/system/tcpborphserver.service
[Unit]
Description=TCPBorphServer allows programming and communication with the 
FPGA

Wants=network.target
After=syslog.target network-online.target

[Service]
Type=simple
ExecStart=/bin/tcpborphserver3 -b /lib/firmware
Restart=on-failure
RestartSec=10
KillMode=process

[Install]
WantedBy=multi-user.target


==

:~$ ps aux | grep pyth
root         484  0.0  0.4  28860 16780 ?        Ss   19:03   0:00 /usr/ 
bin/python3 /usr/bin/networkd-dispatcher --run-sts
casper       884  0.0  0.0   5832   660 ttyPS0   S+   20:35   0:00 grep 
--color=auto pyth

:~$ cat /usr/bin/networkd-dispatcher
#! /usr/bin/python3
# networkd-dispatcher
#   Dispatcher service for systemd-networkd
# Copyright(c) 2016 by wave++ "Yuri D'Elia" 
# Distributed under GPLv3+ (see COPYING) WITHOUT ANY WARRANTY.
# Copyright(c) 2018 by craftyguy "Clayton Craft" 
# Distributed under GPLv3+ (see COPYING) WITHOUT ANY WARRANTY.


from __future__ import print_function, division, generators, 
unicode_literals


On Sunday, December 8, 2024 at 12:57:51 PM UTC-5 Kaj Wiik wrote:

Hi Ken,

Here's the repository of the tcpborphserver3:
https://gitlab.ras.byu.edu/alpaca/casper/katcp/-/tree/master/
tcpborphserver3 

There are a number of branches that support e.g. setting the RFDC NCO.

We will have a similar need to stream lowish-speed polarimetry data
using the 1 GbE interface. I plan not to do the data readout using
KATCP but mapping the BRAM addresses directly to Julia data
structures, forming HDF5 files in ramdisk and copying them to a NFS
mounted server disk. Let's see how it goes ;-), I have to design the
analog interface first :-D.

Cheers and good luck,
Kaj

On Sun, 8 Dec 2024 at 18:53, Ken Semanov  wrote:

I need to know everything about what is happening in the PS side
with the  katcp server. Otherwise I cannot email the author of
the katcp package to ask questions about it.  He will need to
know exactly what we are doing with CASPERized Linux.   This is
already going to be difficult to communicate to him as it is.

On Sunday, December 8, 2024 at 11:46:07 AM UTC-5 Ken Semanov wrote:

The Python version of the server side of KATCP is shown
below.   (there may be a C version somewhere)
.
https://github.com/ska-sa/katcp-python/blob/master/katcp/
server.py 

This uses futures from concurrent.futures  and  tornado
futures.  As some may already know,  "threading" in Python
is fake.  All threads stall all other threads due to a
Global Interpreter Lock  (GIL). This is exactly the
behavior that I am seeing from network speed tests.   One of
my own processes can retrieve from a Snapshot at 2.1
Megabytes/sec.     However, for 8 concurrent processes,  I
only see  6.1 Megabytes/sec.  It is exactly as if the
parallel connections refuse to stack up correctly.  If the
CASPERized Linux on the PS of the board side is running
Python Katcp (and not the C version)  then it would be
stalling connections until the GIL allows them into the
single thread-of-execution.

Can anyone on this forum confirm any of this?   What is
happening on the CASPERized Linux image?  Where could I see
this for myself?   What processes in ps aux might I see
related to  katcp servers on the PS side?



On Sunday, December 8, 2024 at 12:26:26 AM UTC-5 Ken Semanov
wrote:

Is th

Re: [casper] Re: KATCP server configuration

2024-12-08 Thread Ken Semanov 
Correction to earlier claim :  tcpborphserver is written in C.  
The -b switch is specifying BOF files.  These .bin and .dtbo looks like 
board-specific bitstreams.  (not Beacon Object Files)  

:~$ ps aux | grep tcp
root 497  1.4  0.0   5728  3032 ?Ss   19:03   1:10 
/bin/tcpborphserver3 -b /lib/firmware
casper   872  0.0  0.0   5832   640 ttyPS0   S+   20:27   0:00 grep 
--color=auto tcp
 
 
=

 
:~$ ls -lathr /lib/firmware/
total 33M
drwxr-xr-x. 47 root root 4.0K Apr 27  2021 ..
-rwx--   1 root root 2.0K Jul 27  2021 250M_PL_125M_SYSREF_10M.txt
-rwx--   1 root root 2.0K Dec 24 15:48 extn250M_PL_125M_SYSREF_10M.txt
drwxr-xr-x.  2 root root 4.0K Dec 26 20:20 .
-rwx--   1 root root  33M May 25 19:56 tcpborphserver.bin
-rwx--   1 root root 2.3K May 25 19:56 tcpborphserver.dtbo
 
 
===
 
:~$ cat /etc/systemd/system/tcpborphserver.service
[Unit]
Description=TCPBorphServer allows programming and communication with the 
FPGA
Wants=network.target
After=syslog.target network-online.target
 
[Service]
Type=simple
ExecStart=/bin/tcpborphserver3 -b /lib/firmware
Restart=on-failure
RestartSec=10
KillMode=process
 
[Install]
WantedBy=multi-user.target
 
 
==
 
:~$ ps aux | grep pyth
root 484  0.0  0.4  28860 16780 ?Ss   19:03   0:00 
/usr/bin/python3 /usr/bin/networkd-dispatcher --run-sts
casper   884  0.0  0.0   5832   660 ttyPS0   S+   20:35   0:00 grep 
--color=auto pyth
:~$ cat /usr/bin/networkd-dispatcher
#! /usr/bin/python3
# networkd-dispatcher
#   Dispatcher service for systemd-networkd
# Copyright(c) 2016 by wave++ "Yuri D'Elia" 
# Distributed under GPLv3+ (see COPYING) WITHOUT ANY WARRANTY.
# Copyright(c) 2018 by craftyguy "Clayton Craft" 
# Distributed under GPLv3+ (see COPYING) WITHOUT ANY WARRANTY.
 
 
from __future__ import print_function, division, generators, 
unicode_literals

On Sunday, December 8, 2024 at 12:57:51 PM UTC-5 Kaj Wiik wrote:

> Hi Ken,
>
> Here's the repository of the tcpborphserver3:
>
> https://gitlab.ras.byu.edu/alpaca/casper/katcp/-/tree/master/tcpborphserver3
>
> There are a number of branches that support e.g. setting the RFDC NCO.
>
> We will have a similar need to stream lowish-speed polarimetry data using 
> the 1 GbE interface. I plan not to do the data readout using KATCP but 
> mapping the BRAM addresses directly to Julia data structures, forming HDF5 
> files in ramdisk and copying them to a NFS mounted server disk. Let's see 
> how it goes ;-), I have to design the analog interface first :-D.
>
> Cheers and good luck,
> Kaj
>
> On Sun, 8 Dec 2024 at 18:53, Ken Semanov  wrote:
>
>> I need to know everything about what is happening in the PS side with 
>> the  katcp server. Otherwise I cannot email the author of the katcp package 
>> to ask questions about it.  He will need to know exactly what we are doing 
>> with CASPERized Linux.   This is already going to be difficult to 
>> communicate to him as it is.  
>>
>> On Sunday, December 8, 2024 at 11:46:07 AM UTC-5 Ken Semanov wrote:
>>
>>> The Python version of the server side of KATCP is shown below.   (there 
>>> may be a C version somewhere) 
>>> .
>>> https://github.com/ska-sa/katcp-python/blob/master/katcp/server.py
>>>
>>> This uses futures from concurrent.futures  and  tornado futures.  As 
>>> some may already know,  "threading" in Python is fake.  All threads stall 
>>> all other threads due to a Global Interpreter Lock  (GIL). This is 
>>> exactly the behavior that I am seeing from network speed tests.   One of my 
>>> own processes can retrieve from a Snapshot at 2.1 Megabytes/sec. 
>>> However, for 8 concurrent processes,  I only see  6.1 Megabytes/sec.  It is 
>>> exactly as if the parallel connections refuse to stack up correctly.  If 
>>> the CASPERized Linux on the PS of the board side is running Python Katcp 
>>> (and not the C version)  then it would be stalling connections until the 
>>> GIL allows them into the single thread-of-execution. 
>>>
>>> Can anyone on this forum confirm any of this?   What is happening on the 
>>> CASPERized Linux image?  Where could I see this for myself?   What 
>>> processes in ps aux might I see related to  katcp servers on the PS side? 
>>>
>>>
>>>
>>>
>>> On Sunday, December 8, 2024 at 12:26:26 AM UTC-5 Ken Semanov wrote:
>>>
 Is there a way I could nose around the katcp server on the board side 
 in order to see how it is configured?   

 Could I reconfigure this server after booting into the board? 

 I am using a CASPERized image on a ZCU216.  I am migrating data out of 
 snapshot blocks over network.  After several network tests, it appears 
 that 
 the katcp on the board side (PS)  is artificially limited to 3 or 4 
 simultaneous network connections over RJ45 ethernet.   My use-case needs 
 this to be more like 8 to 16 connections.  I may also need to change the 
 packet size. 

 Thanks. 

>>> -- 
>> You received

Re: [casper] Re: KATCP server configuration

2024-12-08 Thread Ken Semanov 
Kaj Wiik,

Hello. Thank you for the links. Do you happen to know what is happening 
with the network on the PS/board side of a CASPERized Linux image?  

I think if I could discover how the server is getting data from a Snapshot 
block on the FPGA fabric -- then I could take it from there, and mock up a 
server to fit our needs. 
On Sunday, December 8, 2024 at 12:57:51 PM UTC-5 Kaj Wiik wrote:

> Hi Ken,
>
> Here's the repository of the tcpborphserver3:
>
> https://gitlab.ras.byu.edu/alpaca/casper/katcp/-/tree/master/tcpborphserver3
>
> There are a number of branches that support e.g. setting the RFDC NCO.
>
> We will have a similar need to stream lowish-speed polarimetry data using 
> the 1 GbE interface. I plan not to do the data readout using KATCP but 
> mapping the BRAM addresses directly to Julia data structures, forming HDF5 
> files in ramdisk and copying them to a NFS mounted server disk. Let's see 
> how it goes ;-), I have to design the analog interface first :-D.
>
> Cheers and good luck,
> Kaj
>
> On Sun, 8 Dec 2024 at 18:53, Ken Semanov  wrote:
>
>> I need to know everything about what is happening in the PS side with 
>> the  katcp server. Otherwise I cannot email the author of the katcp package 
>> to ask questions about it.  He will need to know exactly what we are doing 
>> with CASPERized Linux.   This is already going to be difficult to 
>> communicate to him as it is.  
>>
>> On Sunday, December 8, 2024 at 11:46:07 AM UTC-5 Ken Semanov wrote:
>>
>>> The Python version of the server side of KATCP is shown below.   (there 
>>> may be a C version somewhere) 
>>> .
>>> https://github.com/ska-sa/katcp-python/blob/master/katcp/server.py
>>>
>>> This uses futures from concurrent.futures  and  tornado futures.  As 
>>> some may already know,  "threading" in Python is fake.  All threads stall 
>>> all other threads due to a Global Interpreter Lock  (GIL). This is 
>>> exactly the behavior that I am seeing from network speed tests.   One of my 
>>> own processes can retrieve from a Snapshot at 2.1 Megabytes/sec. 
>>> However, for 8 concurrent processes,  I only see  6.1 Megabytes/sec.  It is 
>>> exactly as if the parallel connections refuse to stack up correctly.  If 
>>> the CASPERized Linux on the PS of the board side is running Python Katcp 
>>> (and not the C version)  then it would be stalling connections until the 
>>> GIL allows them into the single thread-of-execution. 
>>>
>>> Can anyone on this forum confirm any of this?   What is happening on the 
>>> CASPERized Linux image?  Where could I see this for myself?   What 
>>> processes in ps aux might I see related to  katcp servers on the PS side? 
>>>
>>>
>>>
>>>
>>> On Sunday, December 8, 2024 at 12:26:26 AM UTC-5 Ken Semanov wrote:
>>>
 Is there a way I could nose around the katcp server on the board side 
 in order to see how it is configured?   

 Could I reconfigure this server after booting into the board? 

 I am using a CASPERized image on a ZCU216.  I am migrating data out of 
 snapshot blocks over network.  After several network tests, it appears 
 that 
 the katcp on the board side (PS)  is artificially limited to 3 or 4 
 simultaneous network connections over RJ45 ethernet.   My use-case needs 
 this to be more like 8 to 16 connections.  I may also need to change the 
 packet size. 

 Thanks. 

>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "cas...@lists.berkeley.edu" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to casper+un...@lists.berkeley.edu.
>> To view this discussion visit 
>> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/386ccab3-d2a3-4507-9542-1b4889b473f3n%40lists.berkeley.edu
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"casper@lists.berkeley.edu" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to casper+unsubscr...@lists.berkeley.edu.
To view this discussion visit 
https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/228cd5af-4297-4333-b5b3-86f8bd1affb5n%40lists.berkeley.edu.

Re: [casper] Re: KATCP server configuration

2024-12-08 Thread Kaj Wiik 
Hi Ken,

Here's the repository of the tcpborphserver3:
https://gitlab.ras.byu.edu/alpaca/casper/katcp/-/tree/master/tcpborphserver3

There are a number of branches that support e.g. setting the RFDC NCO.

We will have a similar need to stream lowish-speed polarimetry data using
the 1 GbE interface. I plan not to do the data readout using KATCP but
mapping the BRAM addresses directly to Julia data structures, forming HDF5
files in ramdisk and copying them to a NFS mounted server disk. Let's see
how it goes ;-), I have to design the analog interface first :-D.

Cheers and good luck,
Kaj

On Sun, 8 Dec 2024 at 18:53, Ken Semanov  wrote:

> I need to know everything about what is happening in the PS side with the
> katcp server. Otherwise I cannot email the author of the katcp package to
> ask questions about it.  He will need to know exactly what we are doing
> with CASPERized Linux.   This is already going to be difficult to
> communicate to him as it is.
>
> On Sunday, December 8, 2024 at 11:46:07 AM UTC-5 Ken Semanov wrote:
>
>> The Python version of the server side of KATCP is shown below.   (there
>> may be a C version somewhere)
>> .
>> https://github.com/ska-sa/katcp-python/blob/master/katcp/server.py
>>
>> This uses futures from concurrent.futures  and  tornado futures.  As some
>> may already know,  "threading" in Python is fake.  All threads stall all
>> other threads due to a Global Interpreter Lock  (GIL). This is exactly
>> the behavior that I am seeing from network speed tests.   One of my own
>> processes can retrieve from a Snapshot at 2.1 Megabytes/sec. However,
>> for 8 concurrent processes,  I only see  6.1 Megabytes/sec.  It is exactly
>> as if the parallel connections refuse to stack up correctly.  If the
>> CASPERized Linux on the PS of the board side is running Python Katcp (and
>> not the C version)  then it would be stalling connections until the GIL
>> allows them into the single thread-of-execution.
>>
>> Can anyone on this forum confirm any of this?   What is happening on the
>> CASPERized Linux image?  Where could I see this for myself?   What
>> processes in ps aux might I see related to  katcp servers on the PS side?
>>
>>
>>
>>
>> On Sunday, December 8, 2024 at 12:26:26 AM UTC-5 Ken Semanov wrote:
>>
>>> Is there a way I could nose around the katcp server on the board side in
>>> order to see how it is configured?
>>>
>>> Could I reconfigure this server after booting into the board?
>>>
>>> I am using a CASPERized image on a ZCU216.  I am migrating data out of
>>> snapshot blocks over network.  After several network tests, it appears that
>>> the katcp on the board side (PS)  is artificially limited to 3 or 4
>>> simultaneous network connections over RJ45 ethernet.   My use-case needs
>>> this to be more like 8 to 16 connections.  I may also need to change the
>>> packet size.
>>>
>>> Thanks.
>>>
>> --
> You received this message because you are subscribed to the Google Groups "
> casper@lists.berkeley.edu" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to casper+unsubscr...@lists.berkeley.edu.
> To view this discussion visit
> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/386ccab3-d2a3-4507-9542-1b4889b473f3n%40lists.berkeley.edu
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"casper@lists.berkeley.edu" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to casper+unsubscr...@lists.berkeley.edu.
To view this discussion visit 
https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADpJ3s1PpjiUMZ1PGTNWDk%3DhY%2BFGPuCtn8b4VtOq8d9UgV8Qgg%40mail.gmail.com.