Re: [PD] Arbitrary buffer sizes in pd, libpd
It never even occurred to me to even test other buffer sizes than 64, 128, 512, 1024. Never assume that hardware buffer sizes are a power-of-2! There are even some built-in sound cards that *only* run on a single fixed weird buffer size. Conversely, never assume that all audio processing algorithms run at power-of-2 block sizes. For example, the Opus encoder/decoder requires block sizes like 120, 240, 480, 960, etc. (multiples of 2.5 ms @ 48 kHz). --- Finally, be aware that VST plugins don't use a fixed block size! The block size passed to the setup function (VST3: "maxSamplesPerBlock" in "Vst::ProcessSetup") is only the *maximum* number of samples you will ever attempt to process. The actual number of samples passed to the process function (VST3: "numSamples" in "Vst::ProcessData") can vary between processing calls! This is necessary so that DAWs can process "incomplete" blocks, e.g. for playback start, for accurate looping or before tempo changes (so that the tempo change happens exactly at the start of a block). If a processor depends on a fixed processing block size - as is the case with Pd - you generally have to buffer the inputs/outputs. This obviously adds latency, but most DAWs have latency compensation. Note that you also have to buffer MIDI messages (and use appropriate time stamps)! For example, this is what Camomile does: https://github.com/pierreguillot/Camomile/blob/dev/v1.0.8/Source/PluginProcessor.cpp#L297 One general problem to be aware of: Once a processing block is smaller than or not a multiple of DEFDACBLKSIZE, all following blocks will effectively be offset by that amount of samples. For MIDI messages this is not a real issue because they are timestamped. However, any kind of transport information, like the project play head or tempo changes, is only provided once per block. If the plugin relies on this information, the output might not be perfectly aligned with the rest of the project. --- Of course, it would be nice if this was already handled in libpd itself (as an option). Buffering the audio input/output would be the easy part. libpd would also have to automatically delay incoming MIDI messages and buffer + delay outgoing MIDI messages. I think it's doable, but it's not trivial. Christof ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] JACK and blocksize
On 1/12/22 21:04, Roman Haefeli wrote: I guess my question boils down to: Is it possible to make that a configurable (at runtime) parameter? Would it be worthwhile feature besides an assumingly lower latency in special cases? i prepared a patch that did that in 2004 or so. i don't think i still have it anywhere (though you could check on sourceforge). gdsmf IOhannes OpenPGP_signature Description: OpenPGP digital signature ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
[PD] Arbitrary buffer sizes in pd, libpd
Apparently some applications on some OSes allow the user to set any buffer size in their audio settings. Sometimes the buffer size will be a slider to set any value small to large. One user of our VST3 plugin running libpd inside complained about sporadic crackles and discontinuities in the audio. The user was running the plugin in Reaper on the Mac with a buffer size of 1200. It never even occurred to me to even test other buffer sizes than 64, 128, 512, 1024. However apparently Reaper allows these settings and they will lead to discontinuities since Pd and therefore libpd aren't prepared for such arbitrary buffer sizes. We could add more ring buffers in our plugin code, but this would introduce latency for everyone else. Just wanted to share this here, since it is something to consider for libpd implementations. m. ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] JACK and blocksize
Hi Thanks for your detailed explanations, Christof. On Wed, 2022-01-12 at 16:13 +0100, Christof Ressi wrote: > > Generally, there is no way to get lower I/O latency than 64 samples > in > Pd without changing DEFDACBLKSIZE and recompiling. I guess my question boils down to: Is it possible to make that a configurable (at runtime) parameter? Would it be worthwhile feature besides an assumingly lower latency in special cases? Roman signature.asc Description: This is a digitally signed message part ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] JACK and blocksize
Sorry, I didn't check this myself, but can {block~] be used in a
top-level patch perhaps?
You can, but then you can't use [adc~] and [dac~]. Also, it wouldn't
solve the latency problem. Generally, reblocking to a *lower* block size
does not affect latency.
___
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management ->
https://lists.puredata.info/listinfo/pd-list
Re: [PD] JACK and blocksize
Hi Roman, I wonder if the blocksize of 64 is deeply baked into Pd or if it could theoretically be made configurable. Pd's scheduler block size is a constant value (64 samples), defined as DEFDACBLKSIZE in "s_stuff.h". In theory, you can change the #define in "s_stuff.h" and recompile Pd (and all externals, just in case some of them rely on it), but that's obviously not a good solution. Now, the "callbackprocess" function in "s_audio_jack.c" checks if the Jack block size is multiple of DEFDACBLKSIZE. As you noticed, this breaks for Jack buffer sizes smaller than DEFDACBLKSIZE. Actually, Pd's Jack callback implementation could use a small ring buffer for such situations. For example, if the Jack blocksize was 16 samples, Pd would call m_audio_callbackfn() every 4 blocks to fill the buffer. Note that you would still have 48 samples of extra latency, but at least sound would be working :-) Generally, there is no way to get lower I/O latency than 64 samples in Pd without changing DEFDACBLKSIZE and recompiling. I found the '-blocksize' flag, but it seems it only sets the blocksize for audio I/O and doesn't affect Pd's internal blocksize Yes. "block size" is really a misnomer. At least it should be called "hardware blocksize" or even "hardware buffer size". It does *not* set Pd's scheduling block size, which is really fixed at 64 samples. (and doesn't have any effect at all when running with -jack). In the case of Jack, the hardware blocksize can't be set dynamically in the client. Instead, Pd obtains the value from the Jack server. The same is true for the sample rate, btw. ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] JACK and blocksize
On Wed, 2022-01-12 at 15:28 +0100, Peter P. wrote:
> * Roman Haefeli [2022-01-12 15:08]:
> > Hey all
> >
> > Using callbacks is certainly interesting for low-latency
> > applications.
> > I noticed that JACK allows blocksizes below 64, namely 32 and 16.
> > However, those can only be used with callbacks disabled which means
> > having to use an additional buffer again.
> >
> > I wonder if the blocksize of 64 is deeply baked into Pd or if it
> > could
> > theoretically be made configurable.
> Sorry, I didn't check this myself, but can {block~] be used in a
> top-level patch perhaps?
According to the documentation, [dac~ ]/[adc~] cannot reside in re-
blocked (sub)patches. There is no mentioning of top-level patches.
However, my point was to allow to connect pd to jack running at bs=32
or bs=16 _with_ callbacks.
Roman
signature.asc
Description: This is a digitally signed message part
___
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management ->
https://lists.puredata.info/listinfo/pd-list
Re: [PD] Store data in memory more efficiently than in arrays
I read once in IRC that one value in a Pd-array requires not 4 bytes, but 8 bytes on 64-bit systems. Yes. Pd's graphical arrays (and Pd's data structure arrays in general) are implemented as a linear array of "words" (t_word). A "word" can hold one of several possible types. It is implemented as a C union, so the overall size is always that of the largest member. In our case, the largest member is a pointer (e.g. t_symbol *), which is 4 bytes on a 32-bit system and 8 bytes on a 64-bit systems. This means that even if you would add a "byte" type, the overall size of "t_word" would stay the same. However, you can always implement your own byte array object as an external. But as you noted, this is not necessary except you're on a very tight memory and/or CPU budget. Christof On 12.01.2022 14:20, Roman Haefeli wrote: Hi Sometimes I stored byte data (lists of bytes) in arrays. IIRC, I read once in IRC that one value in a Pd-array requires not 4 bytes, but 8 bytes on 64-bit systems. Since storing plain bytes seems not such an uncommon use case for me, I wonder if it can be done more efficiently. Not that I ever hit a memory limit, I'm just curious. With the new (amazing!) [file] object, dealing with byte lists has become even more appealing, so the desire to store them in memory increases. Roman ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] JACK and blocksize
* Roman Haefeli [2022-01-12 15:08]:
> Hey all
>
> Using callbacks is certainly interesting for low-latency applications.
> I noticed that JACK allows blocksizes below 64, namely 32 and 16.
> However, those can only be used with callbacks disabled which means
> having to use an additional buffer again.
>
> I wonder if the blocksize of 64 is deeply baked into Pd or if it could
> theoretically be made configurable.
Sorry, I didn't check this myself, but can {block~] be used in a
top-level patch perhaps?
cheersz, P
___
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management ->
https://lists.puredata.info/listinfo/pd-list
Re: [PD] irreversibly high CPU usage with 'use callbacks' (macOS)
no such luck here… mac OS 11.6.2 Pd 0.52-1-really open test audio+midi patch + cpu load meter set core audio to 48kh / blocksize 64 /5ms activate callbacks, switch on audio = instant crash. log attached hans Process: pd [97091] Path: /Applications/Pd-0.52-1-really.app/Contents/Resources/bin/pd Identifier:pd Version: ??? Code Type: X86-64 (Native) Parent Process:Pd [97089] Responsible: Pd [97089] User ID: 501 Date/Time: 2022-01-12 15:15:31.871 +0100 OS Version:macOS 11.6.2 (20G314) Report Version:12 Anonymous UUID:93EE2292-1DCF-4ED3-BEA8-482512C78291 Sleep/Wake UUID: 3722F958-E979-4E67-A2CE-1F32AA0B04EE Time Awake Since Boot: 89 seconds Time Since Wake: 12000 seconds System Integrity Protection: enabled Crashed Thread:0 Dispatch queue: com.apple.main-thread Exception Type:EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x0400 Exception Note:EXC_CORPSE_NOTIFY Termination Signal:Segmentation fault: 11 Termination Reason:Namespace SIGNAL, Code 0xb Terminating Process: exc handler [97091] VM Regions Near 0x400: --> __TEXT 10c389000-10c4a8000[ 1148K] r-x/rwx SM=COW /Applications/Pd-0.52-1-really.app/Contents/Resources/bin/pd Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libsystem_platform.dylib0x7fff2036210c _platform_memmove$VARIANT$Haswell + 268 Thread 1:: AMCP Logging Spool 0 libsystem_kernel.dylib 0x7fff202ea2f6 semaphore_wait_trap + 10 1 com.apple.audio.caulk 0x7fff283dd8da caulk::mach::semaphore::wait_or_error() + 16 2 com.apple.audio.caulk 0x7fff283ca836 caulk::semaphore::timed_wait(double) + 110 3 com.apple.audio.caulk 0x7fff283ca784 caulk::concurrent::details::worker_thread::run() + 30 4 com.apple.audio.caulk 0x7fff283ca502 void* caulk::thread_proxy > >(void*) + 45 5 libsystem_pthread.dylib 0x7fff2031f8fc _pthread_start + 224 6 libsystem_pthread.dylib 0x7fff2031b443 thread_start + 15 Thread 2: 0 libsystem_pthread.dylib 0x7fff2031b420 start_wqthread + 0 Thread 3: 0 libsystem_kernel.dylib 0x7fff202ea2ba mach_msg_trap + 10 1 libsystem_kernel.dylib 0x7fff202ea62c mach_msg + 60 2 com.apple.audio.midi.CoreMIDI 0x7fff34d45bd6 XServerMachPort::ReceiveMessage(int&, void*, int&) + 94 3 com.apple.audio.midi.CoreMIDI 0x7fff34d76473 MIDIProcess::MIDIInPortThread::Run() + 111 4 com.apple.audio.midi.CoreMIDI 0x7fff34d5c4e2 XThread::RunHelper(void*) + 10 5 com.apple.audio.midi.CoreMIDI 0x7fff34d5d65b CAPThread::Entry(CAPThread*) + 77 6 libsystem_pthread.dylib 0x7fff2031f8fc _pthread_start + 224 7 libsystem_pthread.dylib 0x7fff2031b443 thread_start + 15 Thread 4: 0 libsystem_kernel.dylib 0x7fff202eccde __psynch_cvwait + 10 1 libsystem_pthread.dylib 0x7fff2031fe49 _pthread_cond_wait + 1298 2 py.pd_darwin0x00010c68710b flext_multi::ThrHelper(void*) + 187 3 libsystem_pthread.dylib 0x7fff2031f8fc _pthread_start + 224 4 libsystem_pthread.dylib 0x7fff2031b443 thread_start + 15 Thread 5: 0 libsystem_kernel.dylib 0x7fff202eccde __psynch_cvwait + 10 1 libsystem_pthread.dylib 0x7fff2031fe49 _pthread_cond_wait + 1298 2 libopenblas.0.dylib 0x0001112a620f blas_thread_server + 207 3 libsystem_pthread.dylib 0x7fff2031f8fc _pthread_start + 224 4 libsystem_pthread.dylib 0x7fff2031b443 thread_start + 15 Thread 6: 0 libsystem_kernel.dylib 0x7fff202eccde __psynch_cvwait + 10 1 libsystem_pthread.dylib 0x7fff2031fe49 _pthread_cond_wait + 1298 2 py.pd_darwin0x00010c68872e flext_multi::ThrCond::TimedWait(double) + 158 3 py.pd_darwin0x00010c68f124 flext_base_multi::QWorker(flext_multi::thr_params*) + 52 4 py.pd_darwin0x00010c686da1 LaunchHelper(thr_entry*) + 33 5 libsystem_pthread.dylib 0x7fff2031f8fc _pthread_start + 224 6 libsystem_pthread.dylib 0x7fff2031b443 thread_start + 15 Thread 7: 0 libsystem_kernel.dylib 0x7fff202f2656 __select + 10 1 libpython3.7m.dylib 0x0001109890f3 time_sleep + 323 2 libpython3.7m.dylib 0x0001107cb379 _PyMethodDef_RawFastCallDict + 665 3 libpython3.7m.dylib 0x0001107cc546 PyCFunction_Call + 166 4 py.pd_darwin0x00010c67c22b pybase::pyworker(flext_multi
[PD] JACK and blocksize
Hey all Using callbacks is certainly interesting for low-latency applications. I noticed that JACK allows blocksizes below 64, namely 32 and 16. However, those can only be used with callbacks disabled which means having to use an additional buffer again. I wonder if the blocksize of 64 is deeply baked into Pd or if it could theoretically be made configurable. There might be other reasons why smaller (or larger) blocksizes would be interesting, but low-latency seems to be one. I found the '-blocksize' flag, but it seems it only sets the blocksize for audio I/O and doesn't affect Pd's internal blocksize (and doesn't have any effect at all when running with -jack). Roman signature.asc Description: This is a digitally signed message part ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
[PD] Store data in memory more efficiently than in arrays
Hi Sometimes I stored byte data (lists of bytes) in arrays. IIRC, I read once in IRC that one value in a Pd-array requires not 4 bytes, but 8 bytes on 64-bit systems. Since storing plain bytes seems not such an uncommon use case for me, I wonder if it can be done more efficiently. Not that I ever hit a memory limit, I'm just curious. With the new (amazing!) [file] object, dealing with byte lists has become even more appealing, so the desire to store them in memory increases. Roman signature.asc Description: This is a digitally signed message part ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] irreversibly high CPU usage with 'use callbacks' (macOS)
Just another data point: I'm not able to reproduce this here: * macOS 10.15.7 * Pd 0.52-1-really * MacBook Pro 2016 Tested with CoreAudio and JACK Roman On Tue, 2022-01-11 at 23:43 +0100, Christof Ressi wrote: > Hi, > > I have noticed this myself just a few days ago (Pd 0.52-1-really, > Windows 7). I had a quick look and couldn't immediately spot the > problem, but I will have a closer look. Actually, I have been working > on > a branch with several scheduler improvements and this issue is > definitely on my to-do list. > > Christof > > On 11.01.2022 20:43, William Chang wrote: > > Hi all, > > > > If I turn on 'use callbacks' in audio preferences, then turn on > > DSP, > > pd's CPU usage goes to > 100% and stays there even if I turn DSP > > back > > off. This sometimes locks up the GUI and I have to restart Pd. > > Without > > checking 'use callbacks' it seems to be fine. Tried this with > > default > > audio devices, MOTU Ultralite mk4, and Blackhole 2ch. Can anyone > > reproduce? > > > > Pd: 0.52-1-really > > macOS: 12.0.1 > > MacBook Pro 14" M1 Pro > > > > Thanks, > > Will > > > > ___ > > Pd-list@lists.iem.at mailing list > > UNSUBSCRIBE and account-management -> > > https://lists.puredata.info/listinfo/pd-list > > > ___ > Pd-list@lists.iem.at mailing list > UNSUBSCRIBE and account-management -> > https://lists.puredata.info/listinfo/pd-list signature.asc Description: This is a digitally signed message part ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] [deken] new feature : download a list of externals
Thanks Iohannes. I didn't see this feature. It could be a nice way for newbies. And I see on github/deken that some requests are similar. The ideal way I guess, could be to write "zexy cyclone iemlib" and download them in oneshot. I will check the github page. Best. j. - Mail original - > De: "IOhannes m zmölnig" > À: pd-list@lists.iem.at, "abel jerome" > Envoyé: Mardi 11 Janvier 2022 20:11:00 > Objet: Re: [PD] [deken] new feature : download a list of externals > > Am 11. Jänner 2022 18:52:19 MEZ schrieb abel.jer...@free.fr: > >Hi, > > > >Thanks for Deken development, it is really helpful. > > > >I wonder if it would be possible to add a feature to deken, like : > >"download a list of externals" or show a window with checkboxes per > >lib. > > > >It could be very useful in workshops to use few libraries with > >students. So download them once. > > > right click the desired externals and "select" multiple ones, then > download them? > > also there's a an issue tracker at https://GitHub.com/pure-data/deken > (Be sure to check the existing issues first) > > > mfg.sfg.jfd > IOhannes > > ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] More adventures in [file] land - spaces in paths
No worries! Actually, your question really shows how the current behavior of symbol atoms can be confusing and why it needs to be fixed. On 12.01.2022 12:38, Pierre Alexandre Tremblay wrote: How did I miss that when I search for issues I don’t know. Sorry for the noise. p On 12 Jan 2022, at 10:17, Christof Ressi wrote: No, it's just a display problem: https://github.com/pure-data/pure-data/issues/1477 On 12.01.2022 11:04, Pierre Alexandre Tremblay wrote: Hello Did anyone have problems with spaces in paths? If one uses the help to demonstrate, entering the path (don’t worry this is not my real path :) /User/pa/my\ hard\ disk/my\ files/test.pd One gets at the bottom: /User/pa/my\\\ hard\\\ disk/my\\\ files/text.pd === If I investigate the list length I get the right number of components which mean that [path split] is working great of keeping the space escapement So I reckon it is a ‘bug’ in [file join] ? Or is it an expected behaviour? ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] More adventures in [file] land - spaces in paths
No, it's just a display problem: https://github.com/pure-data/pure-data/issues/1477 On 12.01.2022 11:04, Pierre Alexandre Tremblay wrote: Hello Did anyone have problems with spaces in paths? If one uses the help to demonstrate, entering the path (don’t worry this is not my real path :) /User/pa/my\ hard\ disk/my\ files/test.pd One gets at the bottom: /User/pa/my\\\ hard\\\ disk/my\\\ files/text.pd === If I investigate the list length I get the right number of components which mean that [path split] is working great of keeping the space escapement So I reckon it is a ‘bug’ in [file join] ? Or is it an expected behaviour? ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
[PD] More adventures in [file] land - spaces in paths
Hello Did anyone have problems with spaces in paths? If one uses the help to demonstrate, entering the path (don’t worry this is not my real path :) /User/pa/my\ hard\ disk/my\ files/test.pd One gets at the bottom: /User/pa/my\\\ hard\\\ disk/my\\\ files/text.pd === If I investigate the list length I get the right number of components which mean that [path split] is working great of keeping the space escapement So I reckon it is a ‘bug’ in [file join] ? Or is it an expected behaviour? smime.p7s Description: S/MIME cryptographic signature ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
