Re: [PD] help making sense of [readsf~]

[Christof Ressi]

not necessarily as a "always delay before playing" sort of thing but 
more a short description of how it works,
I agree! It's always better when the user actually understands *why* 
they need to do something.


In general, though, I would advice everyone to consider a delay between 
"open" and "start" when you design your patches, in particular if you 
plan to distribute it to other people.


[open , start( is convenient, but it can be hard to refactor at a 
later stage!


Personally, when I build a playlist in Pd, I have one or more [readsf~] 
objects and cycle between them. For example:


| cue  | readsf~ 1 | readsf~ 2 |
|  | - | - |
| 1| play 1| open 2|
| 2| open 3| play 2|
| 3| play 3| open 4|
| 4| open 5| play 4|
| 5| play 5| open 6|

etc.

I think you get the pattern.

Christof

On 04.03.2024 13:07, Dan Wilcox wrote:
*This* is a good point and worth noting, not necessarily as a "always 
delay before playing" sort of thing but more a short description of 
how it works, then a note like "if you experience occasional dropouts 
on first accessing a file, consider adding a small delay after opening 
but *before* playing."



On Mar 4, 2024, at 1:01 PM, pd-list-requ...@lists.iem.at wrote:

Message: 1
Date: Mon, 4 Mar 2024 12:55:54 +0100
From: Christof Ressi 
To:pd-list@lists.iem.at
Subject: Re: [PD] help making sense of [readsf~]
Message-ID: <2176516f-edbe-4982-bcfa-b7002952a...@christofressi.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"

Actually, I forgot something important:

Of course, the worker thread must also *open* the file! If the file is
not yet cached by the OS, this can indeed take a few milliseconds.If you
don't add some delay between "open" and "start", you might notice that
you get a dropout the very first time, but not on subsequent times.

In fact, if you don't wait between "open" and "start", the perform
method almost certainly blocks. However, often we don't notice because
it may be "absorbed" by Pd's own ringbuffer (= "Delay" in the audio
settings).

Anyway, I agree that the help needs some more clarification! (Just make
sure you really understand how the object works before changing the help
patch :)

?Christof



Dan Wilcox
danomatika.com 
robotcowboy.com 
___
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management -> 
https://lists.puredata.info/listinfo/pd-list

Re: [PD] help making sense of [readsf~]

[Christof Ressi]

As Christof says, fine tuning the reader buffer size could be thought 
of as fine tuning the audio buffer length, ie. a 64 block size is 
faster but has less space/time if your patch does some really CPU 
intensive stuff. If you don't need super responsively, you can 
increase the block size.


Just to clarify: the buffer for readsf~ itself does not cause any delay, 
so larger values don't hurt (except for wasting some kilobytes). The 
latency is explicitly controlled by the user as the time between the 
"open" and "start" message.


(The reason why the buffer itself does not cause a delay is because the 
worker thread is "eager", i.e. it already has all the data available and 
so it always fills the buffer as fast as possible. This is different to 
an audio device where the audio input is produced at the same rate as it 
is consumed.)


I have never personally needed to use anything but the default readsf~ 
buffer size.
Yes, as I noted, the default buffer size is 65356 samples, which is over 
1 second of audio at 48 kHz. This is much more than we will ever need on 
modern systems. The user only needs to think about the wait time between 
"open" and "start" to avoid *initial* dropouts.


Christof
___
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management -> 
https://lists.puredata.info/listinfo/pd-list

Re: [PD] help making sense of [readsf~]

[Dan Wilcox]

*This* is a good point and worth noting, not necessarily as a "always delay 
before playing" sort of thing but more a short description of how it works, 
then a note like "if you experience occasional dropouts on first accessing a 
file, consider adding a small delay after opening but *before* playing."

> On Mar 4, 2024, at 1:01 PM, pd-list-requ...@lists.iem.at wrote:
> 
> Message: 1
> Date: Mon, 4 Mar 2024 12:55:54 +0100
> From: Christof Ressi mailto:i...@christofressi.com>>
> To: pd-list@lists.iem.at 
> Subject: Re: [PD] help making sense of [readsf~]
> Message-ID: <2176516f-edbe-4982-bcfa-b7002952a...@christofressi.com 
> >
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
> 
> Actually, I forgot something important:
> 
> Of course, the worker thread must also *open* the file! If the file is 
> not yet cached by the OS, this can indeed take a few milliseconds.If you 
> don't add some delay between "open" and "start", you might notice that 
> you get a dropout the very first time, but not on subsequent times.
> 
> In fact, if you don't wait between "open" and "start", the perform 
> method almost certainly blocks. However, often we don't notice because 
> it may be "absorbed" by Pd's own ringbuffer (= "Delay" in the audio 
> settings).
> 
> Anyway, I agree that the help needs some more clarification! (Just make 
> sure you really understand how the object works before changing the help 
> patch :)
> 
> ?Christof


Dan Wilcox
danomatika.com 
robotcowboy.com 
___
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management -> 
https://lists.puredata.info/listinfo/pd-list

Re: [PD] help making sense of [readsf~]

[Dan Wilcox]

To echo Christof, each object has a parallel worker thread as the file I/O 
could block Pd's audio thread. It takes time to read the beginning of the file 
(or  from the start point) before it can be played back.

IMO the warning / concern in the help file is really from a time of *much* 
slower media where the file I/O would take a lot longer, especially if a 
spinning hard drive has to access multiple files in multiple physical 
locations. Nowadays, speed and (pre-)caching make access times so much faster 
we don't necessarily think about it except for very large files we need to be 
able to randomly access.

As Christof says, fine tuning the reader buffer size could be thought of as 
fine tuning the audio buffer length, ie. a 64 block size is faster but has less 
space/time if your patch does some really CPU intensive stuff. If you don't 
need super responsively, you can increase the block size. I have never 
personally needed to use anything but the default readsf~ buffer size.

> On Mar 4, 2024, at 12:00 PM, pd-list-requ...@lists.iem.at wrote:
> 
> Message: 2
> Date: Mon, 4 Mar 2024 06:23:16 +0100
> From: Christof Ressi mailto:i...@christofressi.com>>
> To: pd-list@lists.iem.at 
> Subject: Re: [PD] help making sense of [readsf~]
> Message-ID: <93bd5dc6-baa9-4097-ac07-ae7683b75...@christofressi.com 
> >
> Content-Type: text/plain; charset=UTF-8; format=flowed
> 
> Hi,
> 
>> making it seem it would load the file into an internal memory and not 
>> just read directly from the disk
> The help file says "soundfile playback from disk"...
> 
> Here's how the object works:
> 
> A worker thread is reading data from the given file and fills a buffer. 
> If the buffer is full, it waits until there is space. The thread starts 
> to do its job right after the [open( message.
> 
> Once we send the [start( message, the perform method simply tries to 
> read a block of samples from the buffer and copy it to the outlets. If 
> there is not enough data in the buffer, the method blocks - which is 
> something we definitely want to avoid! This is exactly the reason why we 
> need to wait a little bit between the [open( message and the [start( 
> message; otherwise the perform routine might have to wait for the 
> buffer, causing a dropout.
> 
> The second argument for [readsf~] is the size of the buffer. The default 
> value seems to be 262144 bytes (per channel). In single-precision Pd 
> that corresponds to 65536 samples, which should be more than enough. I 
> think this value comes from the times where everybody had slow HDDs with 
> unpredictable seek times; for modern SSDs it can be much smaller, but we 
> probably don't care about a few kilobytes.
> 
> (BTW, I have no idea why the help patch uses a buffer size of 1 MB...)


Dan Wilcox
danomatika.com 
robotcowboy.com 
___
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management -> 
https://lists.puredata.info/listinfo/pd-list

Re: [PD] help making sense of [readsf~]

[Christof Ressi]

Actually, I forgot something important:

Of course, the worker thread must also *open* the file! If the file is 
not yet cached by the OS, this can indeed take a few milliseconds.If you 
don't add some delay between "open" and "start", you might notice that 
you get a dropout the very first time, but not on subsequent times.


In fact, if you don't wait between "open" and "start", the perform 
method almost certainly blocks. However, often we don't notice because 
it may be "absorbed" by Pd's own ringbuffer (= "Delay" in the audio 
settings).


Anyway, I agree that the help needs some more clarification! (Just make 
sure you really understand how the object works before changing the help 
patch :)


 Christof

On 04.03.2024 06:23, Christof Ressi wrote:

Hi,

making it seem it would load the file into an internal memory and not 
just read directly from the disk

The help file says "soundfile playback from disk"...

Here's how the object works:

A worker thread is reading data from the given file and fills a 
buffer. If the buffer is full, it waits until there is space. The 
thread starts to do its job right after the [open( message.


Once we send the [start( message, the perform method simply tries to 
read a block of samples from the buffer and copy it to the outlets. If 
there is not enough data in the buffer, the method blocks - which is 
something we definitely want to avoid! This is exactly the reason why 
we need to wait a little bit between the [open( message and the 
[start( message; otherwise the perform routine might have to wait for 
the buffer, causing a dropout.


The second argument for [readsf~] is the size of the buffer. The 
default value seems to be 262144 bytes (per channel). In 
single-precision Pd that corresponds to 65536 samples, which should be 
more than enough. I think this value comes from the times where 
everybody had slow HDDs with unpredictable seek times; for modern SSDs 
it can be much smaller, but we probably don't care about a few kilobytes.


(BTW, I have no idea why the help patch uses a buffer size of 1 MB...)

---

[writesf~] behaves just the otherway round. Since the perform routine 
is the producer, we don't need to wait after the [open( message. But 
after we send [stop(, we should to wait for the worker thread to drain 
the buffer and write the data to disk before we send another [open( 
message.


Christof

On 04.03.2024 05:24, Alexandre Torres Porres wrote:
Hi, a discussion on facebook led me to revise the help file of 
[readsf~] and I'm making some edits and changes. It seems it wasn't 
all to clear how it works, making it seem it would load the file into 
an internal memory and not just read directly from the disk, and in 
fact I'm not really sure how it works.


Also, the help has been saying forever how one should open a file "a 
bit" in advance, which is vague and it also doesn't make it clear why 
and how long soon... it also says it starts reading from the file 
right away, but doesn't play it until you say so... this is what 
makes it a bit confusing to people I guess, cause it seems to load 
into memory. Now, I believe there is some operation that is done in 
advance and then it adds some latency maybe, so if you do it in 
advance you get to manage this a bit better, is that it?


Can we have and add a bit more information about it?

Also, is it the case that this used to be a somewhat significant 
issue back in the day, which means this has become not significant 
all for a while?


Last, but not least, I can't make much sense of the 2nd argument. I 
had a look at the code to see what is the default value (which is 
also the minimum allowed value), something I always put on the help 
files, and I wonder if this is some kind of memory buffer that we 
load the file into... Moreover, why would someone need a bigger 
buffer than the default value?


I need this information well sorted to make this help file as nice as 
the others.


Cheers!

___
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management -> 
https://lists.puredata.info/listinfo/pd-list




___
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management -> 
https://lists.puredata.info/listinfo/pd-list___
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management -> 
https://lists.puredata.info/listinfo/pd-list