Re: [FFmpeg-devel] [PATCH v2 01/14] vvcdec: add thread executor

2023-07-18 Thread Michael Niedermayer 
On Tue, Jul 18, 2023 at 07:53:09PM +0800, Nuo Mi wrote:
> On Tue, Jul 18, 2023 at 6:14 AM Michael Niedermayer 
> wrote:
> 
> > On Sun, Jul 16, 2023 at 02:28:53PM +0800, Nuo Mi wrote:
> > > On Mon, Jul 10, 2023 at 3:41 PM Nuo Mi  wrote:
> > >
> > > >
> > > >
> > > > On Sun, Jul 9, 2023 at 5:52 AM Michael Niedermayer <
> > mich...@niedermayer.cc>
> > > > wrote:
> > > >
> > > >> On Fri, Jul 07, 2023 at 10:05:27PM +0800, Nuo Mi wrote:
> > > >> > The executor design pattern was inroduced by java
> > > >> > <
> > > >>
> > https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/Executor.html
> > > >> >
> > > >> > it also adapted by python
> > > >> > 
> > > >> > Compared to handcrafted thread pool management, it greatly
> > simplifies
> > > >> the thread code.
> > > >> > ---
> > > >> >  libavcodec/Makefile |   1 +
> > > >> >  libavcodec/executor.c   | 182
> > 
> > > >> >  libavcodec/executor.h   |  67 +++
> > > >> >  libavcodec/vvc/Makefile |   4 +
> > > >> >  4 files changed, 254 insertions(+)
> > > >> >  create mode 100644 libavcodec/executor.c
> > > >> >  create mode 100644 libavcodec/executor.h
> > > >> >  create mode 100644 libavcodec/vvc/Makefile
> > > >>
> > > >> This seems to need some fallback if pthreads are unavailable
> > > >>
> > > >> src/libavcodec/executor.c: In function ‘executor_worker_task’:
> > > >> src/libavcodec/executor.c:64:5: error: implicit declaration of
> > function
> > > >> ‘pthread_mutex_lock’; did you mean ‘ff_mutex_lock’?
> > > >> [-Werror=implicit-function-declaration]
> > > >>  pthread_mutex_lock(&e->lock);
> > > >>  ^~
> > > >>  ff_mutex_lock
> > > >> ...
> > > >> cc1: some warnings being treated as errors
> > > >> ffmpeg/ffbuild/common.mak:81: recipe for target
> > 'libavcodec/executor.o'
> > > >> failed
> > > >> make: *** [libavcodec/executor.o] Error 1
> > > >> make: *** Waiting for unfinished jobs
> > > >>
> > > >> thx
> > > >>
> > > >> [...]
> > > >> --
> > > >> Michael GnuPG fingerprint:
> > 9FF2128B147EF6730BADF133611EC787040B0FAB
> > > >>
> > > >> Why not whip the teacher when the pupil misbehaves? -- Diogenes of
> > Sinope
> > > >> ___
> > > >> ffmpeg-devel mailing list
> > > >> ffmpeg-devel@ffmpeg.org
> > > >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > >>
> > > >> To unsubscribe, visit link above, or email
> > > >> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> > > >>
> > > > Hi Michael,
> > > > Thank you for the information. I will fixed it
> > > >
> > >
> > > Hi Michael,
> > > FFmpeg application has a dependency on threads
> > > Once I --disable-pthreads, I can't build ffmpeg for the decoder md5 test.
> > > Could you share with me how to test the decoder when we disabled threads?
> >
> > it seems this failure occurred on a ppc cross compile
> > ../configure --target-os=linux --enable-cross-compile --disable-iconv
> > --disable-pthreads && make -j32
> 
> 
> 
> 
> > but its reproduceable on ubuntu x86-64 with a simple
> >  make distclean ; ./configure   --disable-pthreads && make -j32
> >
> I used a similar configuration, but both my configuration and the one you
> provided do not generate the ffmpeg executable. We only get the ffprobe.
> However, I need the ffmpeg executable to obtain the MD5 for the conformance
> test.

I dont understand fully

the command results in a build failure of libavcodec, theres no need to build
the ffmpeg excutable, it should fail before

Its true that once the build failure is corrected that testing this without 
pthreads
would be impossible with tools depending on pthreads and build in the same pass.
I dont think testing beyond build is critical here.
But you can test it likely either with older ffmpeg or with a bit of "duct tape"
building with shared libs building ffmpeg with threads and linking to libs build
without. Again, iam not asking fr that to be done, iam happy if the build 
failure is
fixed and its believed to work.

thx

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

While the State exists there can be no freedom; when there is freedom there
will be no State. -- Vladimir Lenin


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 01/14] vvcdec: add thread executor

2023-07-18 Thread Nuo Mi 
On Tue, Jul 18, 2023 at 6:14 AM Michael Niedermayer 
wrote:

> On Sun, Jul 16, 2023 at 02:28:53PM +0800, Nuo Mi wrote:
> > On Mon, Jul 10, 2023 at 3:41 PM Nuo Mi  wrote:
> >
> > >
> > >
> > > On Sun, Jul 9, 2023 at 5:52 AM Michael Niedermayer <
> mich...@niedermayer.cc>
> > > wrote:
> > >
> > >> On Fri, Jul 07, 2023 at 10:05:27PM +0800, Nuo Mi wrote:
> > >> > The executor design pattern was inroduced by java
> > >> > <
> > >>
> https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/Executor.html
> > >> >
> > >> > it also adapted by python
> > >> > 
> > >> > Compared to handcrafted thread pool management, it greatly
> simplifies
> > >> the thread code.
> > >> > ---
> > >> >  libavcodec/Makefile |   1 +
> > >> >  libavcodec/executor.c   | 182
> 
> > >> >  libavcodec/executor.h   |  67 +++
> > >> >  libavcodec/vvc/Makefile |   4 +
> > >> >  4 files changed, 254 insertions(+)
> > >> >  create mode 100644 libavcodec/executor.c
> > >> >  create mode 100644 libavcodec/executor.h
> > >> >  create mode 100644 libavcodec/vvc/Makefile
> > >>
> > >> This seems to need some fallback if pthreads are unavailable
> > >>
> > >> src/libavcodec/executor.c: In function ‘executor_worker_task’:
> > >> src/libavcodec/executor.c:64:5: error: implicit declaration of
> function
> > >> ‘pthread_mutex_lock’; did you mean ‘ff_mutex_lock’?
> > >> [-Werror=implicit-function-declaration]
> > >>  pthread_mutex_lock(&e->lock);
> > >>  ^~
> > >>  ff_mutex_lock
> > >> ...
> > >> cc1: some warnings being treated as errors
> > >> ffmpeg/ffbuild/common.mak:81: recipe for target
> 'libavcodec/executor.o'
> > >> failed
> > >> make: *** [libavcodec/executor.o] Error 1
> > >> make: *** Waiting for unfinished jobs
> > >>
> > >> thx
> > >>
> > >> [...]
> > >> --
> > >> Michael GnuPG fingerprint:
> 9FF2128B147EF6730BADF133611EC787040B0FAB
> > >>
> > >> Why not whip the teacher when the pupil misbehaves? -- Diogenes of
> Sinope
> > >> ___
> > >> ffmpeg-devel mailing list
> > >> ffmpeg-devel@ffmpeg.org
> > >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >>
> > >> To unsubscribe, visit link above, or email
> > >> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> > >>
> > > Hi Michael,
> > > Thank you for the information. I will fixed it
> > >
> >
> > Hi Michael,
> > FFmpeg application has a dependency on threads
> > Once I --disable-pthreads, I can't build ffmpeg for the decoder md5 test.
> > Could you share with me how to test the decoder when we disabled threads?
>
> it seems this failure occurred on a ppc cross compile
> ../configure --target-os=linux --enable-cross-compile --disable-iconv
> --disable-pthreads && make -j32




> but its reproduceable on ubuntu x86-64 with a simple
>  make distclean ; ./configure   --disable-pthreads && make -j32
>
I used a similar configuration, but both my configuration and the one you
provided do not generate the ffmpeg executable. We only get the ffprobe.
However, I need the ffmpeg executable to obtain the MD5 for the conformance
test.



>
> [...]
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Those who are too smart to engage in politics are punished by being
> governed by those who are dumber. -- Plato
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 01/14] vvcdec: add thread executor

2023-07-17 Thread Michael Niedermayer 
On Sun, Jul 16, 2023 at 02:28:53PM +0800, Nuo Mi wrote:
> On Mon, Jul 10, 2023 at 3:41 PM Nuo Mi  wrote:
> 
> >
> >
> > On Sun, Jul 9, 2023 at 5:52 AM Michael Niedermayer 
> > wrote:
> >
> >> On Fri, Jul 07, 2023 at 10:05:27PM +0800, Nuo Mi wrote:
> >> > The executor design pattern was inroduced by java
> >> > <
> >> https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/Executor.html
> >> >
> >> > it also adapted by python
> >> > 
> >> > Compared to handcrafted thread pool management, it greatly simplifies
> >> the thread code.
> >> > ---
> >> >  libavcodec/Makefile |   1 +
> >> >  libavcodec/executor.c   | 182 
> >> >  libavcodec/executor.h   |  67 +++
> >> >  libavcodec/vvc/Makefile |   4 +
> >> >  4 files changed, 254 insertions(+)
> >> >  create mode 100644 libavcodec/executor.c
> >> >  create mode 100644 libavcodec/executor.h
> >> >  create mode 100644 libavcodec/vvc/Makefile
> >>
> >> This seems to need some fallback if pthreads are unavailable
> >>
> >> src/libavcodec/executor.c: In function ‘executor_worker_task’:
> >> src/libavcodec/executor.c:64:5: error: implicit declaration of function
> >> ‘pthread_mutex_lock’; did you mean ‘ff_mutex_lock’?
> >> [-Werror=implicit-function-declaration]
> >>  pthread_mutex_lock(&e->lock);
> >>  ^~
> >>  ff_mutex_lock
> >> ...
> >> cc1: some warnings being treated as errors
> >> ffmpeg/ffbuild/common.mak:81: recipe for target 'libavcodec/executor.o'
> >> failed
> >> make: *** [libavcodec/executor.o] Error 1
> >> make: *** Waiting for unfinished jobs
> >>
> >> thx
> >>
> >> [...]
> >> --
> >> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> >>
> >> Why not whip the teacher when the pupil misbehaves? -- Diogenes of Sinope
> >> ___
> >> ffmpeg-devel mailing list
> >> ffmpeg-devel@ffmpeg.org
> >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >>
> >> To unsubscribe, visit link above, or email
> >> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> >>
> > Hi Michael,
> > Thank you for the information. I will fixed it
> >
> 
> Hi Michael,
> FFmpeg application has a dependency on threads
> Once I --disable-pthreads, I can't build ffmpeg for the decoder md5 test.
> Could you share with me how to test the decoder when we disabled threads?

it seems this failure occurred on a ppc cross compile
../configure --target-os=linux --enable-cross-compile --disable-iconv 
--disable-pthreads && make -j32

but its reproduceable on ubuntu x86-64 with a simple
 make distclean ; ./configure   --disable-pthreads && make -j32


[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Those who are too smart to engage in politics are punished by being
governed by those who are dumber. -- Plato 


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 01/14] vvcdec: add thread executor

2023-07-16 Thread Nuo Mi 
On Sat, Jul 15, 2023 at 6:38 AM Michael Niedermayer 
wrote:

> On Fri, Jul 07, 2023 at 10:05:27PM +0800, Nuo Mi wrote:
> > The executor design pattern was inroduced by java
> > <
> https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/Executor.html
> >
> > it also adapted by python
> > 
> > Compared to handcrafted thread pool management, it greatly simplifies
> the thread code.
> > ---
> >  libavcodec/Makefile |   1 +
> >  libavcodec/executor.c   | 182 
> >  libavcodec/executor.h   |  67 +++
> >  libavcodec/vvc/Makefile |   4 +
> >  4 files changed, 254 insertions(+)
> >  create mode 100644 libavcodec/executor.c
> >  create mode 100644 libavcodec/executor.h
> >  create mode 100644 libavcodec/vvc/Makefile
>
> [...]
>
> > +++ b/libavcodec/executor.h
> > @@ -0,0 +1,67 @@
> > +/*
> > + * Copyright (C) 2022 Nuo Mi
> > + *
> > + * This file is part of FFmpeg.
> > + *
> > + * FFmpeg is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU Lesser General Public
> > + * License as published by the Free Software Foundation; either
> > + * version 2.1 of the License, or (at your option) any later version.
> > + *
> > + * FFmpeg is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > + * Lesser General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU Lesser General Public
> > + * License along with FFmpeg; if not, write to the Free Software
> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA
> > + */
> > +
> > +#ifndef AVCODEC_EXECUTOR_H
> > +#define AVCODEC_EXECUTOR_H
> > +
> > +typedef struct Executor Executor;
> > +typedef struct Tasklet Tasklet;
> > +
> > +struct Tasklet {
> > +Tasklet *next;
> > +};
> > +
> > +typedef struct TaskletCallbacks {
> > +void *user_data;
> > +
> > +int local_context_size;
> > +
> > +// return 1 if a's priority > b's priority
> > +int (*priority_higher)(const Tasklet *a, const Tasklet *b);
> > +
> > +// task is ready for run
> > +int (*ready)(const Tasklet *t, void *user_data);
> > +
> > +// run the task
> > +int (*run)(Tasklet *t, void *local_context, void *user_data);
> > +} TaskletCallbacks;
> > +
> > +/**
> > + * Alloc executor
> > + * @param callbacks callback strucutre for executor
> > + * @param thread_count worker thread number
> > + * @return return the executor
> > + */
> > +Executor* ff_executor_alloc(const TaskletCallbacks *callbacks, int
> thread_count);
> > +
> > +/**
> > + * Free executor
> > + * @param e  pointer to executor
> > + */
> > +void ff_executor_free(Executor **e);
> > +
> > +/**
> > + * Add task to executor
> > + * @param e pointer to executor
> > + * @param t pointer to task. If NULL, it will wakeup one work thread
> > + */
> > +void ff_executor_execute(Executor *e, Tasklet *t);
> > +
> > +#endif //AVCODEC_EXECUTOR_H
>
> This would be quite useful outside libavcodec
> In fact id like to use it in libavradio to do the first stage of FFTs
> (which
> are the overwhelming bulk of computations ATM) and maybe eventually later
> stages too
>
Thank you. Glad to hear it.

>
> Maybe this could be made available to the outside of libavcodec
> as avpriv_ or av_ ?
> maybe from libavutil ?
>
avpriv_  is better since it is not a public API for ffmpeg's users(yet)
I will move it to libavutil and send separate patches for the executor
only.

>
> thx
>
> [...]
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Take away the freedom of one citizen and you will be jailed, take away
> the freedom of all citizens and you will be congratulated by your peers
> in Parliament.
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 01/14] vvcdec: add thread executor

2023-07-15 Thread Nuo Mi 
On Mon, Jul 10, 2023 at 3:41 PM Nuo Mi  wrote:

>
>
> On Sun, Jul 9, 2023 at 5:52 AM Michael Niedermayer 
> wrote:
>
>> On Fri, Jul 07, 2023 at 10:05:27PM +0800, Nuo Mi wrote:
>> > The executor design pattern was inroduced by java
>> > <
>> https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/Executor.html
>> >
>> > it also adapted by python
>> > 
>> > Compared to handcrafted thread pool management, it greatly simplifies
>> the thread code.
>> > ---
>> >  libavcodec/Makefile |   1 +
>> >  libavcodec/executor.c   | 182 
>> >  libavcodec/executor.h   |  67 +++
>> >  libavcodec/vvc/Makefile |   4 +
>> >  4 files changed, 254 insertions(+)
>> >  create mode 100644 libavcodec/executor.c
>> >  create mode 100644 libavcodec/executor.h
>> >  create mode 100644 libavcodec/vvc/Makefile
>>
>> This seems to need some fallback if pthreads are unavailable
>>
>> src/libavcodec/executor.c: In function ‘executor_worker_task’:
>> src/libavcodec/executor.c:64:5: error: implicit declaration of function
>> ‘pthread_mutex_lock’; did you mean ‘ff_mutex_lock’?
>> [-Werror=implicit-function-declaration]
>>  pthread_mutex_lock(&e->lock);
>>  ^~
>>  ff_mutex_lock
>> ...
>> cc1: some warnings being treated as errors
>> ffmpeg/ffbuild/common.mak:81: recipe for target 'libavcodec/executor.o'
>> failed
>> make: *** [libavcodec/executor.o] Error 1
>> make: *** Waiting for unfinished jobs
>>
>> thx
>>
>> [...]
>> --
>> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>>
>> Why not whip the teacher when the pupil misbehaves? -- Diogenes of Sinope
>> ___
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>>
> Hi Michael,
> Thank you for the information. I will fixed it
>

Hi Michael,
FFmpeg application has a dependency on threads
Once I --disable-pthreads, I can't build ffmpeg for the decoder md5 test.
Could you share with me how to test the decoder when we disabled threads?
Thank you
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 01/14] vvcdec: add thread executor

2023-07-14 Thread Michael Niedermayer 
On Fri, Jul 07, 2023 at 10:05:27PM +0800, Nuo Mi wrote:
> The executor design pattern was inroduced by java
> 
> it also adapted by python
> 
> Compared to handcrafted thread pool management, it greatly simplifies the 
> thread code.
> ---
>  libavcodec/Makefile |   1 +
>  libavcodec/executor.c   | 182 
>  libavcodec/executor.h   |  67 +++
>  libavcodec/vvc/Makefile |   4 +
>  4 files changed, 254 insertions(+)
>  create mode 100644 libavcodec/executor.c
>  create mode 100644 libavcodec/executor.h
>  create mode 100644 libavcodec/vvc/Makefile

[...]

> +++ b/libavcodec/executor.h
> @@ -0,0 +1,67 @@
> +/*
> + * Copyright (C) 2022 Nuo Mi
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 
> USA
> + */
> +
> +#ifndef AVCODEC_EXECUTOR_H
> +#define AVCODEC_EXECUTOR_H
> +
> +typedef struct Executor Executor;
> +typedef struct Tasklet Tasklet;
> +
> +struct Tasklet {
> +Tasklet *next;
> +};
> +
> +typedef struct TaskletCallbacks {
> +void *user_data;
> +
> +int local_context_size;
> +
> +// return 1 if a's priority > b's priority
> +int (*priority_higher)(const Tasklet *a, const Tasklet *b);
> +
> +// task is ready for run
> +int (*ready)(const Tasklet *t, void *user_data);
> +
> +// run the task
> +int (*run)(Tasklet *t, void *local_context, void *user_data);
> +} TaskletCallbacks;
> +
> +/**
> + * Alloc executor
> + * @param callbacks callback strucutre for executor
> + * @param thread_count worker thread number
> + * @return return the executor
> + */
> +Executor* ff_executor_alloc(const TaskletCallbacks *callbacks, int 
> thread_count);
> +
> +/**
> + * Free executor
> + * @param e  pointer to executor
> + */
> +void ff_executor_free(Executor **e);
> +
> +/**
> + * Add task to executor
> + * @param e pointer to executor
> + * @param t pointer to task. If NULL, it will wakeup one work thread
> + */
> +void ff_executor_execute(Executor *e, Tasklet *t);
> +
> +#endif //AVCODEC_EXECUTOR_H

This would be quite useful outside libavcodec
In fact id like to use it in libavradio to do the first stage of FFTs (which
are the overwhelming bulk of computations ATM) and maybe eventually later
stages too

Maybe this could be made available to the outside of libavcodec
as avpriv_ or av_ ?
maybe from libavutil ?

thx

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Take away the freedom of one citizen and you will be jailed, take away
the freedom of all citizens and you will be congratulated by your peers
in Parliament.


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 01/14] vvcdec: add thread executor

2023-07-10 Thread Nuo Mi 
On Sun, Jul 9, 2023 at 5:52 AM Michael Niedermayer 
wrote:

> On Fri, Jul 07, 2023 at 10:05:27PM +0800, Nuo Mi wrote:
> > The executor design pattern was inroduced by java
> > <
> https://docs.oracle.com/en/java/javase/20/docs/api/java.base/java/util/concurrent/Executor.html
> >
> > it also adapted by python
> > 
> > Compared to handcrafted thread pool management, it greatly simplifies
> the thread code.
> > ---
> >  libavcodec/Makefile |   1 +
> >  libavcodec/executor.c   | 182 
> >  libavcodec/executor.h   |  67 +++
> >  libavcodec/vvc/Makefile |   4 +
> >  4 files changed, 254 insertions(+)
> >  create mode 100644 libavcodec/executor.c
> >  create mode 100644 libavcodec/executor.h
> >  create mode 100644 libavcodec/vvc/Makefile
>
> This seems to need some fallback if pthreads are unavailable
>
> src/libavcodec/executor.c: In function ‘executor_worker_task’:
> src/libavcodec/executor.c:64:5: error: implicit declaration of function
> ‘pthread_mutex_lock’; did you mean ‘ff_mutex_lock’?
> [-Werror=implicit-function-declaration]
>  pthread_mutex_lock(&e->lock);
>  ^~
>  ff_mutex_lock
> src/libavcodec/executor.c:78:13: error: implicit declaration of function
> ‘pthread_mutex_unlock’; did you mean ‘ff_mutex_unlock’?
> [-Werror=implicit-function-declaration]
>  pthread_mutex_unlock(&e->lock);
>  ^~~~
>  ff_mutex_unlock
> src/libavcodec/executor.c:83:13: error: implicit declaration of function
> ‘pthread_cond_wait’; did you mean ‘__fread_chk_warn’?
> [-Werror=implicit-function-declaration]
>  pthread_cond_wait(&e->cond, &e->lock);
>  ^
>  __fread_chk_warn
> src/libavcodec/executor.c: In function ‘ff_executor_alloc’:
> src/libavcodec/executor.c:108:11: error: implicit declaration of function
> ‘pthread_mutex_init’; did you mean ‘ff_mutex_init’?
> [-Werror=implicit-function-declaration]
>  ret = pthread_mutex_init(&e->lock, NULL);
>^~
>ff_mutex_init
> src/libavcodec/executor.c:112:11: error: implicit declaration of function
> ‘pthread_cond_init’ [-Werror=implicit-function-declaration]
>  ret = pthread_cond_init(&e->cond, NULL);
>^
> src/libavcodec/executor.c:119:15: error: implicit declaration of function
> ‘pthread_create’; did you mean ‘ff_thread_setname’?
> [-Werror=implicit-function-declaration]
>  ret = pthread_create(&ti->thread, NULL, executor_worker_task, ti);
>^~
>ff_thread_setname
> src/libavcodec/executor.c:129:5: error: implicit declaration of function
> ‘pthread_cond_broadcast’ [-Werror=implicit-function-declaration]
>  pthread_cond_broadcast(&e->cond);
>  ^~
> src/libavcodec/executor.c:132:9: error: implicit declaration of function
> ‘pthread_join’; did you mean ‘ff_thread_once’?
> [-Werror=implicit-function-declaration]
>  pthread_join(e->threads[j].thread, NULL);
>  ^~~~
>  ff_thread_once
> src/libavcodec/executor.c:133:5: error: implicit declaration of function
> ‘pthread_cond_destroy’ [-Werror=implicit-function-declaration]
>  pthread_cond_destroy(&e->cond);
>  ^~~~
> src/libavcodec/executor.c:135:5: error: implicit declaration of function
> ‘pthread_mutex_destroy’; did you mean ‘ff_mutex_destroy’?
> [-Werror=implicit-function-declaration]
>  pthread_mutex_destroy(&e->lock);
>  ^
>  ff_mutex_destroy
> src/libavcodec/executor.c: In function ‘ff_executor_execute’:
> src/libavcodec/executor.c:180:5: error: implicit declaration of function
> ‘pthread_cond_signal’ [-Werror=implicit-function-declaration]
>  pthread_cond_signal(&e->cond);
>  ^~~
> cc1: some warnings being treated as errors
> ffmpeg/ffbuild/common.mak:81: recipe for target 'libavcodec/executor.o'
> failed
> make: *** [libavcodec/executor.o] Error 1
> make: *** Waiting for unfinished jobs
>
> thx
>
> [...]
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Why not whip the teacher when the pupil misbehaves? -- Diogenes of Sinope
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
Hi Michael,
Thank you for the information. I will fixed it
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 01/14] vvcdec: add thread executor

2023-07-08 Thread Michael Niedermayer 
On Fri, Jul 07, 2023 at 10:05:27PM +0800, Nuo Mi wrote:
> The executor design pattern was inroduced by java
> 
> it also adapted by python
> 
> Compared to handcrafted thread pool management, it greatly simplifies the 
> thread code.
> ---
>  libavcodec/Makefile |   1 +
>  libavcodec/executor.c   | 182 
>  libavcodec/executor.h   |  67 +++
>  libavcodec/vvc/Makefile |   4 +
>  4 files changed, 254 insertions(+)
>  create mode 100644 libavcodec/executor.c
>  create mode 100644 libavcodec/executor.h
>  create mode 100644 libavcodec/vvc/Makefile

This seems to need some fallback if pthreads are unavailable

src/libavcodec/executor.c: In function ‘executor_worker_task’:
src/libavcodec/executor.c:64:5: error: implicit declaration of function 
‘pthread_mutex_lock’; did you mean ‘ff_mutex_lock’? 
[-Werror=implicit-function-declaration]
 pthread_mutex_lock(&e->lock);
 ^~
 ff_mutex_lock
src/libavcodec/executor.c:78:13: error: implicit declaration of function 
‘pthread_mutex_unlock’; did you mean ‘ff_mutex_unlock’? 
[-Werror=implicit-function-declaration]
 pthread_mutex_unlock(&e->lock);
 ^~~~
 ff_mutex_unlock
src/libavcodec/executor.c:83:13: error: implicit declaration of function 
‘pthread_cond_wait’; did you mean ‘__fread_chk_warn’? 
[-Werror=implicit-function-declaration]
 pthread_cond_wait(&e->cond, &e->lock);
 ^
 __fread_chk_warn
src/libavcodec/executor.c: In function ‘ff_executor_alloc’:
src/libavcodec/executor.c:108:11: error: implicit declaration of function 
‘pthread_mutex_init’; did you mean ‘ff_mutex_init’? 
[-Werror=implicit-function-declaration]
 ret = pthread_mutex_init(&e->lock, NULL);
   ^~
   ff_mutex_init
src/libavcodec/executor.c:112:11: error: implicit declaration of function 
‘pthread_cond_init’ [-Werror=implicit-function-declaration]
 ret = pthread_cond_init(&e->cond, NULL);
   ^
src/libavcodec/executor.c:119:15: error: implicit declaration of function 
‘pthread_create’; did you mean ‘ff_thread_setname’? 
[-Werror=implicit-function-declaration]
 ret = pthread_create(&ti->thread, NULL, executor_worker_task, ti);
   ^~
   ff_thread_setname
src/libavcodec/executor.c:129:5: error: implicit declaration of function 
‘pthread_cond_broadcast’ [-Werror=implicit-function-declaration]
 pthread_cond_broadcast(&e->cond);
 ^~
src/libavcodec/executor.c:132:9: error: implicit declaration of function 
‘pthread_join’; did you mean ‘ff_thread_once’? 
[-Werror=implicit-function-declaration]
 pthread_join(e->threads[j].thread, NULL);
 ^~~~
 ff_thread_once
src/libavcodec/executor.c:133:5: error: implicit declaration of function 
‘pthread_cond_destroy’ [-Werror=implicit-function-declaration]
 pthread_cond_destroy(&e->cond);
 ^~~~
src/libavcodec/executor.c:135:5: error: implicit declaration of function 
‘pthread_mutex_destroy’; did you mean ‘ff_mutex_destroy’? 
[-Werror=implicit-function-declaration]
 pthread_mutex_destroy(&e->lock);
 ^
 ff_mutex_destroy
src/libavcodec/executor.c: In function ‘ff_executor_execute’:
src/libavcodec/executor.c:180:5: error: implicit declaration of function 
‘pthread_cond_signal’ [-Werror=implicit-function-declaration]
 pthread_cond_signal(&e->cond);
 ^~~
cc1: some warnings being treated as errors
ffmpeg/ffbuild/common.mak:81: recipe for target 'libavcodec/executor.o' failed
make: *** [libavcodec/executor.o] Error 1
make: *** Waiting for unfinished jobs

thx

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Why not whip the teacher when the pupil misbehaves? -- Diogenes of Sinope


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 01/14] vvcdec: add thread executor

2023-07-07 Thread Nuo Mi 
The executor design pattern was inroduced by java

it also adapted by python

Compared to handcrafted thread pool management, it greatly simplifies the 
thread code.
---
 libavcodec/Makefile |   1 +
 libavcodec/executor.c   | 182 
 libavcodec/executor.h   |  67 +++
 libavcodec/vvc/Makefile |   4 +
 4 files changed, 254 insertions(+)
 create mode 100644 libavcodec/executor.c
 create mode 100644 libavcodec/executor.h
 create mode 100644 libavcodec/vvc/Makefile

diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 1b0226c089..4a3c7a7a1f 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -62,6 +62,7 @@ OBJS = ac3_parser.o   
  \
xiph.o   \
 
 # subsystems
+include $(SRC_PATH)/libavcodec/vvc/Makefile
 OBJS-$(CONFIG_AANDCTTABLES)+= aandcttab.o
 OBJS-$(CONFIG_AC3DSP)  += ac3dsp.o ac3.o ac3tab.o
 OBJS-$(CONFIG_ADTS_HEADER) += adts_header.o 
mpeg4audio_sample_rates.o
diff --git a/libavcodec/executor.c b/libavcodec/executor.c
new file mode 100644
index 00..c5094f608a
--- /dev/null
+++ b/libavcodec/executor.c
@@ -0,0 +1,182 @@
+/*
+ * VVC video Decoder
+ *
+ * Copyright (C) 2022 Nuo Mi
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "libavutil/avutil.h"
+#include "libavutil/thread.h"
+
+#include "executor.h"
+
+typedef struct ThreadInfo {
+Executor *e;
+pthread_t thread;
+} ThreadInfo;
+
+struct Executor {
+TaskletCallbacks cb;
+ThreadInfo *threads;
+uint8_t *local_contexts;
+int thread_count;
+
+pthread_mutex_t lock;
+pthread_cond_t cond;
+int die;
+Tasklet *tasks;
+};
+
+static void remove_task(Tasklet **prev, Tasklet *t)
+{
+*prev  = t->next;
+t->next = NULL;
+}
+
+static void add_task(Tasklet **prev, Tasklet *t)
+{
+t->next = *prev;
+*prev   = t;
+}
+
+static void *executor_worker_task(void *data)
+{
+ThreadInfo *ti = (ThreadInfo*)data;
+Executor *e = ti->e;
+void *lc   = e->local_contexts + (ti - e->threads) * 
e->cb.local_context_size;
+Tasklet **prev;
+TaskletCallbacks *cb = &e->cb;
+
+pthread_mutex_lock(&e->lock);
+while (1) {
+Tasklet* t = NULL;
+if (e->die) break;
+
+for (prev = &e->tasks; *prev; prev = &(*prev)->next) {
+if (cb->ready(*prev, cb->user_data)) {
+t = *prev;
+break;
+}
+}
+if (t) {
+//found one task
+remove_task(prev, t);
+pthread_mutex_unlock(&e->lock);
+cb->run(t, lc, cb->user_data);
+pthread_mutex_lock(&e->lock);
+} else {
+//no task in one loop
+pthread_cond_wait(&e->cond, &e->lock);
+}
+}
+pthread_mutex_unlock(&e->lock);
+return NULL;
+}
+
+Executor* ff_executor_alloc(const TaskletCallbacks *cb, int thread_count)
+{
+Executor *e;
+int i, j, ret;
+if (!cb || !cb->user_data || !cb->ready || !cb->run || 
!cb->priority_higher)
+return NULL;
+e = av_calloc(1, sizeof(*e));
+if (!e)
+return NULL;
+e->cb = *cb;
+
+e->local_contexts = av_calloc(thread_count, e->cb.local_context_size);
+if (!e->local_contexts)
+goto free_executor;
+
+e->threads = av_calloc(thread_count, sizeof(*e->threads));
+if (!e->threads)
+goto free_contexts;
+ret = pthread_mutex_init(&e->lock, NULL);
+if (ret)
+goto free_threads;
+
+ret = pthread_cond_init(&e->cond, NULL);
+if (ret)
+goto destroy_lock;
+
+for (i = 0; i < thread_count; i++) {
+ThreadInfo *ti = e->threads + i;
+ti->e = e;
+ret = pthread_create(&ti->thread, NULL, executor_worker_task, ti);
+if (ret)
+goto join_threads;
+}
+e->thread_count = thread_count;
+return e;
+
+join_threads:
+pthread_mutex_lock(&e->lock);
+e->die = 1;
+pthread_cond_broadcast(&e->cond);
+pthread_mutex_unlock(&e->