On date Tuesday 2024-06-04 15:47:21 +0100, Andrew Sayers wrote:
> Derived from explanations kindly provided by Stefano Sabatini and others:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2024-April/325903.html
> ---
> doc/context.md | 430 +++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 430 insertions(+)
> create mode 100644 doc/context.md
> diff --git a/doc/context.md b/doc/context.md
> new file mode 100644
> index 0000000000..bd8cb58696
> --- /dev/null
> +++ b/doc/context.md
> @@ -0,0 +1,430 @@
> +@page Context Introduction to contexts
> +
> +@tableofcontents
> +
> +FFmpeg uses the term “context” to refer to an idiom
> +you have probably used before:
> +
> +```c
> +// C structs often share context between functions:
> +
> +FILE *my_file; // my_file stores information about a filehandle
> +
> +printf(my_file, "hello "); // my_file provides context to this function,
> +printf(my_file, "world!"); // and also to this function
> +```
> +
> +```python
> +# Python classes provide context for the methods they contain:
> +
> +class MyClass:
> + def print(self,message):
> + if self.prev_message != message:
> + self.prev_message = message
> + print(message)
> +```
> +
> +<!-- marked "c" because Doxygen doesn't support JS highlighting: -->
> +```c
> +// Many JavaScript callbacks accept an optional context argument:
> +
> +const my_object = {};
> +
> +my_array.forEach(function_1, my_object);
> +my_array.forEach(function_2, my_object);
> +```
> +
> +Be careful comparing FFmpeg contexts to things you're already familiar with -
> +FFmpeg may sometimes happen to reuse words you recognise, but mean something
> +completely different. For example, the AVClass struct has nothing to do with
> +[object-oriented
> classes](https://en.wikipedia.org/wiki/Class_(computer_programming)).
> +
> +If you've used contexts in other C projects, you may want to read
> +@ref Context_comparison before the rest of the document.
My impression is that this is growing out of scope for a
reference. The doxy is a reference, therefore it should be clean and
terse, and we should avoid adding too much information, enough
information should be right enough. In fact, a reference is different
from a tutorial, and much different from a C tutorial. Also this is
not a treatise comparing different languages and frameworks, as this
would confuse beginners and would annoy experienced developers.
I propose to cut this patch to provide the minimal information you can
expect in a reference, but not more than that. Addition can be added
later, but I think we should try to avoid any unnecessary content, in
the spirit of keeping this a reference. More extensive discussions
might be done in a separate place (the wiki, a blog post etc.), but in
the spirit of a keeping this a reference they should not be put here.
> +
> +@section Context_general “Context” as a general concept
> +
> +@par
> +A context is any data structure used by several functions
> +(or several instances of the same function) that all operate on the same
> entity.
> +
> +In the broadest sense, “context” is just a way to think about code.
> +You can even use it to think about code written by people who have never
> +heard the term, or who would disagree with you about what it means.
> +Consider the following snippet:
> +
> +```c
> +struct DualWriter {
> + int fd1, fd2;
> +};
> +
> +ssize_t write_to_two_files(
> + struct DualWriter *my_writer,
> + uint8_t *buf,
> + int buf_size
> +) {
> +
> + ssize_t bytes_written_1 = write(my_writer->fd1, buf, buf_size);
> + ssize_t bytes_written_2 = write(my_writer->fd2, buf, buf_size);
> +
> + if ( bytes_written_1 != bytes_written_2 ) {
> + // ... handle this edge case ...
> + }
> +
> + return bytes_written_1;
> +
> +}
> +
> +int main() {
> +
> + struct DualWriter my_writer;
> + my_writer.fd1 = open("file1", 0644, "wb");
> + my_writer.fd2 = open("file2", 0644, "wb");
> +
> + write_to_two_files(&my_writer, "hello ", sizeof("hello "));
> + write_to_two_files(&my_writer, "world!", sizeof("world!"));
> +
> + close( my_writer.fd1 );
> + close( my_writer.fd2 );
> +
> +}
> +```
> +
> +The term “context” doesn't appear anywhere in the snippet. But `DualWriter`
> +is passed to several instances of `write_to_two_files()` that operate on
> +the same entity, so it fits the definition of a context.
> +
> +When reading code that isn't explicitly described in terms of contexts,
> +remember that your interpretation may differ from other people's.
> +For example, FFmpeg's avio_alloc_context() accepts a set of callback
> functions
> +and an `opaque` argument - even though this function guarantees to *return*
> +a context, it does not require `opaque` to *provide* context for the callback
> +functions. So you could choose to pass a struct like `DualWriter` as the
> +`opaque` argument, or you could pass callbacks that use `stdin` and `stdout`
> +and just pass a `NULL` argument for `opaque`.
I'd skip all this part, as we assume the reader is already familiar
with C language and with data encapsulation through struct, if he is
not this is not the right place where to teach about C language
fundamentals.
> +
> +When reading code that *is* explicitly described in terms of contexts,
> +remember that the term's meaning is guaranteed by *the project's community*,
> +not *the language it's written in*. That means guarantees may be more
> flexible
> +and change more over time. For example, programming languages that use
> +[encapsulation](https://en.wikipedia.org/wiki/Encapsulation_(computer_programming))
> +will simply refuse to compile code that violates its rules about access,
> +while communities can put up with special cases if they improve code quality.
> +
This looks a bit vague so I'd rather drop this.
> +The next section will discuss what specific conventions FFmpeg developers
> mean
> +when they describe parts of their code as using “contexts”.
> +
> +@section Context_ffmpeg FFmpeg contexts
> +
> +This section discusses specific context-related conventions used in FFmpeg.
> +Some of these are used in other projects, others are unique to this project.
> +
> +@subsection Context_indicating Indicating context: “Context”, “ctx” etc.
> +
> +```c
> +// Context struct names usually end with `Context`:
> +struct AVSomeContext {
> + ...
> +};
> +
> +// Functions are usually named after their context,
> +// context parameters usually come first and are often called `ctx`:
> +void av_some_function(AVSomeContext *ctx, ...);
> +```
> +
> +FFmpeg struct names usually signal whether they are contexts (e.g.
> AVBSFContext
> +or AVCodecContext). Exceptions to this rule include AVMD5, which is only
> +identified as a context by @ref libavutil/md5.c "the functions that call it".
> +
> +Function names usually signal the context they're associated with (e.g.
> +av_md5_alloc() or avcodec_alloc_context3()). Exceptions to this rule include
> +@ref avformat.h "AVFormatContext's functions", many of which begin with
> +just `av_`.
> +
> +Functions usually signal their context parameter by putting it first and
> +naming it some variant of `ctx`. Exceptions include av_bsf_alloc(), which
> puts
> +its context argument second to emphasise it's an out variable.
> +
> +Some functions fit awkwardly within FFmpeg's context idiom, so they send
> mixed
> +signals. For example, av_ambient_viewing_environment_create_side_data()
> creates
> +an AVAmbientViewingEnvironment context, then adds it to the side-data of an
> +AVFrame context. So its name hints at one context, its parameter hints at
> +another, and its documentation is silent on the issue. You might prefer to
> +think of such functions as not having a context, or as “receiving” one
> context
> +and “producing” another.
I'd skip this paragraph. In fact, I think that API makes perfect
sense, OOP languages adopt such constructs all the time, for example
this could be a static module/class constructor. In other words, we
are not telling anywhere that all functions should take a "context" as
its first argument, and the documentation specify exactly how this
works, if you feel this is not clear or silent probably this is a sign
that that function documentation should be extended.
> +
> +@subsection Context_data_hiding Data hiding: private contexts
> +
> +```c
> +// Context structs often hide private context:
> +struct AVSomeContext {
> + void *priv_data; // sometimes just called "internal"
> +};
> +```
> +
> +Contexts present a public interface, so changing a context's members forces
> +everyone that uses the library to at least recompile their program,
> +if not rewrite it to remain compatible. Many contexts reduce this problem
> +by including a private context with a type that is not exposed in the public
> +interface. Hiding information this way ensures it can be modified without
> +affecting downstream software.
> +
> +Private contexts often store variables users aren't supposed to see
> +(similar to an
> [OOP](https://en.wikipedia.org/wiki/Object-oriented_programming)
> +private block), but can be used for more than just access control. They can
> +also store information shared between some but not all instances of a context
> +(e.g. codec-specific functionality), and @ref Context_avoptions
> +"AVOptions-enabled structs" can provide user configuration options through
> +the @ref avoptions "AVOptions API".
I'll skip this section as well: data hiding is a common C technique,
and AVOptions are already covered later in the document or in another
dedicated section.
> +@subsection Context_lifetime Manage lifetime: creation, use, destruction
> +
> +```c
> +void my_function(...) {
> +
> + // Context structs are allocated then initialized with associated
> functions:
> +
> + AVSomeContext *ctx = av_some_context_alloc(...);
> +
> + // ... configure ctx ...
> +
> + av_some_context_init(ctx, ...);
> +
> + // ... use ctx ...
> +
> + // Context structs are closed then freed with associated functions:
> +
> + av_some_context_close(ctx);
> + av_some_context_free(ctx);
> +
> +}
> +```
> +FFmpeg contexts go through the following stages of life:
> +
> +1. allocation (often a function that ends with `_alloc`)
> + * a range of memory is allocated for use by the structure
> + * memory is allocated on boundaries that improve caching
> + * memory is reset to zeroes, some internal structures may be initialized
> +2. configuration (implemented by setting values directly on the context)
> + * no function for this - calling code populates the structure directly
> + * memory is populated with useful values
> + * simple contexts can skip this stage
> +3. initialization (often a function that ends with `_init`)
> + * setup actions are performed based on the configuration (e.g. opening
> files)
> +5. normal usage
> + * most functions are called in this stage
> + * documentation implies some members are now read-only (or not used at
> all)
> + * some contexts allow re-initialization
> +6. closing (often a function that ends with `_close()`)
> + * teardown actions are performed (e.g. closing files)
> +7. deallocation (often a function that ends with `_free()`)
> + * memory is returned to the pool of available memory
> +
> +This can mislead object-oriented programmers, who expect something more like:
> +
> +1. allocation (usually a `new` keyword)
> + * a range of memory is allocated for use by the structure
> + * memory *may* be reset (e.g. for security reasons)
> +2. initialization (usually a constructor)
> + * memory is populated with useful values
> + * related setup actions are performed based on arguments (e.g. opening
> files)
> +3. normal usage
> + * most functions are called in this stage
> + * compiler enforces that some members are read-only (or private)
> + * no going back to the previous stage
> +4. finalization (usually a destructor)
> + * teardown actions are performed (e.g. closing files)
> +5. deallocation (usually a `delete` keyword)
> + * memory is returned to the pool of available memory
> +
> +The remainder of this section discusses FFmpeg's differences from OOP, to
> help
> +object-oriented programmers avoid misconceptions. You can safely skip this
> +section if you aren't familiar with the OOP lifetime described above.
> +
> +FFmpeg's allocation stage is broadly similar to the OOP stage of the same
> name.
> +Both set aside some memory for use by a new entity, but FFmpeg's stage can
> also
> +do some higher-level operations. For example, @ref Context_avoptions
> +"AVOptions-enabled structs" set their AVClass member during allocation.
> +
> +FFmpeg's configuration stage involves setting any variables you want before
> +you start using the context. Complicated FFmpeg structures like
> AVCodecContext
> +tend to have many members you *could* set, but in practice most programs set
> +few if any of them. The freeform configuration stage works better than
> bundling
> +these into the initialization stage, which would lead to functions with
> +impractically many parameters, and would mean each new option was an
> +incompatible change to the API. One way to understand the problem is to read
> +@ref Context_avoptions "the AVOptions section below" and think how a
> constructor
> +would handle those options.
> +
> +FFmpeg's initialization stage involves calling a function that sets the
> context
> +up based on your configuration.
> +
> +FFmpeg's first three stages do the same job as OOP's first two stages.
> +This can mislead object-oriented developers, who expect to do less work in
> the
> +allocation stage, and more work in the initialization stage. To simplify
> this,
> +most FFmpeg contexts provide a combined allocator and initializer function.
> +For historical reasons, suffixes like `_alloc`, `_init`, `_alloc_context` and
> +even `_open` can indicate the function does any combination of allocation and
> +initialization.
> +
> +FFmpeg's "closing" stage is broadly similar to OOP's "finalization" stage,
> +but some contexts allow re-initialization after finalization. For example,
> +SwrContext lets you call swr_close() then swr_init() to reuse a context.
> +Be aware that some FFmpeg functions happen to use the word "finalize" in a
> way
> +that has nothing to do with the OOP stage (e.g. av_bsf_list_finalize()).
> +
> +FFmpeg's "deallocation" stage is broadly similar to OOP, but can perform some
> +higher-level functions (similar to the allocation stage).
> +
> +Closing functions usually end with "_close", while deallocation
> +functions usually end with "_free". Very few contexts need the flexibility
> of
> +separate "closing" and "deallocation" stages, so many "_free" functions
> +implicitly close the context first.
About this I have mixed feelings, but to me it sounds like a-posteriori
rationalization.
I don't think there is a general rule with the allocation/closing/free
rule for the various FFmpeg APIs, and giving the impression that this
is the case might be misleading. In practice the user needs to master
only a single API at a time (encodering/decoding, muxing/demuxing,
etc.) each one with possibly slight differences in how the term
close/allocation/free are used. This is probably not optimal, but in
practice it works as the user do not really need to know all the
possible uses of the API (she will work through what she is interested
for the job at hand).
> +
> +@subsection Context_avoptions Configuration options: AVOptions-enabled
> structs
> +
> +The @ref avoptions "AVOptions API" is a framework to configure user-facing
> +options, e.g. on the command-line or in GUI configuration forms.
This looks wrong. AVOptions is not at all about CLI or GUI options, is
just some way to set/get fields (that is "options") defined in a
struct (a context) using a high level API including: setting multiple
options at once (through a textual encoding or a dictionary),
input/range validation, setting more fields based on a single option
(e.g. the size) etc.
Then you can query the options in a given struct and create
corresponding options in a UI, but this is not the point of AVOptions.
> +To understand FFmpeg's configuration requirements, run `ffmpeg -h full` on
> the
> +command-line, then ask yourself how you would implement all those options
> +with the C standard [`getopt`
> function](https://en.wikipedia.org/wiki/Getopt).
> +You can also ask the same question for other approaches - for example, how
> would
> +you maintain a GUI with 15,000+ configuration options?
> +
> +Most solutions assume you can just put all options in a single code block,
> +which is unworkable at FFmpeg's scale. Instead, we split configuration
> +across many *AVOptions-enabled structs*, which use the @ref avoptions
> +"AVOptions API" to inspect and configure options, including in private
> contexts.
> +
> +AVOptions-accessible members of a context should be accessed through the
> +@ref avoptions "AVOptions API" whenever possible, even if they're not hidden
> +in a private context. That ensures values are validated as they're set, and
> +means you won't have to do as much work if a future version of FFmpeg changes
> +the allowed values.
> +
> +Although not strictly required, it is best to only modify options during
> +the configuration stage. Initialized structs may be accessed by internal
> +FFmpeg threads, and modifying them can cause weird intermittent bugs.
> +
> +@subsection Context_logging Logging: AVClass context structures
> +
> +FFmpeg's @ref lavu_log "logging facility" needs to be simple to use,
> +but flexible enough to let people debug problems. And much like options,
> +it needs to work the same across a wide variety of unrelated structs.
> +
> +FFmpeg structs that support the logging framework are called *@ref AVClass
> +context structures*. The name @ref AVClass was chosen early in FFmpeg's
> +development, but in practice it only came to store information about
> +logging, and about options.
OTOH hand AVOptions and logging should be discussed in the relevant
files, to avoid duplication.
> +
> +@section Context_further Further information about contexts
> +
> +So far, this document has provided a theoretical guide to FFmpeg contexts.
> +This final section provides some alternative approaches to the topic,
> +which may help round out your understanding.
> +
> +@subsection Context_example Learning by example: context for a codec
> +
> +It can help to learn contexts by doing a deep dive into a specific struct.
> +This section will discuss AVCodecContext - an AVOptions-enabled struct
> +that contains information about encoding or decoding one stream of data
> +(e.g. the video in a movie).
> +
> +The name "AVCodecContext" tells us this is a context. Many of
> +@ref libavcodec/avcodec.h "its functions" start with an `avctx` parameter,
> +indicating this parameter provides context for that function.
> +
> +AVCodecContext::internal contains the private context. For example,
> +codec-specific information might be stored here.
> +
> +AVCodecContext is allocated with avcodec_alloc_context3(), initialized with
> +avcodec_open2(), and freed with avcodec_free_context(). Most of its members
> +are configured with the @ref avoptions "AVOptions API", but for example you
> +can set AVCodecContext::draw_horiz_band() if your program happens to need it.
> +
> +AVCodecContext provides an abstract interface to many different *codecs*.
> +Options supported by many codecs (e.g. "bitrate") are kept in AVCodecContext
> +and exposed with AVOptions. Options that are specific to one codec are
> +stored in the private context, and also exposed with AVOptions.
> +
> +AVCodecContext::av_class contains logging metadata to ensure all
> codec-related
> +error messages look the same, plus implementation details about options.
> +
> +To support a specific codec, AVCodecContext's private context is set to
> +an encoder-specific data type. For example, the video codec
> +[H.264](https://en.wikipedia.org/wiki/Advanced_Video_Coding) is supported via
> +[the x264 library](https://www.videolan.org/developers/x264.html), and
> +implemented in X264Context. Although included in the documentation,
> X264Context
> +is not part of the public API. That means FFmpeg's @ref ffmpeg_versioning
> +"strict rules about changing public structs" aren't as important here, so a
> +version of FFmpeg could modify X264Context or replace it with another type
> +altogether. An adverse legal ruling or security problem could even force us
> to
> +switch to a completely different library without a major version bump.
> +
> +The design of AVCodecContext provides several important guarantees:
> +
> +- lets you use the same interface for any codec
> +- supports common encoder options like "bitrate" without duplicating code
> +- supports encoder-specific options like "profile" without bulking out the
> public interface
> +- exposes both types of options to users, with help text and detection of
> missing options
> +- provides uniform logging output
> +- hides implementation details (e.g. its encoding buffer)
> +
> +@subsection Context_comparison Learning by comparison: FFmpeg vs. Curl
> contexts
About this, I'm still not really convinced that this should be part of
a reference, in the sense that it is adding more information than
really needed and it treats concepts related to the C language rather
than to the FFmpeg API itself.
[...]
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
