I'm developing a library in which you can push in a stream of bytes to a
decoder for a specific codec (right now I'm testing with OGG), and it
will send back partial audio packets as it becomes available, plus a
final stream of decoded audio once the user indicates the stream is
finished. This data could come in at any time with any size, and I keep
a buffer of the unprocessed data until it's consumed. I want to be able
to support partial frames/streaming as well, so I need to have the data
processing in the background before the finish method is called. The
library is written in Java and is interfaced with in Lua, but it uses a
native wrapper to FFmpeg in C (https://github.com/Manevolent/ffmpeg4j).
For example, here's an example of how a user could pipe data from a file
and then wait for the result:
local file = io.open("audio.ogg", "rb") local bus =
peripheral.wrap("back") -- gets the decoder object repeat local data =
file:read(16384) if data == nil then break end bus.input(1, data) --
send 16384 bytes to the decoder in slot 1 and process it in the
background until #data < 16384 file:close() bus.finish(1) -- tell the
decoder that the data is finished local event, side, slot, ok, data =
coroutine.yield("asicraft.result") -- wait for processing to finish &
get the result -- do things with data
I'm currently using the Java library's wrapper around an `InputStream`
to send the data. I have my own `InputStream` which holds the buffered
input, and the library will call its `read` method through an
implementation of libavformat `AVIOContext`'s `read_packet` function. If
the buffer runs out, the read method returns -1, and I used some
mixins/reflection on the Java library to make it return
`AVERROR(EAGAIN)` to libavformat. This worked for the first packet sent
to the stream, but despite always sending `EAGAIN` I was still getting
EOFs simply from returning less data than the demuxer expected. To solve
this, I forced the `eof_reached` flag to be reset to 0 each time data is
added to the stream, but because a partial amount of data was read
during the last decoding period, it ends up skipping that frame.
I've considered putting the audio processing on a separate thread, and
having the `read_packet` function wait for more input before returning
(which luckily Java makes pretty easy). But this would mean that I would
need a new thread for every single input process, which theoretically
could be opened and then left running forever, and I want to avoid
ending up with thread exhaustion (this needs to be able to run on public
servers where users can trigger the decoder at any time). I also tried
throwing an exception to attempt to unwind the reader back across
`av_frame_read`, but this failed spectacularly as I expected.
What's the most optimal way to decode/demux a data stream in chunks that
may arrive at any time, without losing any frames in the process? And,
more specifically for my current method, is there any way to figure out
where in the data stream the last frame was read by the demuxer? If I
can know that, I can just rewind the data stream to that point and
continue the decoder.
If anyone wants to look at my current code, it's available at
https://github.com/MCJack123/ASICraft/tree/ffmpeg-decode/common/src/main/java/cc/craftospc/ASICraft/algorithms/OGGDecodeAlgorithm.java.
I don't expect anyone to want to poke through it, but it's there if
someone does. There's also some related code in
`util/FFmpegInputFixed.java` and `mixin/FFmpegSourceStreamInject.java`
for patching a few things - mainly, switching it to use `EAGAIN` instead
of EOF.
I would appreciate any help in getting this working, as it drives a
major part of my project, and I don't want to have to drop it or limit
the features because I can't get libav to cooperate with my input scheme.
_______________________________________________
Libav-user mailing list
Libav-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/libav-user
To unsubscribe, visit link above, or email
libav-user-requ...@ffmpeg.org with subject "unsubscribe".
