[Mlt-devel] How to view simultaneously unprocessed frames and processed ones in two X windows?
Hi all. I am writing a basic video editor based on MLT. At the moment, I use an mlt_playlist, which lets me append video files, break them in clips, and connect different filters to different clips. So far so good, I appreciate the clean C interface of MLT: congratulations. But now I'm facing a problem I don't know how to manage. In my editor, I would like to always see two pictures, the original one and the manipulated one. I thought I could use a "multi" consumer, but this multi consumer should not be attached to the playlist itself because, I suppose, the playlist spits out frames already processed by the different filters attached to the single clips. Perhaps I should attach a multi consumer to *every* clip I append in the playlist? Could it work? Then every clip would have a multi consumer which would duplicate the frames: one leg would go to the subsequent filter chain, and the other leg would send unprocessed frames to my "original clip" window. Or is there another way, more simple? Thank you, regards, linuxfan ___ Mlt-devel mailing list Mlt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mlt-devel
[Mlt-devel] R: Re: How to view simultaneously unprocessed frames and processed ones in two X windows?
Thank you for your reply. >> I thought I could use >> a "multi" consumer, but ... > > That will not work. MLT does not have a general tee-like component. > I have never tried to do what you are doing, but I think you are going to > have to compose and run distinct graphs: one that is unfiltered and another > that is filtered and attach a consumer to each. Yes, that was the idea, but how can I compose distinct graphs if there is not a tee component? I could try to write one, but in my plans that was ahead in time. > Alternatively, you could > write a new filter that stores a snapshot of the frame's current image to > some unique property. Add this filter before all other filters. Then, when > the consumer emits a frame-displayed event, get this snapshot image from > the frame to display in another window. Also this idea seems viable. I would try the "distinct graphs" way first, if you tell me how to do it. If that fails, I will try the filter way. Have a nice day, regards, linuxfan ___ Mlt-devel mailing list Mlt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mlt-devel
[Mlt-devel] R: Re: How to view simultaneously unprocessed frames and processed ones in two X windows?
Hi all, I succeeded in creating a preview of the unfiltered frames, as suggested by Dan: I wrote a filter, first in the chain, which extracts the image. Now I have another problem: I would like to process frames to inspect the audio levels. I thought I can use the audiolevel filter and ask it to process every single frame. I make a loop for every frame I am interested in, call mlt_service_get_frame(...), then call mlt_filter_process(audiolevel, frame). It works, but only if I make a call to sleep(). It seems to me, that it works only because the consumer, because the sleep(), has a chance to request the final frame to display, and doing so all the filters are run. Is there a way I can get frame by frame, and run on them only the filters I want, without using a consumer? Thank you, linuxfan ___ Mlt-devel mailing list Mlt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mlt-devel
[Mlt-devel] R: Re: R: Re: Extract audio levels from a clip
> The filter will not process the audio until you call mlt_frame_get_audio() Thank you for the answer. I tried to follow the sources of mlt and shotcut, but without success... I will try your suggestion. >Why not listen to the "consumer-frame-render" event from the consumer and request the level of the frame after all the filters have been processed? Because I want the maximum speed. Actually, I want to draw the peak audio level of an entire clip, like shotcut does. So I inspected shotcut, didn't fully understand it, and thought anyway that I could use the audiolevel filter. This is just the first job, probably I will in future try to implement other batch jobs, for example a scene change detector or other statistics analisys. -- >Now I have another problem: I would like to process frames >to inspect the audio levels. I thought I can use the audiolevel filter >and ask it to process every single frame. I make a loop for every frame >I am interested in, call >mlt_service_get_frame(...), then call >mlt_filter_process(audiolevel, frame). > >It works, but only if I make a call to sleep(). >Is there a way I can >get frame by frame, and run on them only the filters I >want, without >using a consumer? > >Thank you, >linuxfan > > > > >___ >Mlt-devel mailing list >Mlt-devel@lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/mlt-devel> ___ Mlt-devel mailing list Mlt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mlt-devel
[Mlt-devel] Problem using mlt framework
Hello all.
I wrote an mlt-based video editor that is working quite
well. Now,
I am trying to write a program to extract thumbnails from a
video clip,
but nothing works as expected... and it is strange because
I am
using the same code snippets of the previous (video editor)
program.
I am getting crazy because there must be something I am
overlooking,
but the code is so simple, I don't know what else to try
or debug.
The code is in pascal, but should be similar enough to C to
be
understandable:
--
mltrepo := mlt_factory_init(''); // <- parameter=empty string
mltprofile := mlt_profile_init('');
consumer := mlt_factory_consumer
(mltprofile, 'sdl2', '320x240');
mlt_properties_set_int
(mlt_consumer_properties(consumer), 'audio_off', 1);
//
mlt_properties_set_int(mlt_consumer_properties(consumer), 'real_time',
1);
// mlt_properties_save(mlt_consumer_properties(consumer),
'/tmp/tbe_consfirst.txt');
producer := mlt_factory_producer
(mltprofile, 'avformat', thefilename);
// reopen to make mlt read
always the correct length (?)
mlt_profile_from_producer(mltprofile,
producer);
mlt_producer_close(producer);
producer :=
mlt_factory_producer(mltprofile, 'avformat', thefilename);
duration :
= mlt_properties_get_int(mlt_producer_properties(producer), 'length');
mlt_consumer_connect(consumer, mlt_producer_service(producer));
mlt_consumer_start(consumer);
--
By executing the code
above, an SDL window opens and the first frame
of the video shows up. I
have a trackbar (widget) which does the
following:
mlt_producer_seek
(producer, tbPos.Position);
By moving the tbPos cursor, the correct
seeks are made and the SDL window correctly and quickly shows the right
video frames.
But if I, instead, execute the following line:
mlt_producer_set_speed(producer, 1);
the video does not play and the
trackbar tbPos ceases to work.
I have another problem. What I really
want is not to play the video,
but to extract a thumbnail after a seek
is done. The code is:
--
srv := mlt_producer_service(producer);
mlt_producer_seek(producer,
tbPos.Position);
frame := NIL; // it seems better to clear frame to
NULL
if mlt_service_get_frame(srv, frame, 0)=0 then begin
image :
= NIL;
if mlt_frame_get_image(frame, @image, @iformat, @iwidth,
@iheigth, 0)=0 then begin
framenum := mlt_properties_get_int
(mlt_frame_properties(frame), '_position');
case iformat of
mlt_image_yuv422: ... (not implemented yet: extract from
yuv422)
mlt_image_rgb24: ... (not implemented yet: extract
from rgb24)
end;
// mlt_frame_close(frame); // closing the
frame seems worse than not doing it
end
--
If, after a seek(), I try
to get the image with the above code, I obtain
the correct iformat,
iwidth and iheight, but the SDL window stops to work.
The frame remains
untouched.
I am forgetting something? The same chunks of code posted
here work
quite well in my other program, the video editor, included
the routines
which extract audio and images frame by frame.
Thank you
and regards,
linuxfan
___
Mlt-devel mailing list
Mlt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mlt-devel
[Mlt-devel] Problem with crop
Hi all,
I am writing a video editor using mlt. The editor basically
edits a playlist, inserts filters, and does preview via a swallowed SDL
window. When requested, the editor generates an XML file to be fed to
melt(1). So far so good.
In order to crop clips, I used until now the
stock "crop" filter; it works well but, in the editor, I have to put
two instances of it, otherwise the crop is not performed. So, I wrote
my own crop filter, which crops in-place instead of setting some
properties and letting another instance of itself to do the work. The
filter works well when used in the editor ("internal instance") but,
when used by melt(1), the result is wrong. I am sure my filter does the
correct job, but it seems the image it receives, to be cropped, gets
scaled down, even if width and height are correct.
For example, I take
a 320x240 clip and crop 160 pixels from left. In the editor, the result
is a 160x240 picture - correct. When using melt, the output written in
the file (consumer avformat) is correctly 160x240, but the contents are
the wanted picture 80 pixels wide, padded to the right with 80 blank
pixels.
It is about a week I am trying to solve. I tried to set frame
properties in the filter, "resize_width", "resize_height", with all the
possible values, before and after the cropping - nothing happens, the
result is always the same.
Then I ran melt with -debug, and saw that a
swscale filter kicks in and resizes 320 -> 160. Of course, no resizing
should be done, and in fact this does not happen when I am using my
interactive editor. Still this does not explain why the final image is
blank-padded on the right, maybe another unwanted filter kicks in.
My
question is: why melt uses somewhere a rescale filter, when nobody
asked for it? How can I just take the original frame, crop/resize it at
will, and pass it to the consumer (or other filters)? When the
"pipeline" is constructed in the editor it works well, apparently melt
does something under the hood which I need to exclude. What should I do
to make melt replicate the same behavior of the interactive editor?
Regards,
linuxfan
___
Mlt-devel mailing list
Mlt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mlt-devel
[Mlt-devel] R: Re: Problem with crop
Thank you Dan for your reply.
I completely agree with you - a crop
filter should be the first in the chain (and it is, even in my
implementation), and the MLT framework should not be changed only
because it would be handy for me.
Meanwhile, I searched another way to
go ahead: i wrote the "dbvemelt" program, essentialy a cut-down version
of melt(1), which has a GUI to see the progress. This program loads an
XML, then inspects every clip in the playlist described by the XML, and
creates a new playlist. The processing is the same used by the editor
when it loads a "project". This dbvemelt should give me more control
over the framework behavior. I've not finished to experiment, but I've
already found a strange thing: after creating a playlist, which already
contains my own crop filter where applicable, the program instantiates
an avformat consumer, and passes to it the parameters chosen via the
editor (compression, bitrates and so on). Same problem! But the problem
goes away if I omit to tell avformat the size of the output video (i.e.
omit to set s=WxH). Given that I have full control over my own crop
filter, now I am getting totally confused. The filter can both return
the dimensions (WxH) it is asked for (by consumer avformat), or decide
to change them... but it seems that if avformat is told (via
s=property) the final dimensions, again some other component kicks in
and makes a mess. Actually it is not clear to me how to manage these
things, but I will try to understand better.
Have nice day, regards,
linuxfan
>
>melt and the xml module require using the loader
producer. The loader
>producer is very special and chooses which real
producer to use based on
>the resource string - configured in loader.
dict - as well as adding what
>we call normalization (aka conform)
filters - configured in loader.ini.
>Extremely important filters that
loader adds that are not in loader.ini are
>image and audio converters
to convert image and audio data between services
>as needed. However,
the API lets you easily bypass this loader producer
>when you
explicitly pass the name of the service as an argument to the
>corresponding factory function. If you do not use loader, then you
really
>ought to add the conversion filters manually to each producer.
>
>See mlt_factory_producer() in
>https://www.mltframework.org/doxygen/mlt__factory_8c.html#a093871823fdf4dad4aee091e98832586
>
>It says "service" is optional and to see about MLT_PRODUCER. Not
mentioned
>is that this is an environment variable. It is documented
under "Related
>Pages" of Doxygen docs:
>https://www.mltframework.org/doxygen/envvars.html>
>And then here is
docs about loader producer:
>https://mltframework.org/plugins/ProducerLoader/>
>Basically, if you
want full control as you have done, then you will not be
>able to use
MLT XML and melt. Sorry, but I will not make changes to make
>these to
work without the loader producer. You can set the environment
>variable
MLT_PRODUCER to empty but then xml and melt are essentially
>broken.
The concept of the loader producer is so heavily integrated into
>how
all of the mature, established video editors and melt use MLT that I am
>unwilling to risk the change. It also goes without saying that MLT is
>really only ever used and tested using the loader producer. If you
want to
>continue to not use the loader producer, then you must create
your own way
>to serialize, deserialize, and render.
>
>For the crop
filter, this is a source image crop and, by design, must come
>before
all other filters including the ones that loader adds to
>deinterlace,
scale, and pad to conform the source to the mlt_profile. The
>second
instance of the crop filter then provides only the parameters to the
>one previously added by loader. There is also a qtcrop filter that
fills
>the area outside of a shape with a color including alpha.
>
>
>On Sun, Jan 31, 2021 at 1:39 PM linux...@tin.it
wrote:
>
>> Hi all,
>>
>> I am writing a video editor using mlt. The
editor basically
>> edits a playlist, inserts filters, and does preview
via a swallowed SDL
>> window. When requested, the editor generates an
XML file to be fed to
>> melt(1). So far so good.
>>
>> In order to
crop clips, I used until now the
>> stock "crop" filter; it works well
but, in the editor, I have to put
>> two instances of it, otherwise the
crop is not performed. So, I wrote
>> my own crop filter, which crops
in-place instead of setting some
>> properties and letting another
instance of itself to do the work. The
>> filter works well when used
in the editor ("internal instance") but,
>> when used by melt(1), the
result is wrong. I am sure my
[Mlt-devel] R: Re: Problem with crop
Hi again, I only want to signal that I discovered something strange: my crop filter has 6 properties: left-right-top-bottom, plus "width" and "height". They work correctly in the editor but, when exporting in XML, "width" and "height" are omitted from the XML file. Of course this fact polluted my previous tries... I renamed those properties "nwidth" and "nheight", and now they get correctly written out. I suspect there is a reason for that, but I can't see it. Regards, linuxfan ___ Mlt-devel mailing list Mlt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mlt-devel
