Hi all, it's been a while since the last LiVES news, so I thought it was worth an update. Last month saw the release of version 1.6.2 which added support for fast opening of hdv clips amongst other things. Since then onlt two minor bugs have been reported: a problem with initial startup when pulseaudio is not installed (http://lives.svn.sourceforge.net/viewvc/lives/trunk/src/startup.c?r1=1446&r2=1509) and a problem with filenames being truncated in the plugin interfaces (http://lives.svn.sourceforge.net/viewvc/lives/trunk/src/paramspecial.c?r1=1447&r2=1510).
Now, some of you may recal that I am currently working for a Masters in Applied IT, and my research project involves digital signal processing in LiVES. I have just completed the first half of my course and so it is a good time to mention my progress on that. The idea behind the project is that LiVES should be able to process other type of data as well as video, audio and subtitles. In order to achieve this I have created some new types of realtime plugins. Video analyser plugins: - blank frame detector: this plugin looks for frames where the luminosity (brightness) of all the pixels is below a certain user-defined threshold value. As well as the threshold, there is also a "count" parameter which defines how many consecutive frames must pass this test. The plugin has a boolean (int) output, which is triggered (TRUE or 1) when the conditions are met. - farneback motion detector: this plugin uses openCV, in particular the Farneback dense motion detection algorithm. The outputs are two float alpha channels, the x and y values for each pixel indicating its motion vector. - haar analyser: this plugin performs a Haar transform (a kind of 2 dimensional Fourier transform), on YUV version of the input frames. The outputs are 3 arrays of integer (one for each plane Y,U,V). The array length can be set by the user. This type of plugin can be used to classify similar looking images. Data processing plugins: data can be processed without touching video or audio or subtitles. I created an experimental plugin which linked to Weka (a java based data processing tool). However this plugin was a little complicated to get installed and running (due to use of java JNI), so it is isabled by default. However the code can be found in the "experimental" subdirectory of weed-plugins in case anybody wants to experiment with linking LiVES with java. Data visualiser plugins: - cairo vector visualiser: this plugin takes two float alpha channel inputs and overlays arrows on the video frame. This may be used as the output for the Farneback analyser for example. Audio generator plugins: - tone generator: this plugin takes two float inputs - a frequency (Hz) value, and a multiplier value (which can be used to scale the frequency). It generates a square wave at the selected frequency. Audio analyser plugins: - audio_fft plugin: a work-in-progress, this plugin takes the audio and outputs a float amplitude for a selected frequency. - beat detector: this has been requested on here before, but it will use FFT and statistical methods to detect beats in the audio. It has two outputs: - a beat pulse output, and a beat & hold output. There is also a reset input which will reset the beat & hold output. This is so that video effects can log the beat from this filter (which runs at audio speed), and then reset. Still to come: Audio processing plugins: I hope to be able to create a wrapper around some LADSPA plugins to enable realtime audio effects. So, as you can see this work is coming on quite well in the background. The next phase will be to create a graphical interface which will allow the data connections to be made within LiVES (most probably in the realtime effect mapper). The details for this are still up in the air, so if anybody has any good suggestions we can discuss this on the lives-devel mailing list. And of course, it should be possible to connect and disconnect data pipes through the LiVES OSC interface. Other ideas which I would like to implement if time allows: - chaining of plugins into a single effect slot (for example motion detector -> averager -> frame shifter could give a nice camera shake reduction filter). - more plugins: - speech to text, text to speech, music generator, "kicker" plugin (for output from the beat detector). If anybody is interested in knowing more about this, please feel free to ask. I am open to suggestions if anybody has any ideas related to this. Regards, Gabriel "salsaman", main developer, LiVES. http://lives.sourceforge.net https://www.ohloh.net/accounts/salsaman ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Lives-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/lives-users
