Hi Codec2 list, I'm working together with Samih, looking at shrinking Khan Academy and other educational content for our offline library app. I've been trying to figure out optimal codec2 encoding / decoding parameters.
We know who the speaker is in each clip. As far as I can understand, the best approach for us to achieve optimal results with a fixed speaker set, with having access to the original would be using the masking model outlined here: https://www.rowetel.com/?p=4454. Is this masking model per speaker, or per clip? I haven't managed to get the masking model running yet, but I made a basic script ( https://gist.github.com/mikedawson/1d66a1d35bd1538b2a9950246ef061a2 ) to generate comparison tables using a basket of clips and different parameter combinations. The audio from 4 Khan Academy clips with different codec2 settings is here: https://www.ustadmobile.com/files/codec2/out/ Using VP9 compression, the video in a 3.5 min clip can be shrunk to just under 100kB. If we used 2.4kbps codec2 for the audio, we could get the audio to around 70kB. As there are around 15,000 videos (only in English), codec2 could save a huge amount of space and bandwidth. That makes it around 60-70% smaller than the smallest 'mobile friendly' mp4 version from Khan Academy. On the LPCNet topic: this is definitely interesting, but will need further investigation. The examples from the masking model sounded pretty good. One obstacle I can see is the size of the training file. The app has to work offline and we have to keep the app size itself as small as possible. Perhaps with a limited speaker set, and no need to work on untrained files, this would not be so bad. We would also need to get the model to work with Tensorflow lite. Finally, in many places where low bandwidth and device space is an issue, the phones themselves often have limited capacity (Android 4.4 is still very much alive). Any further suggestion on what would be the current recommended / optimal approach for a fixed set of speakers would be much appreciated! We're very excited about the potential of this to make this education content more accessible. Thanks! -Mike CEO Ustad Mobile Email: [email protected] Web: www.ustadmobile.com Twitter: @ustadmobile Facebook: www.facebook.com/Ustad.Mobile _______________________________________________ Freetel-codec2 mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/freetel-codec2
