Hi FFmpeg Development Community, My name is Mujtaba Shah, and I am a final-year Computer Science student. I am writing to propose a custom GSoC 2026 project focused on expanding FFmpeg’s machine learning capabilities, specifically by introducing an AI-powered video evaluation filter within libavfilter.
*The Concept:* While FFmpeg has made great strides with its DNN module and the modernized LibTorch backend, there is a growing demand for local, privacy-respecting video analysis. My proposal is to build a filter that integrates offline, local vision models directly into the FFmpeg pipeline. This would allow users to perform automated video evaluations—such as frame tagging, scene detection, or metadata extraction—entirely from the command line without relying on external API calls. *Proposed Scope of Work:* - Extend the existing DNN backend infrastructure to support a broader range of localized AI vision models for direct video evaluation. - Implement a new filter within libavfilter designed specifically for automated visual analysis and tagging of video streams. - Develop a robust FATE test suite to ensure the filter’s stability and accuracy across different environments. - Ensure seamless, zero-copy interactions where possible to maintain high throughput and low CPU-GPU latency. *Why I Am a Good Fit:* I have a strong background in AI, machine learning, and systems architecture. In my recent work, I have built specialized pipelines for the automated AI evaluation of video submissions and have hands-on experience configuring and deploying completely offline, uncensored AI models. This practical experience aligns perfectly with the goal of bringing robust, localized machine learning analysis directly into FFmpeg’s media processing workflows. *Next Steps & Qualification:* I understand that proposing a custom project requires finding a dedicated mentor. I would love to hear your thoughts on this idea's feasibility and whether anyone in the community would be interested in mentoring this effort. In the meantime, to demonstrate my familiarity with the codebase and build systems, I am currently looking into the "Improving Fate (self tests) coverage" tasks as my qualification requirement. Thank you for your time and for maintaining such an incredible project. I look forward to your feedback! Best regards, Mujtaba Shah _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
