We have a subtitles automation tool, made on ReactJS on FE. and NestJS on BE.
So the flow is that on the frontend first we show the user a preview of the subtitles, and give him options to modify the styling, the position of the subtitles or the subtitle container, and on the preview everything is done using CSS and JS. We are looking for a sync up between the values from the frontend to apply accurately on the backend. Our current approach is by generating PNG images using node-canvas and then streaming them out on the video at particular timestamps using overlay_filter. The first concern is of the scaling or the dimensions, as on the frontend we are using VideoJS, and of course for better view we are resizing the container as per devices etc, so the video's width & height on the frontend is less (mostly) than the actual dimensions of the video. While using that dimensions, the subtitles are too short to be rendered on the video (smaller) as the width and height of the canvas gets shorter. And also for the position, we are using react-draggble library for making the subtitles draggable and place them on video In the preview, still we lack on syncing up both the positions on the backend. We are just looking for a proper roadmap or solution for these blockers. Would appreciate your help on this. Thanks. Here's the current implementation of the code: wrapText(context, text, x, y, maxWidth, lineHeight) { const words = text.split(' '); let line = ''; for (let n = 0; n < words.length; n++) { const testLine = line + words[n] + ' '; const metrics = context.measureText(testLine); const testWidth = metrics.width; if (testWidth > maxWidth && n > 0) { context.fillText(line, x, y); line = words[n] + ' '; y += lineHeight; } else { line = testLine; } } context.fillText(line, x, y); } generateSubtitlePNG(transcription: any, outputPath: string, w, h) { const canvas = createCanvas(720, 200); const ctx = canvas.getContext('2d'); ctx.fillStyle = 'rgba(0, 0, 0, 0)'; ctx.fillRect(0, 0, canvas.width, canvas.height); const fontSize = 41; ctx.font = `bold ${fontSize}px Arial`; ctx.fillStyle = 'white'; ctx.textAlign = 'center'; const maxWidth = canvas.width; // 20 pixels padding on each side const lineHeight = fontSize * 1.4; // Adjust as needed const x = canvas.width / 2; const y = (canvas.height - lineHeight) / 2 + fontSize / 2; // Adjusted for font height this.wrapText(ctx, transcription.text, x, y, maxWidth, lineHeight); const buffer = canvas.toBuffer('image/png'); fs.writeFileSync(outputPath, buffer); } async applySubtitlesNew( addSubtitlesDto: AddSubtitlestDto, userId: number, project: any, ) { if (userId !== project.user.id) { throw new UnauthorizedException(); } const videoPath = `uploads/${project.originalVideoFile}`; const transcriptions = JSON.parse(addSubtitlesDto.transcriptions); const pngPaths = []; const dimensions = await this.getVideoDimensions(videoPath); console.log(dimensions); // Generate PNGs for each transcription for (let i = 0; i < transcriptions.length; i++) { const outputPath = `subtitles/${project.originalVideoFile}-subtitle-${i}.png`; this.generateSubtitlePNG( transcriptions[i], outputPath, dimensions?.width, dimensions?.height, ); pngPaths.push(outputPath); } let filterComplex = '[0:v]'; for (let i = 0; i < pngPaths.length; i++) { const start = transcriptions[i].start.toFixed(2); const end = transcriptions[i].end.toFixed(2); const overlayX = '(W-w)/2'; const overlayY = `H-h-10`; filterComplex += `[${ i + 1 }:v] overlay=${overlayX}:${overlayY}:enable='between(t,${start},${end})'`; if (i < pngPaths.length - 1) { filterComplex += '[vout];[vout]'; } } console.log(filterComplex); // Run FFmpeg return new Promise((resolve, reject) => { const ffmpegCommand = ffmpeg(); // Add video input ffmpegCommand.input(videoPath); // Add PNG inputs for (const pngPath of pngPaths) { ffmpegCommand.input(pngPath); } ffmpegCommand .complexFilter(filterComplex) .outputOptions('-c:v', 'libx264') .output('output.mp4') .on('end', () => { // Handle completion // Clean up PNG files for (const pngPath of pngPaths) { fs.unlinkSync(pngPath); } resolve('done'); }) .on('error', (error, stdout, stderr) => { console.log(stderr); // Handle error reject(error); }) .run(); }); } _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".