Hello all, first post. I am trying to segment mov videos from multiple Zoom Q8 cameras with exactly the same specs, by using the Linux command ltcdump, bash and ffmpeg.
The LTC is being generated by a Tentacle Sync for each camera, that was "jammed" to a Zoom F8 which is master audio I use to sync the final video via LTC ... I have this working fine on each camera individually using Ardour. The goal is to concat the segments from these cameras into one video. The main idea is the videos switch cameras at a fixed duration determined at runtime by a simple constant like 20 seconds, determined by reading the output from ltcdump on each video. I have this very close to working. The last step is fixing an audio drift problem caused during the transition from one camera to another. This drift is because I cannot figure out how to exactly match an LTC audio frame to each segment. I tried lots of numbers for the segment and was unsuccessful. Here is an example of a transition and the problem I am trying to fix. Q81 is camera 1, Q82 is camera 2. I am missing frames 28 and 29 - the last .xx digits are the frame and not milliseconds. And the LTC frame of somewhere between 1597 and 1601 samples at 48KHZ does not match the segment exactly. ### ltcdump output from file q81_out280.mov.wav , last two .xx digits are frames #User bits Timecode | Pos. (samples) #DISCONTINUITY 00000000 00:26:34:27 | 1221 2821 ### ltcdump output from file q82_out305.mov.wav , last two .xx digits are frames #User bits Timecode | Pos. (samples) #DISCONTINUITY 00000000 00:26:35:00 | 818 2418 What I believe I need to do is get the segment to match the LTC frame as closely as possible. In these examples the segment I believe I need is 1600 / 48000. However I tried and could not get the segments to match. Here's my code to show what I am doing. Its about 300 lines. #!/bin/bash # This script generates a single video from parts of videos from 2 cameras by using ltc. # Videos are mov at 1280x740 with audio at 48KHZ and 16 bits # # usage: # /home/myuser/input> ls # output parseLTC.sh q81.mov q82.mov # /home/myuser/input> cd output/ # /home/myuser/input/output> sh ../parseLTC.sh # clear any files from previous processing rm -f *.txt rm -f *all* rm -f *out* # Trim first camera so the LTC frame starts as closely as possible to zero # 00000000 00:26:15:00 | 2 1615 ffmpeg -y -v error -ss 00:00:06.067697 -t 00:30:15.365 -i ../q81.mov -c copy q81t.mov mv q81t.mov q81.mov mv ../q82.mov q82.mov # debug ffmpeg -y -v error -i q81.mov -vn -c:a pcm_s16le -ar 48000 -ac 2 q81.wav ffmpeg -y -v error -i q82.mov -vn -c:a pcm_s16le -ar 48000 -ac 2 q82.wav # split file by key frames in a .07 second duration, need a couple entries that # ltcdump can find in each file - ideally just a single entry. # first Q8 ffmpeg -y -v error -fflags +genpts -i q81.mov -vsync 1 -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af "aresample=async=1:min_hard_comp=0.100000:first_pts=0" -segment_time 0.07 -segment_list q81t.ffcat -sc_threshold 0 -force_key_frames "expr:gte(t,n_forced*0.07)" -f segment -reset_timestamps 1 -crf 0 q81_out%03d.mov # second Q8 ffmpeg -y -v error -fflags +genpts -i ../q82.mov -vsync 1 -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af "aresample=async=1:min_hard_comp=0.100000:first_pts=0" -segment_time 0.07 -segment_list q82t.ffcat -sc_threshold 0 -force_key_frames "expr:gte(t,n_forced*0.07)" -f segment -reset_timestamps 1 -crf 0 q82_out%03d.mov # save the ffcat output to run again if need be while debugging cp q81t.ffcat sav_q81t.ffcat cp q82t.ffcat sav_q82t.ffcat # some errors I found while playing option bingo with ffmpeg validate() { fileToProcess='' fileToProcess=$1 epoch_secs_ltc_to_process=0 ltc_to_process_formatted='' isLTCDumpOutputValid=0 found_stop_time=0 # test for shell error printf "\nExecuting: ltcdump -f 29.97 "$fileToProcess" " # adding frame rate removes this DISCONTINUITY # 00000000 00:26:53:24 | 7224 8824 #DISCONTINUITY ltcdump -f 29.97 "${fileToProcess}" > /dev/null 2>&1 if [ $? -ne 0 ] ; then printf "\nError processing file %s\n" "${fileToProcess}" printf "\nshell returned non zero value on validity check" return fi printf "\nExecuting: ltcdump -f 29.97 "${fileToProcess}" 2>/dev/null | grep -v '#' | grep ':' | wc -w " hasLTC=0 hasLTC=`ltcdump -f 29.97 "${fileToProcess}" 2>/dev/null | grep -v '#' | grep ':' | wc -w` if [[ $hasLTC -eq 0 ]]; then printf "\nError processing file %s\n" "${fileToProcess}" printf "\ncannot parse timecode\n" return fi rm -f ltc.txt # find timecode in format 'HH:MM:SS.00' with frames as last 2 digits used for .00 time. # remove occasional '.' in the form 00:00:00.0 instead of 00:00:00:0 # remove negative values # disallow 0000000f 00:12:25:45 printf "\nExecuting: ltcdump -fps 29.97 "${fileToProcess}" 2>/dev/null | grep -v '#' | grep 00000000 | grep -v '\.' | grep -v '-' > ltc.txt" ltcdump -f 29.97 "${fileToProcess}" 2>/dev/null | grep -v '#' | grep 00000000 | grep -v '\.' | grep -v '-' > ltc.txt lastTimecodeInSegment=0 lastTimecodeInSegment=`tail -n 1 ltc.txt | awk '{ print $2 }' | sed 's/\(.*\):/\1./' ` # can sometimes receive unparsable dates if [[ ${#lastTimecodeInSegment} -ne 11 ]]; then lastTimecodeInSegment=0 printf "\nError processing file %s\n" "${fileToProcess}" printf "\ntimecode number of chars is not correct: " printf " %s" "${#lastTimecodeInSegment}" return fi # can receive invalid time such as: 00000680 00:13:61:10 | 22 1609 if ! date -d "$lastTimecodeInSegment" >/dev/null 2>&1; then printf "\nError processing file %s\n" "${fileToProcess}" printf "\ninvalid date format:" printf " %s" "$lastTimecodeInSegment" return fi epoch_secs_ltc_to_process=$(date +%s --date="$timecode_date ${lastTimecodeInSegment}") ltc_to_process_formatted="$timecode_date ${lastTimecodeInSegment}" printf "\nltc_to_process_formatted: $ltc_to_process_formatted" if [[ $epoch_secs_ltc_to_process -ne 0 && $epoch_secs_stop_time -ne 0 && $epoch_secs_ltc_to_process -ge $epoch_secs_stop_time ]]; then found_stop_time=1 printf "\nskipping file %s\n" "${fileToProcess}" printf "\ntimecode found:" printf " %s" "${ltc_to_process_formatted}" printf "\nis after or equal to stop time: " printf " %s" "${epoch_secs_stop_time_formatted}" return fi printf "\nfileToProcess seems valid:" "${fileToProcess}" isLTCDumpOutputValid=1 } # define timecode start time, just add date timecode_date=2018-05-02 # skip segments before this timecode epoch_secs_start_time=$(date +%s --date="$timecode_date 00:26:15") start_time_formatted="$timecode_date 00:26:15.00" # define timecode stop time epoch_secs_stop_time=$(date +%s --date="$timecode_date 00:29:00") stop_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_stop_time seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` stop_time_formatted="$timecode_date ${stop_timecode}" timeDivisionInSeconds=20 let epoch_secs_timecode_to_find=$((epoch_secs_start_time + timeDivisionInSeconds)) find_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_timecode_to_find seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` find_timecode_formatted="$timecode_date ${find_timecode}" let epoch_secs_next_timecode_to_find=$epoch_secs_timecode_to_find+$timeDivisionInSeconds next_timecode_to_find=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_next_timecode_to_find seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` next_timecode_to_find_formatted="$timecode_date ${next_timecode_to_find}" q81HasLTCStartTime=0 hasRejectBeforeTime=0 q81TimeCodeInProgress=0 q82TimeCodeInProgress=0 saveQ81CurrentTimeCodeInProgressAlreadyProcessed=0 rejectBeforeTime_formatted='' saveQ81CurrentTimeCodeInProgress_to_find_formatted='' while read i; do # skip auto generated version line if [[ $i =~ .*ffconcat.* ]]; then continue fi q81VideoFile=`echo $i | cut -d " " -f2-`; if [[ ! -f "${q81VideoFile}.wav" ]]; then # convert q81 video segments to wav, # so ltcdump can find the timecode ffmpeg -v error -nostdin -i $q81VideoFile -vn -acodec pcm_s16le -ar 48000 -ac 2 "${q81VideoFile}.wav" fi validate "${q81VideoFile}.wav" if [[ $found_stop_time -eq 1 ]]; then printf "\nq81 video is after stop time, executing break\n" printf "%s\n" "${q81VideoFile}" break fi if [[ $q81HasLTCStartTime -eq 0 && $isLTCDumpOutputValid -eq 0 ]]; then printf "\non q81HasLTCStartTime=0 and isLTCDumpOutputValid=0 .. q81 video is the default, occasional bad timecode values are ok" printf "\nadding video to file list to send to all.mov: " printf "file %s\n" "${q81VideoFile}" printf "file %s\n" "${q81VideoFile}" >> all.txt printf "%s\n" "${q81VideoFile}.wav" >> debug.txt continue fi if [[ $q81HasLTCStartTime -eq 0 && $isLTCDumpOutputValid -eq 1 ]]; then dateDiff=0 dateDiff=`date -d "$ltc_to_process_formatted $(date -d "$start_time_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "0" "$dateDiff" then q81HasLTCStartTime=1 printf "\non dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is later than or equal to start_time_formatted $start_time_formatted" printf "\nproceeding on file %s\n" "${q81VideoFile}" else printf "\non dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is before start_time_formatted $start_time_formatted" printf "\nskipping file %s\n" "${q81VideoFile}" continue fi fi if [[ $isLTCDumpOutputValid -eq 0 && $q81TimeCodeInProgress -eq 1 ]]; then printf "\nq81 video is the default, occasional bad timecode values are ok" printf "\nadding video to file list to send to all.mov: " printf "file %s\n" "${q81VideoFile}" printf "file %s\n" "${q81VideoFile}" >> all.txt printf "%s\n" "${q81VideoFile}.wav" >> debug.txt continue fi if [[ $isLTCDumpOutputValid -eq 0 && $q81TimeCodeInProgress -eq 0 ]]; then printf "\nskipping q81 video file , ltc is invalid and q81TimeCodeInProgress is false: " printf "file %s\n" "${q81VideoFile}" continue fi if [[ $hasRejectBeforeTime -eq 1 ]]; then dateDiff=0 dateDiff=`date -d "$ltc_to_process_formatted $(date -d "$rejectBeforeTime_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "$dateDiff" "0" then printf "\nexecuting continue on file "${q81VideoFile}" , dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is before or equal to rejectBeforeTime_formatted $rejectBeforeTime_formatted \n" continue else printf "\nproceeding on file "${q81VideoFile}" , dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is after $rejectBeforeTime_formatted \n" fi fi dateDiff=0 dateDiff=`date -d "$ltc_to_process_formatted $(date -d "$find_timecode_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "0" "$dateDiff" then printf "\non dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is later than or equal to find_timecode_formatted $find_timecode_formatted \n" else q81TimeCodeInProgress=1 printf "\non dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is less than find_timecode_formatted $find_timecode_formatted \n" printf "\nadding video to file list to send to all.mov: " printf "file %s\n" "${q81VideoFile}" printf "file %s\n" "${q81VideoFile}" >> all.txt printf "%s\n" "${q81VideoFile}.wav" >> debug.txt continue fi printf "\nfound find_timecode: " printf " %s" "${find_timecode}" printf "\nin file: " printf " %s" "${q81VideoFile}.wav" printf "\nwill now process the q82 files looking for timecode later than or equal to: " printf " %s" "${find_timecode_formatted}" printf "\nand timecode before or equal to: " printf " %s" "${next_timecode_to_find_formatted}" q81TimeCodeInProgress=0 saveQ81CurrentTimeCodeInProgress_to_find_formatted=$ltc_to_process_formatted saveQ81CurrentTimeCodeInProgressAlreadyProcessed=0 saveQ82CurrentTimeCodeInProgressAlreadyProcessed=0 while read f; do # skip auto generated version line if [[ $f =~ .*ffconcat.* ]]; then continue fi q82VideoFile=`echo $f | cut -d " " -f2-`; printf "%s\n" "${q82VideoFile}" >> q82_processed.txt # convert Q82 video segments to wav, # so that ltcdump can find the timecode if [[ ! -f "${q82VideoFile}.wav" ]]; then ffmpeg -v error -nostdin -i $q82VideoFile -vn -c:a pcm_s16le -ar 48000 -ac 2 "${q82VideoFile}.wav" fi validate "${q82VideoFile}.wav" if [[ $found_stop_time -eq 1 ]]; then printf "\nq82 video is after stop time, executing break\n" break fi if [[ $saveQ81CurrentTimeCodeInProgressAlreadyProcessed -eq 0 && $q82TimeCodeInProgress -eq 0 && $isLTCDumpOutputValid -eq 1 ]]; then saveQ81CurrentTimeCodeInProgressAlreadyProcessed=1 dateDiff=0 dateDiff=`date -d "$saveQ81CurrentTimeCodeInProgress_to_find_formatted $(date -d "$ltc_to_process_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "$dateDiff" "0" then printf "\nadding q81 file in q82 loop on dateDiff $dateDiff , $saveQ81CurrentTimeCodeInProgress_to_find_formatted is before or equal to $ltc_to_process_formatted" printf "file %s\n" "${q81VideoFile}" >> all.txt printf "%s\n" "${q81VideoFile}.wav" >> debug.txt else printf "\nin q82 loop no action taken on q81 video file "${q81VideoFile}" , dateDiff $dateDiff , $saveQ81CurrentTimeCodeInProgress_to_find_formatted greater than $ltc_to_process_formatted" fi fi if [[ $isLTCDumpOutputValid -eq 0 && $q82TimeCodeInProgress -eq 1 ]]; then printf "\nfound LTC error while processing files in range, just accept it" printf "\nadding video to q82 file processed list" printf "file %s\n" "${q82VideoFile}" >> all.txt printf "%s\n" "${q82VideoFile}.wav" >> debug.txt continue fi if [[ $isLTCDumpOutputValid -eq 0 ]]; then printf "\nFound ltc error outside time range while processing a q82 file out of range, reject all q82 videos when this happens" printf "\nskipping file: " printf " %s" "${q82VideoFile}" continue fi dateDiff=0 dateDiff=`date -d "$ltc_to_process_formatted $(date -d "$find_timecode_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "0" "$dateDiff" then printf "\nproceeding on file "${q82VideoFile}" , dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is later or equal to find_timecode_formatted $find_timecode_formatted" else printf "\nskipping file "${q82VideoFile}" on dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is before find_timecode_formatted $find_timecode_formatted" continue fi dateDiff=0 dateDiff=`date -d "$ltc_to_process_formatted $(date -d "$next_timecode_to_find_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "$dateDiff" "0" then printf "\nproceeding on dateDiff $dateDiff , $ltc_to_process_formatted is before or equal to $next_timecode_to_find_formatted" else printf "\nexecuting break on file "${q82VideoFile}" , dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is greater than next_timecode_to_find $next_timecode_to_find" break fi q82TimeCodeInProgress=1 printf "\nfound ltc in range" printf "\nadding video to q82 file processed list" printf "file %s\n" "${q82VideoFile}" >> all.txt printf "%s\n" "${q82VideoFile}.wav" >> debug.txt done < q82t.ffcat q82TimeCodeInProgress=0 if [[ ! -f "q82_processed.txt" ]]; then printf "\n\nCould not find file: q82_processed.txt , no timecode in q82 files found between $timecode_to_find and $next_timecode_to_find\n" exit else printf "\nq82 loop completed, removing files already processed from the queue" grep -F -v -f q82_processed.txt q82t.ffcat > q82t_files_not_processed.txt mv q82t_files_not_processed.txt q82t.ffcat fi # reset everything for next loop let epoch_secs_rejectBeforeTime=$epoch_secs_next_timecode_to_find+1 reject_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_rejectBeforeTime seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` rejectBeforeTime_formatted="$timecode_date ${reject_timecode}" hasRejectBeforeTime=1 increment=$(( $timeDivisionInSeconds * 2 )) let epoch_secs_timecode_to_find=$epoch_secs_timecode_to_find+$increment find_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_timecode_to_find seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` find_timecode_formatted="$timecode_date ${find_timecode}" let epoch_secs_next_timecode_to_find=$epoch_secs_next_timecode_to_find+$increment next_timecode_to_find=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_next_timecode_to_find seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` next_timecode_to_find_formatted="$timecode_date ${next_timecode_to_find}" printf "\nloop completed, timecode_to_find has been reset: " printf " %s" "${find_timecode_formatted}" printf "\nnext_timecode_to_find_formatted has been reset: " printf " %s" "${next_timecode_to_find_formatted}" printf "\nrejectBeforeTime_formatted has been reset: " printf " %s" "${rejectBeforeTime_formatted}" done < q81t.ffcat printf "\nExecuting: ffmpeg -y -v error -nostdin -f concat -fflags +genpts -safe 0 -i all.txt -async 1 -vsync 1 -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af "aresample=async=1:min_hard_comp=0.100000:first_pts=0" -crf 0 all.mov" ffmpeg -y -v error -nostdin -f concat -fflags +genpts -safe 0 -i all.txt -async 1 -vsync 1 -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af "aresample=async=1:min_hard_comp=0.100000:first_pts=0" -crf 0 all.mov # debug ffmpeg -y -v error -nostdin -i all.mov -vn -c:a pcm_s16le -ar 48000 -ac 2 all.wav while read wavFile; do ltc_out=`ltcdump $wavFile 2>/dev/null ` printf "### output from file %s\n\n" "${wavFile}" >> debug_out.txt printf "%s\n\n" "${ltc_out}" >> debug_out.txt done < debug.txt | sort -V exit _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".