Send Motion-user mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/motion-user
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Motion-user digest..."
Today's Topics:
1. Re: Corrupt/incomplete JPG images (tosiara)
2. Re: Corrupt/incomplete JPG images (Rich Bloch)
3. Re: Corrupt/incomplete JPG images (tosiara)
----------------------------------------------------------------------
Message: 1
Date: Fri, 18 Mar 2016 11:59:29 +0200
From: tosiara <[email protected]>
Subject: Re: [Motion-user] Corrupt/incomplete JPG images
To: Motion discussion list <[email protected]>
Message-ID:
<CACHTdwS6bT=EAjkfv8bCELUA-KO=EA1Q4VzVLDGwp2m21T9=c...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Can you increase motion logging and try find corresponding time frame -
would there be any other warnings?
"Potential split boundary" usually does not cause "gray" artifacts. But
warning from libjpeg may come
On Sun, Mar 13, 2016 at 11:19 PM, Rich Bloch <[email protected]> wrote:
> Hello all,
>
> I'm a long-time user of Motion, and very happy with the program.
>
> I've consistently run into an issue that I'd like to get a better
> understanding of so that I might contribute a fix.
>
> The use case is that every so often, some of my more distant wifi IPCams
> (I use HooToo HT-IP206 cams) generate what appears to be an incomplete JPG
> image. Not sure if it's technically a corrupt image, but it certainly is
> incomplete. My understanding of the JPG format is that an incomplete image
> datablock is typically replaced by grey pixels to properly display the
> image. That appears to be what I'm seeing.
>
> I've posted an example here: http://imgur.com/uj24xqS
>
> In line with these incomplete images is what may be a clue in
> motion.log: netcam_read_html_jpeg: Potential split boundary. I can't
> confirm that these are 100% correlated with the bad images, but it appears
> to be the case.
>
> So my questions are:
>
> -
> *Has anyone dealt with these partial JPG images before? *
> - *If so, is there any known resolution?*
>
> Tracing through the Motion codebase (i.e., netcam.c), it appears that the
> logic has been known to be a bit of a hack (as noted in the comments). If I
> better understood the real issue here, I'd be interested in implementing a
> fix and making a pull request to test.
>
> I did some searching through the motion-user archives, and didn't turn up
> much.
>
> Thanks much.
>
>
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
> _______________________________________________
> Motion-user mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/motion-user
> http://www.lavrsen.dk/twiki/bin/view/Motion/WebHome
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
------------------------------
Message: 2
Date: Sun, 20 Mar 2016 03:32:51 +0000
From: Rich Bloch <[email protected]>
Subject: Re: [Motion-user] Corrupt/incomplete JPG images
To: Motion discussion list <[email protected]>
Message-ID:
<CAHdt+sqBqfucXUAnGQzwb+txmMA1rAq=y8HoVt8TUWJPk=n...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
@tosiara,
Yes, I think you're correct that the issue may be related to FFMPEG. At
minimum, I can say that the "Corrupt JPEG data" error message is not from
Motion.
Here's a segment of motion.log that captures the corrupt JPG encoding.
These corruption messages correlate well with bad JPG images generated in
Motion (gray boxes in an incomplete image: the example I had originally
posted):
[1] [DBG] [VID] [Mar 18 17:19:50] v4l2_next: 2) vid_source->pframe 1
> [1] [DBG] [VID] [Mar 18 17:19:50] v4l2_next: 3) vid_source->pframe 2
> vid_source->buf.index 2
> [1] [DBG] [VID] [Mar 18 17:19:50] v4l2_next: vid_source->buf.bytesused
> 614400
> [1] [DBG] [VID] [Mar 18 17:19:50] v4l2_next: the_buffer index 2 Address
> (d5fb4000)
> [1] [DBG] [VID] [Mar 18 17:19:50] v4l2_next: 1) vid_source->pframe 2
> [4] [DBG] [NET] [Mar 18 17:19:50] netcam_read_html_jpeg: Calculated frame
> time 2891344.500000
> [4] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_type: Content-type
> image/jpeg
> [4] [INF] [NET] [Mar 18 17:19:50] netcam_init_jpeg: ***new pic delay
> successful***
> [4] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
> Content-Length -1
> [4] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
> Content-Length 31585
> [4] [INF] [NET] [Mar 18 17:19:50] netcam_read_next_header: Found image
> header record
> [4] [INF] [NET] [Mar 18 17:19:50] netcam_init_jpeg: jpeg_error 0
> [4] [INF] [NET] [Mar 18 17:19:50] netcam_image_conv: jpeg_error 0
> [5] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_type: Content-type
> image/jpeg
> [5] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
> Content-Length -1
> [5] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
> Content-Length 33952
> [5] [INF] [NET] [Mar 18 17:19:50] netcam_read_next_header: Found image
> header record
> [4] [INF] [NET] [Mar 18 17:19:50] netcam_proc_jpeg: processing jpeg image
> - content length 31372
> [3] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_type: Content-type
> image/jpeg
> [3] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
> Content-Length -1
> [3] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
> Content-Length 28648
> [3] [INF] [NET] [Mar 18 17:19:50] netcam_read_next_header: Found image
> header record
> [2] [DBG] [NET] [Mar 18 17:19:50] netcam_read_html_jpeg: Calculated frame
> time 154269.718750
> [2] [INF] [NET] [Mar 18 17:19:50] netcam_init_jpeg: ***new pic delay
> successful***
> [2] [INF] [NET] [Mar 18 17:19:50] netcam_init_jpeg: jpeg_error 0
>
> *[2] [DBG] [NET] [Mar 18 17:19:50] netcam_output_message: Corrupt JPEG
> data: 1 extraneous bytes before marker 0xd9*[2] [INF] [NET] [Mar 18
> 17:19:50] netcam_image_conv: jpeg_error 0
> [2] [INF] [NET] [Mar 18 17:19:50] netcam_proc_jpeg: processing jpeg image
> - content length 46140
I've looked through Motion sources--jpeg_utils.c and video_common.c--and
from the short time walking through the code, decode_jpeg_raw() is getting
called and should return 1 if image corruption is detected. However, it
does not appear to be doing this. Rather, the image corruption is happening
in the FFMPEG lib internally, and that state is does not appear to be
getting passed back into Motion.
I'm currently running Motion 3.2.12 on Ubuntu 15.04. FFMPEG on this box is
v2.5.10, while the current release of FFMPEG just revved to v3.0.0. So, I
may need to rev this lib to see if the issue is still reproducible in the
current (or a prior) release of FFMPEG.
Just to put this into some context, the only time I seem to run into these
corrupt images is with my IP cams with poor (read: variable) WIFI
reception.
This suggests that there is an edge case (likely in FFMPEG) that may not be
fully exercised. That is, if there's a delay during the encoding of a data
stream because of intermittent wireless reception, the encoder may time-out
and fail rather than wait a longer period of time, or retry encoding. This
is all pure speculation at this point, of course.
Going forward, to verify these assumptions, I'll likely need to sit on the
Motion sources, and walk through a debug session. Revving the FFMPEG lib
seems a quicker path to resolution at the moment :).
Thoughts?
BTW, it appears that this issue has a very long history in Motion (causing
me to speculate that the issue perhaps has less to do with FFMPEG unless
this specific issue has been latent in the FFMPEG lib for ~10 years).
Unfortunately, in my backtracking through bug reports, there appears to be
no documented resolution :/.
Thanks
Rich
-------------- next part --------------
An HTML attachment was scrubbed...
------------------------------
Message: 3
Date: Sun, 20 Mar 2016 16:14:58 +0200
From: tosiara <[email protected]>
Subject: Re: [Motion-user] Corrupt/incomplete JPG images
To: Motion discussion list <[email protected]>
Message-ID:
<CACHTdwTS9S7zO4xup_Td=Ggoyo6YKnz98yB66+gEQf-=ebe...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
FFmpeg is not involved in any way here. Network mjpeg stream is downloaded
using socket read and decoded using libjpeg. If it complains about bad jpeg
- it is bad then.
If you want you can hook wireshark on the same box where motion runs and
extract raw frames. If you try to feed raw frames to imagemagic it will
also give you the same warning
On Mar 20, 2016 05:34, "Rich Bloch" <[email protected]> wrote:
> @tosiara,
>
> Yes, I think you're correct that the issue may be related to FFMPEG. At
> minimum, I can say that the "Corrupt JPEG data" error message is not from
> Motion.
>
> Here's a segment of motion.log that captures the corrupt JPG encoding.
> These corruption messages correlate well with bad JPG images generated in
> Motion (gray boxes in an incomplete image: the example I had originally
> posted):
>
> [1] [DBG] [VID] [Mar 18 17:19:50] v4l2_next: 2) vid_source->pframe 1
>> [1] [DBG] [VID] [Mar 18 17:19:50] v4l2_next: 3) vid_source->pframe 2
>> vid_source->buf.index 2
>> [1] [DBG] [VID] [Mar 18 17:19:50] v4l2_next: vid_source->buf.bytesused
>> 614400
>> [1] [DBG] [VID] [Mar 18 17:19:50] v4l2_next: the_buffer index 2 Address
>> (d5fb4000)
>> [1] [DBG] [VID] [Mar 18 17:19:50] v4l2_next: 1) vid_source->pframe 2
>> [4] [DBG] [NET] [Mar 18 17:19:50] netcam_read_html_jpeg: Calculated frame
>> time 2891344.500000
>> [4] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_type: Content-type
>> image/jpeg
>> [4] [INF] [NET] [Mar 18 17:19:50] netcam_init_jpeg: ***new pic delay
>> successful***
>> [4] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
>> Content-Length -1
>> [4] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
>> Content-Length 31585
>> [4] [INF] [NET] [Mar 18 17:19:50] netcam_read_next_header: Found image
>> header record
>> [4] [INF] [NET] [Mar 18 17:19:50] netcam_init_jpeg: jpeg_error 0
>> [4] [INF] [NET] [Mar 18 17:19:50] netcam_image_conv: jpeg_error 0
>> [5] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_type: Content-type
>> image/jpeg
>> [5] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
>> Content-Length -1
>> [5] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
>> Content-Length 33952
>> [5] [INF] [NET] [Mar 18 17:19:50] netcam_read_next_header: Found image
>> header record
>> [4] [INF] [NET] [Mar 18 17:19:50] netcam_proc_jpeg: processing jpeg image
>> - content length 31372
>> [3] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_type: Content-type
>> image/jpeg
>> [3] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
>> Content-Length -1
>> [3] [INF] [NET] [Mar 18 17:19:50] netcam_check_content_length:
>> Content-Length 28648
>> [3] [INF] [NET] [Mar 18 17:19:50] netcam_read_next_header: Found image
>> header record
>> [2] [DBG] [NET] [Mar 18 17:19:50] netcam_read_html_jpeg: Calculated frame
>> time 154269.718750
>> [2] [INF] [NET] [Mar 18 17:19:50] netcam_init_jpeg: ***new pic delay
>> successful***
>> [2] [INF] [NET] [Mar 18 17:19:50] netcam_init_jpeg: jpeg_error 0
>>
>> *[2] [DBG] [NET] [Mar 18 17:19:50] netcam_output_message: Corrupt JPEG
>> data: 1 extraneous bytes before marker 0xd9*[2] [INF] [NET] [Mar 18
>> 17:19:50] netcam_image_conv: jpeg_error 0
>> [2] [INF] [NET] [Mar 18 17:19:50] netcam_proc_jpeg: processing jpeg image
>> - content length 46140
>
>
> I've looked through Motion sources--jpeg_utils.c and video_common.c--and
> from the short time walking through the code, decode_jpeg_raw() is getting
> called and should return 1 if image corruption is detected. However, it
> does not appear to be doing this. Rather, the image corruption is happening
> in the FFMPEG lib internally, and that state is does not appear to be
> getting passed back into Motion.
>
> I'm currently running Motion 3.2.12 on Ubuntu 15.04. FFMPEG on this box is
> v2.5.10, while the current release of FFMPEG just revved to v3.0.0. So, I
> may need to rev this lib to see if the issue is still reproducible in the
> current (or a prior) release of FFMPEG.
>
> Just to put this into some context, the only time I seem to run into these
> corrupt images is with my IP cams with poor (read: variable) WIFI
> reception.
>
> This suggests that there is an edge case (likely in FFMPEG) that may not
> be fully exercised. That is, if there's a delay during the encoding of a
> data stream because of intermittent wireless reception, the encoder may
> time-out and fail rather than wait a longer period of time, or retry
> encoding. This is all pure speculation at this point, of course.
>
> Going forward, to verify these assumptions, I'll likely need to sit on the
> Motion sources, and walk through a debug session. Revving the FFMPEG lib
> seems a quicker path to resolution at the moment :).
>
> Thoughts?
>
> BTW, it appears that this issue has a very long history in Motion (causing
> me to speculate that the issue perhaps has less to do with FFMPEG unless
> this specific issue has been latent in the FFMPEG lib for ~10 years).
> Unfortunately, in my backtracking through bug reports, there appears to be
> no documented resolution :/.
>
> Thanks
>
> Rich
>
>
>
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
> _______________________________________________
> Motion-user mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/motion-user
> http://www.lavrsen.dk/twiki/bin/view/Motion/WebHome
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
------------------------------
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
------------------------------
_______________________________________________
Motion-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/motion-user
End of Motion-user Digest, Vol 118, Issue 8
*******************************************
