Send Motion-user mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/motion-user
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Motion-user digest..."
Today's Topics:
1. Re: Configuration help (Steven Haigh)
----------------------------------------------------------------------
Message: 1
Date: Thu, 07 Apr 2022 01:35:01 +1000
From: Steven Haigh <[email protected]>
To: Motion discussion list <[email protected]>
Subject: Re: [Motion-user] Configuration help
Message-ID: <[email protected]>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
You might be right.... The code to write the picture is here:
<https://github.com/Motion-Project/motion/blob/master/src/picture.c#L824>
I actually wonder if line 827 on that file shouldn't unlink a file if
it already exists.... then the open / write / close. It seems like that
would be the sane way to overwrite files :|
I was trying to find the code that does the snapshots - but I haven't
been able to find that as yet :\
--
Steven Haigh
? [email protected] <mailto:[email protected]>
? https://www.crc.id.au <https://www.crc.id.au/>
On Wed, Apr 6 2022 at 10:11:13 -0500, Roger Heflin
<[email protected]> wrote:
> Well, the posix comment might apply if a new file/inode was being
> created. You seem to be assuming that a new file must be getting
> created, it does not have to be creating a new file, and it appears
> to be re-using the same inode.
>
> I have not looked at the code to see how the write is being done, but
> the inode is not changing, so am guessing it is some sort of append
> type open and a seek to the start of the file since that would act
> this way. From a simple copy test in a loop from the file being
> updated at 15fps, I get 50% of the files with all data and 50% of the
> files with no data (0-size).
>
> Now if I do the copy with a dd using a 4k buffer (standard io if you
> don't make calls to adjust the settings) I get files of various sizes
> some of which are non-zero size and missing parts of the image.
>
> However they are doing the open may not be wise, or there may be a
> reason for them doing the open that way, but with the current way the
> open is being done it won't appear to work.
>
>
>
> On Wed, Apr 6, 2022 at 8:51 AM Steven Haigh <[email protected]
> <mailto:[email protected]>> wrote:
>> Just for the record, POSIX compliant operating systems will change
>> the inode that a new file lives in - meaning that if you open a file
>> while the file is being overwritten, the old inode already opened
>> will be invalidated for the new file, but still persist until all
>> file handles using the old inode are closed. At this point, the old
>> inode is discarded as free.
>>
>> This means as long as the file open occurs before the write, you'll
>> get the old contents of the file - regardless of when you read the
>> rest of that file.
>>
>> As such, the open + write + close should happen on a different inode
>> when overwriting a file - especially if another process contains an
>> active file handle on the old inode.
>>
>> That being said, the corrupted snapshot I provided was directly from
>> the filesystem - not from the ingest method at all - so its being
>> written to disk like that.
>>
>> I have no idea why :(
>> --
>> Steven Haigh
>>
>> ? [email protected] <mailto:[email protected]>
>> ? https://www.crc.id.au <https://www.crc.id.au/>
>>
>> On Tue, Apr 5 2022 at 09:20:55 -0500, Roger Heflin
>> <[email protected] <mailto:[email protected]>> wrote:
>>> ok. There is *NO* way to reliably read a file in on a file change
>>> that is getting re-written often. A file open+write+close is not
>>> an Atomic operation, it is at least 3 separate operations (it it is
>>> only 3 if the code writing the is carefully crafted to write the
>>> data as a single write), and the change will trigger on the open,
>>> and you will open the file before the writing is done (or possibly
>>> even started).
>>>
>>> You probably then need to have a separate snapshot stream using the
>>> high-res and doing a frame a second, and you likely need to add
>>> seconds to the file (so 60 files being written round-robin) and
>>> have the deep stack reading the file from a second ago, or using
>>> the file change trigger + a usleep to delay it enough to allow the
>>> write/close to finish.
>>>
>>> I was using inotify to spawn off a script on a file change and then
>>> copy that file to someplace else. The inotify was pretty often
>>> spawning the script and starting the copy of the file data before
>>> the file was finished writing and the files I was reading were not
>>> getting re-written, so I don't have the potential issue of reading
>>> the file a bit slow and getting caught on the next work.
>>>
>>>
>>>
>>> On Tue, Apr 5, 2022 at 9:02 AM Steven Haigh <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>> The process that I was using to pull the image into deepstack will
>>>> do so on a file change - so after the data has been written to a
>>>> file.
>>>>
>>>> I've added the image as an attachment - because it seems to be the
>>>> same file written again and again...
>>>>
>>>> The actual picture saved by both picture_output and
>>>> picture_output_motion are fine.
>>>>
>>>> I do note however that picture_output is written at 15 fps - which
>>>> is kinda overkill for using the file changed trigger - as even
>>>> though deepstack is GPU accelerated, it can't do 15fps :D
>>>>
>>>> I also validated the RTMP endpoint in VLC - and that plays
>>>> properly - so the problem isn't there...
>>>>
>>>> I also get the correct image via:
>>>> - <http://172.31.1.89:8001/1/current> (single image)
>>>> - <http://172.31.1.89:8001/1/stream> (MJPEG stream)
>>>>
>>>> that being said, it only seems like the snapshot is of the low
>>>> resolution stream as well - not from netcam_high_url.
>>>>
>>>> As such, it seems like I have to use the 'picture_output' option -
>>>> but then I'm still stuck with the framerate problems...
>>>> --
>>>> Steven Haigh
>>>>
>>>> ? [email protected] <mailto:[email protected]>
>>>> ? https://www.crc.id.au <https://www.crc.id.au/>
>>>>
>>>> On Tue, Apr 5 2022 at 08:41:06 -0500, Roger Heflin
>>>> <[email protected] <mailto:[email protected]>> wrote:
>>>>> Do one of the cameras for the 1fps capture (but the snapshot
>>>>> should work anyway).
>>>>>
>>>>> My snapshot filename definition looks like this:
>>>>> snapshot_filename ./snapshots/%Y%m%d/%Y-%m-%d-%H%M%S-%v-%q
>>>>> If you are using the same file name and re-writing it every
>>>>> second and reading it into something else the is a good chance
>>>>> that you will read the file at the wrong time a significant
>>>>> amount of the time. You may want to have the filename just write
>>>>> out minutes and seconds in the name such that the files get
>>>>> re-written each hour, or even just used seconds and let the file
>>>>> get re-written every 60 seconds and have the other tool reading a
>>>>> second behind so that it gets a complete file. I had some code
>>>>> that used inotify and copied the file on the create and often
>>>>> that separate process would copy the file before the file was
>>>>> completely written. I had to put some delay in for it to work.
>>>>>
>>>>> or, do a 2nd camera with the low-res for detection + high res for
>>>>> capture.
>>>>>
>>>>> On the corruption, have you viewed the live camera feed from
>>>>> motion to see if that is also corrupted from time to time?
>>>>>
>>>>> There are some camera models/vendors that don't handle packet
>>>>> loss at all, and lose frames.
>>>>>
>>>>> On Tue, Apr 5, 2022 at 8:27 AM Steven Haigh via Motion-user
>>>>> <[email protected]
>>>>> <mailto:[email protected]>> wrote:
>>>>>> I was trying not to do it that way - as performing motion detect
>>>>>> on the high resolution stream takes up 40-50% of a CPU core on
>>>>>> my system...
>>>>>>
>>>>>> Running motion detect on the low resolution stream seems to be
>>>>>> fine - but has the problem that I can't trigger captures from
>>>>>> the high resolution stream based on motion from the low
>>>>>> resolution one.
>>>>>>
>>>>>> I tried with the 'snapshot' option, but the JPG it seems to
>>>>>> produce is garbled junk :\
>>>>>>
>>>>>> Right now, for the picture, I'm using:
>>>>>>
>>>>>> picture_output on
>>>>>> picture_output_motion off
>>>>>> picture_quality 50
>>>>>> picture_filename /tmp/cameras/front_door
>>>>>>
>>>>>> ## This also kills the /1/stream down to 1fps - which is not
>>>>>> ideal.
>>>>>> #minimum_frame_time 1
>>>>>>
>>>>>> ## The below outputs a corrupted JPG as the snapshot...
>>>>>> #snapshot_interval 1
>>>>>> #snapshot_filename /tmp/cameras/front_door_snapshot
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Steven Haigh
>>>>>>
>>>>>> ? [email protected] <mailto:[email protected]>
>>>>>> ? https://www.crc.id.au <https://www.crc.id.au/>
>>>>>>
>>>>>> On Tue, Apr 5 2022 at 08:19:37 -0500, Roger Heflin
>>>>>> <[email protected] <mailto:[email protected]>> wrote:
>>>>>>> You should be able to setup 2 cameras in motion separately
>>>>>>> handling the 2 streams.
>>>>>>>
>>>>>>> I have a low-res URL going to one motion thread/camera that
>>>>>>> captures 2fps, and I have a separate thread using a different
>>>>>>> URL on the same camera (high resolution) that is used for
>>>>>>> motion capture.
>>>>>>>
>>>>>>> So long as the camera allows you should be able to use 2 of the
>>>>>>> same stream URL in different threads for the same camera.
>>>>>>>
>>>>>>> On Mon, Apr 4, 2022 at 11:52 PM Steven Haigh via Motion-user
>>>>>>> <[email protected]
>>>>>>> <mailto:[email protected]>> wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I'm having a hell of a time trying to do some image processing
>>>>>>>> with motion + nodered + home assistant + deepstack.
>>>>>>>>
>>>>>>>> I have motion configured as follows:
>>>>>>>> netcam_url
>>>>>>>> rtmp://172.31.1.245/bcs/channel0_ext.bcs?channel=0&stream=2&user=x&password=x
>>>>>>>>
>>>>>>>> <http://172.31.1.245/bcs/channel0_ext.bcs?channel=0&stream=2&user=x&password=x>
>>>>>>>> netcam_params framerate=30,capture_rate=30
>>>>>>>> width=896
>>>>>>>> height=672
>>>>>>>>
>>>>>>>> netcam_high_url
>>>>>>>> rtmp://172.31.1.245/bcs/channel0_main.bcs?channel=0&stream=0&user=x&password=x
>>>>>>>>
>>>>>>>> <http://172.31.1.245/bcs/channel0_main.bcs?channel=0&stream=0&user=x&password=x>
>>>>>>>> netcam_high_params framerate=15,capture_rate=15
>>>>>>>>
>>>>>>>> The camera outputs match the settings above - but the
>>>>>>>> resolution on netcam_high_url is 2560x1920.
>>>>>>>>
>>>>>>>> I'm trying to output the netcam_url via the motion API url
>>>>>>>> /1/stream and /1/current to show in the Home Assistant
>>>>>>>> dashboard and feed as a MJPEG camera.
>>>>>>>>
>>>>>>>> I'm also trying to output 1fps as a JPG to a file as
>>>>>>>> /tmp/snapshot.jpg so that I can ingest that into deepstack via
>>>>>>>> nodered and do object recognition.
>>>>>>>>
>>>>>>>> It seems that no matter what I do, I either destroy motion by
>>>>>>>> setting minimum_frame_time to 1, or get the 1fps output with
>>>>>>>> that option - but destroy the stream output.
>>>>>>>>
>>>>>>>> Is there a way I can ratelimit just the saving of pictures to
>>>>>>>> 1 per second from the netcam_high_url input?
>>>>>>>>
>>>>>>>> I've done every possible combination of configuration options,
>>>>>>>> but just can't seem to get this happening.
>>>>>>>> --
>>>>>>>> Steven Haigh
>>>>>>>>
>>>>>>>> ? [email protected] <mailto:[email protected]>
>>>>>>>> ? https://www.crc.id.au <https://www.crc.id.au/>
>>>>>>>> _______________________________________________
>>>>>>>> Motion-user mailing list
>>>>>>>> [email protected]
>>>>>>>> <mailto:[email protected]>
>>>>>>>> <https://lists.sourceforge.net/lists/listinfo/motion-user>
>>>>>>>> <https://motion-project.github.io/>
>>>>>>>>
>>>>>>>> Unsubscribe:
>>>>>>>> <https://lists.sourceforge.net/lists/options/motion-user>
>>>>>> _______________________________________________
>>>>>> Motion-user mailing list
>>>>>> [email protected]
>>>>>> <mailto:[email protected]>
>>>>>> <https://lists.sourceforge.net/lists/listinfo/motion-user>
>>>>>> <https://motion-project.github.io/>
>>>>>>
>>>>>> Unsubscribe:
>>>>>> <https://lists.sourceforge.net/lists/options/motion-user>
-------------- next part --------------
An HTML attachment was scrubbed...
------------------------------
------------------------------
Subject: Digest Footer
_______________________________________________
Motion-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/motion-user
------------------------------
End of Motion-user Digest, Vol 190, Issue 10
********************************************
