Send netdisco-users mailing list submissions to
        netdisco-users@lists.sourceforge.net

To subscribe or unsubscribe via the World Wide Web, visit
        https://lists.sourceforge.net/lists/listinfo/netdisco-users
or, via email, send a message with subject or body 'help' to
        netdisco-users-requ...@lists.sourceforge.net

You can reach the person managing the list at
        netdisco-users-ow...@lists.sourceforge.net

When replying, please edit your Subject line so it is more specific
than "Re: Contents of netdisco-users digest..."
Today's Topics:

   1. Netdisco auto discovery tasks suddenly stopped working (alcatron)
   2. Re: Netdisco auto discovery tasks suddenly stopped working
      (Christian Ramseyer)
--- Begin Message ---
Hi all, just wanting to ask your thoughts on what could be causing netdisco to 
suddenly stop performing auto discovery tasks.

Seems only arpnip is working via scheduled tasks, but discovery/macsuck has 
halted to auto perform. If I go manually to the device on web interface and 
trigger the auto discovery/arpnip/macsuck it works fine on the device.

Nothing has changed on system, running for a few months now, and suddenly the 
auto discovery is broken partly.

If I go to the backend log I see error like this below. The server is running 
and operational as I can still perform the manual to get discovery etc

The server is not out of memory as it has like 16GB and still plenty unused not 
what the messages are indicating..

Thanks for any assistance 😊

DBIx::Class::Schema::Versioned::_on_connect(): Your DB is currently 
unversioned. Please call upgrade on your schema to sync the DB. at 
/home/netdisco/perl5/lib/perl5/DBICx/Sugar.pm line 121
DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: DBI 
connect('dbname=netdisco','netdisco',...) failed: could not connect to server: 
No such file or directory
            Is the server running locally and accepting
            connections on Unix domain socket 
"/var/run/postgresql/.s.PGSQL.5432"? at 
/home/netdisco/perl5/lib/perl5/DBIx/Class/Storage/DBI.pm line 1639. at 
/home/netdisco/perl5/lib/perl5/App/Netdisco/JobQueue/PostgreSQL.pm line 50

[25756] error bless( {'msg' => 'DBIx::Class::Storage::DBI::_dbh_execute(): DBI 
Exception: DBD::Pg::st execute failed: ERROR:  out of memory
DETAIL:  Failed on request of size 16 in memory context "MessageContext". [for 
Statement "SELECT me.job, me.entered, me.started, me.finished, me.device, 
me.port, me.action, me.subaction, me.status, me.username, me.userip, me.log, 
me.debug, me.device_key FROM admin me WHERE ( me.job = ? ) FOR UPDATE" with 
ParamValues: 1=\'186421742\'] at 
/home/netdisco/perl5/lib/perl5/App/Netdisco/JobQueue/PostgreSQL.pm line 267
'}, 'DBIx::Class::Exception' )
[25781] 2022-01-11 01:33:53 error bless( {'msg' => 
'DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute 
failed: ERROR:  out of memory
DETAIL:  Failed on request of size 16 in memory context "MessageContext". [for 
Statement "SELECT me.job, me.entered, me.started, me.finished, me.device, 
me.port, me.action, me.subaction, me.status, me.username, me.userip, me.log, 
me.debug, me.device_key FROM admin me WHERE ( me.job = ? ) FOR UPDATE" with 
ParamValues: 1=\'186420514\'] at 
/home/netdisco/perl5/lib/perl5/App/Netdisco/JobQueue/PostgreSQL.pm line 267

--- End Message ---
--- Begin Message ---
Hi

>  could not connect to
> server: No such file or directory/

This would be very concerning, meaning that Postgres is not running at all. But since you seem to have the web frontend running that is probably not the case currently, so I wouldn't worry too much. Might be an old log entry.


> Failed on request of size 16 in memory context
> "MessageContext".

That on the other hand might be the issue. Postgres uses all kinds of memory parameters, if one of them is too small the total GB of RAM sticks in the server don't matter much.

I had various issues with huge and clogged up discovery queues over the years, as a first measure I'd try to:

stop netdisco-backend
restart Postgres, connect to the database with "netdisco-do psql" and in there run a "delete from admin;".
for good measure, also run "reindex table admin;"
restart netdisco-backend

This sounds dangerous but admin is in fact just the queue of actions to be done, so no important data will be lost.

Also a "select count(*) from admin" first might be interesting, to see how many rows are in there. If it's an absurdly high number (millions) you can run e.g. "create table admin_backup as select * from admin;" for analysis later.

If you're still getting the memory errors afterwards and it still doesn't work, I'd try to configure the memory parameters with this assistant, using the "online transaction processing" db type. https://pgtune.leopard.in.ua/#/about


Cheers
Christian



On 17.01.22 22:03, alcatron wrote:
Hi all, just wanting to ask your thoughts on what could be causing netdisco to suddenly stop performing auto discovery tasks.

Seems only arpnip is working via scheduled tasks, but discovery/macsuck has halted to auto perform. If I go manually to the device on web interface and trigger the auto discovery/arpnip/macsuck it works fine on the device.

Nothing has changed on system, running for a few months now, and suddenly the auto discovery is broken partly.

If I go to the backend log I see error like this below. The server is running and operational as I can still perform the manual to get discovery etc

The server is not out of memory as it has like 16GB and still plenty unused not what the messages are indicating..

Thanks for any assistance 😊

/DBIx::Class::Schema::Versioned::_on_connect(): Your DB is currently unversioned. Please call upgrade on your schema to sync the DB. at /home/netdisco/perl5/lib/perl5/DBICx/Sugar.pm line 121/

/DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: DBI connect('dbname=netdisco','netdisco',...) failed: could not connect to server: No such file or directory/

/            Is the server running locally and accepting/

/            connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? at /home/netdisco/perl5/lib/perl5/DBIx/Class/Storage/DBI.pm line 1639. at /home/netdisco/perl5/lib/perl5/App/Netdisco/JobQueue/PostgreSQL.pm line 50/

//

/[25756] error bless( {'msg' => 'DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  out of memory/

/DETAIL:  Failed on request of size 16 in memory context "MessageContext". [for Statement "SELECT me.job, me.entered, me.started, me.finished, me.device, me.port, me.action, me.subaction, me.status, me.username, me.userip, me.log, me.debug, me.device_key FROM admin me WHERE ( me.job = ? ) FOR UPDATE" with ParamValues: 1=\'186421742\'] at /home/netdisco/perl5/lib/perl5/App/Netdisco/JobQueue/PostgreSQL.pm line 267/

/'}, 'DBIx::Class::Exception' )/

/[25781] 2022-01-11 01:33:53 error bless( {'msg' => 'DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  out of memory/

/DETAIL:  Failed on request of size 16 in memory context "MessageContext". [for Statement "SELECT me.job, me.entered, me.started, me.finished, me.device, me.port, me.action, me.subaction, me.status, me.username, me.userip, me.log, me.debug, me.device_key FROM admin me WHERE ( me.job = ? ) FOR UPDATE" with ParamValues: 1=\'186420514\'] at /home/netdisco/perl5/lib/perl5/App/Netdisco/JobQueue/PostgreSQL.pm line 267/



_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/

--
Christian Ramseyer, netnea ag
Network Management. Security. OpenSource.
https://www.netnea.com
Phone: +41 79 644 77 64



--- End Message ---
_______________________________________________
Netdisco mailing list - Digest Mode
netdisco-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/netdisco-users

Reply via email to