Send netdisco-users mailing list submissions to
netdisco-users@lists.sourceforge.net
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/netdisco-users
or, via email, send a message with subject or body 'help' to
netdisco-users-requ...@lists.sourceforge.net
You can reach the person managing the list at
netdisco-users-ow...@lists.sourceforge.net
When replying, please edit your Subject line so it is more specific
than "Re: Contents of netdisco-users digest..."
Today's Topics:
1. Re: SSH Collector Testing? (Jim Araujo)
2. Re: SSH Collector Testing? (Christian Ramseyer)
--- Begin Message ---
Sure thing Christian.
Both the below Device Types(BigIP * PaloAlto) seem to have similar
issues with Addresses reporting in the DB, but not the WebGUI.
This pate-code is the -DISQ(first link) and DB commands(second link)
from a BigIP F5 i2800
https://pastecode.io/s/avkzee1x
https://pastecode.io/s/y3ekxc8r
This Paste-code is the -DISQ(first link) and DB commands(second link)
from a PaloAlto PA-5220
https://pastecode.io/s/vagnng2z
https://pastecode.io/s/wvsppfdh
On 11/15/2021 7:17 AM, Christian Ramseyer wrote:
Hi Jim
Ok it wasn't quite clear to me earlier in this thread where you're not
finding the expected IPs. For the addresses tab, the data is collected
in the discover step, and written into the table device_ip. This is
most likely empty for your device, you can check with:
db=> select * from device_ip where ip = 'x.x.x.x';
IIRC this uses IP-MIB, but you can see exactly what OIDs are queried,
and what is returned by running
netdisco-do discover -DISQ -d x.x.x.x
Unfortunately your pastecode snipped expired so I can't tell if there
were any errors there, but this should point the search in the right
direction.
Cheers
Christian
On 12.11.21 22:59, Jim Araujo wrote:
Alrighty. I just erased the db from postgres and add a single device
having the issue. Still occuring only with one device.
https://pastecode.io/s/tuuc90gy
and the address page
On 11/12/2021 4:34 PM, Michael Butash wrote:
Why I mentioned comparing the databases, though not sure how they
link to each other, by a device column or some index, or combination
thereof. If the ssh collected mac table is using a wrong device to
link to snmp data, it'll just treat them as two broken sets of
devices vs. merging and associating them. Could just be something
dumb like ip vs. name vs. whatever you used for the sshcollector
device spec in deployment.yml, but the data sounds just unassociated
for whatever reason.
I had to learn to nose around the psql tables and basic sql queries
manually (select * from $table;), but it was helpful to see how
things associate, and what data was available to build my own custom
reports. I'd suggest a crash course in postgres and sql queries in
your spare time if planning to stick with netdisco over time. I
took a day or two over the weekend to do so and actually make a
custom report or 3, if you know linux decently already it's not bad,
and learning new stuff is never bad in general.
-mb
On Fri, Nov 12, 2021 at 1:53 PM Jim Araujo <jimara...@gmail.com> wrote:
I believe you are on to something with the problem associating an
asset to the same device or different. The Palo is part of a HA
pair so the IP and MAC are used by both according to Netdisco. If
I search the database for a MAC or IP owned by the Palo it shows
up on both units. Would this explain why searching in the the web
gui shows two devices listed with the same IP but the unit itself
says no address found?
-
On 11/12/2021 12:42 AM, Michael Butash wrote:
I would think with snmp in any version working you would get
*most* data back to build a device model, and arp would give you
the rest for mac associations as a router without a cam table
would link device interfaces. Even stupid cisco firewalls that
still don't support cdp/lldp or arp mib via polling usually give
me a proper response enough to build a netdisco device, but not
enough to associate them properly to a switch link in topology
without the mac->ip via arp tables polled with ssh as Christian
mentioned.
Do you get lldp neighbors show on the pan/f5 and switch both in
netdisco? Maybe debug the device discovery to see what it's
polling and results, or compare the data between the sql tables
if not matching up? Seems perhaps something isn't associating
the same device, snmp vs ssh collector, or conflicting? Should it?
Good data to probably add to the site for troubleshooting ssh
features out of this, something I was meaning to try at my last
project but didn't get to for our pans there.
-mb
On Thu, Nov 11, 2021 at 5:04 PM Jim Araujo <jimara...@gmail.com>
wrote:
Hi Christian,
Thanks again for looking into this. Perhaps I am confused. I
thought for
these particular devices SSHCollector performs the task of
getting IP
addresses?
I've setup the stanza for
https://metacpan.org/pod/App::Netdisco::SSHCollector::Platform::PaloAlto
and looking at the code on github, it looks like the
SSHCollector
performs a "set cli scripting-mode on" and then a "show arp
all". I
believe the collector is working as the database command you
had me run
shows the database has the interface IP addresses. Just the
Web GUI is
blank. If I search for an IP owned by the Palo it shows up
under the
switch it is plugged into, but not for the PaloAlto itself.
On 11/11/2021 6:46 PM, Christian Ramseyer wrote:
> Ok in the macsuck part it says that the device does not
advertise
> layer 2 functionality in the SNMP attributes, and so no
macsuck is
> performed.
>
> I think if you just enter the IP or Mac in the search box
on top of
> the UI, you should get an entry similar to the attached
image. But
> since there is no L2 mapping from Mac to Port it will not
show up in
> the ports of the device, it's just a disconnected ARP entry.
>
> I have no Palo Altos to try, in some cases you can get
lucky by just
> enabling layer 2 in the snmp settings, but more likely Palo
Alto does
> not support these MIBs and will not give you any useful
data. Maybe
> somebody else on the list has experience with this brand.
>
> Cheers
> Christian
>
>
> On 11.11.21 17:29, Jim Araujo wrote:
>> Hi Christian, thanks for the troubleshooting steps. I see
that the
>> last query shows the database has the IPv4 addresses in
it, but the
>> Web Gui does not. The troubleshooting log is from a Palo
alto PA-5220.
>>
>> https://pastecode.io/s/0i2kdb37
>>
>>
>> On 11/11/2021 8:08 AM, Christian Ramseyer wrote:
>>> Hi Jim
>>>
>>> The SSH collector only collects ARP tables. There are two
more
>>> pieces that are needed for everything to show up properly
in the web
>>> interface:
>>>
>>> - discovery of the L2 device: netdisco-do -D discover
-d <IP>
>>> this will give the list of interfaces and other hardware
>>> - collection of the mac address tables: netdisco-do -D
macsuck -d <IP>
>>> this will give the interface -> connected MAC addresses
>>>
>>> If you see ARP entries collected that later can not be
found in the
>>> UI, most likely one of these is not working properly with
your gear.
>>>
>>> I usually look into the database using SQL after running
>>> discover/macsuck/arpnip: netdisco-do psql , then:
>>>
>>> select * from device where ip = <ip of l2 or l3 device>;
>>> -- if there's nothing here, discovery or credentials is
the problem
>>> -- all your routers and switches involved should show
up here
>>>
>>> select * from device_port where ip = <ip of l2 device>;
>>> -- if there's nothing here, discovery found no ports:
>>> -- shitty SNMP support or switch is not added to netdisco
>>>
>>> select * from node where switch = <ip of l2 device>;
>>> -- here you should see some connected macs on ports,
otherwise
>>> -- macsuck issue
>>>
>>> select * from node_ip where mac = <a mac found on a port
in prev.
>>> step>;
>>> -- you should find your arp tables here, otherwise arpnip
issue or
>>> -- you're not pointing netdisco at the correct L3 devices
>>>
>>> Maybe that helps narrowing the issue down a bit.
>>>
>>> Cheers
>>> Christian
>>>
>>>
>>>
>>>
>>> On 10.11.21 17:58, Jim Araujo wrote:
>>>> I believe SSHCollector is getting the ARP entries as
when i do a
>>>> '~/bin/netdisco-do arpnip -d 1.2.3.4 -D' shows
'processed 43 ARP
>>>> cache entries. However nothing is listed in the web gui
for this
>>>> node...Am I running into a bug?
>>>>
>>>> https://sourceforge.net/p/netdisco/netdisco2/254/
>>>>
>>>> On 11/10/2021 8:43 AM, Jim Araujo wrote:
>>>>> Hello netdisco team! First off great work on this app,
it is
>>>>> incredible. SNMPv3 seems to work correctly, however
when I create
>>>>> a new stanza for SSH collection for devices like BigIP,
PaloAlto,
>>>>> and NXOS is there a way to test if they are working? On
the web
>>>>> gui these particular devices do not show any
information under
>>>>> Addresses, nor anything under Ports...I am hoping to
get this
>>>>> working as we have a large F5 environment and would be
helpful to
>>>>> be able to search for IPs that the F5 or PaloAlto owns.
>>>>>
>>>>> Thanks again!
>>>>>
-- -Jim
_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
-- -Jim
_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
--
-Jim
_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
--
-Jim
--- End Message ---
--- Begin Message ---
From your device_ip dump it looks like the issue is the empty "port"
column, this is what allows to map the IPs back to the physical
interface it is configured on.
I found a Big IP in a rarely visited corner of our network (BIG-IP vCMP
Guest : Linux 2.6x, BIG-IP software release 12.1.4.1).
There, it looks like the F5-BIGIP-SYSTEM-MIB::sysInterfaceName :
.1.3.6.1.4.1.3375.2.1.2.4.1.2.1.1 is the problem. This data needs to be
matched with IP-MIB::ipAdEntIfIndex : .1.3.6.1.2.1.4.20.1.2, but the
former does not use the ifEntry ifindex, but a decimal ascii sequence of
the interface names. This can be visualized with netdisco-do show:
$ netdisco-do show -I -d 10.0.29.24 -e ip_index
SNMP::Info::_load_attr old_ip_index : IP-MIB::ipAdEntIfIndex :
.1.3.6.1.2.1.4.20.1.2
{
10.0.29.24 33,
10.0.29.25 33,
169.254.73.9 112
}
$ netdisco-do show -I -d 10.0.29.24 -e interfaces
SNMP::Info::_load_attr i_index : F5-BIGIP-SYSTEM-MIB::sysInterfaceName :
.1.3.6.1.4.1.3375.2.1.2.4.1.2.1.1
{
6.49.47.48.46.49.57 "1/0.19" (dualvar: 1),
6.49.47.48.46.50.48 "1/0.20" (dualvar: 1),
6.49.47.109.103.109.116 "1/mgmt" (dualvar: 1),
6.50.47.48.46.49.57 "2/0.19" (dualvar: 2),
6.50.47.48.46.50.48 "2/0.20" (dualvar: 2),
6.50.47.109.103.109.116 "2/mgmt" (dualvar: 2)
}
I tried a quick hack in lib/SNMP/Info/Layer3/F5.pm that looks up the
actual ifIndex and adds this as an additional row to the result:
$ netdisco-do show -I -d 10.0.29.24 -e interfaces
SNMP::Info::_load_attr i_index : F5-BIGIP-SYSTEM-MIB::sysInterfaceName :
.1.3.6.1.4.1.3375.2.1.2.4.1.2.1.1
{
33 "1/mgmt" (dualvar: 1),
34 "2/mgmt" (dualvar: 2),
177 "1/0.19" (dualvar: 1),
178 "2/0.19" (dualvar: 2),
193 "1/0.20" (dualvar: 1),
194 "2/0.20" (dualvar: 2),
6.49.47.48.46.49.57 "1/0.19" (dualvar: 1),
6.49.47.48.46.50.48 "1/0.20" (dualvar: 1),
6.49.47.109.103.109.116 "1/mgmt" (dualvar: 1),
6.50.47.48.46.49.57 "2/0.19" (dualvar: 2),
6.50.47.48.46.50.48 "2/0.20" (dualvar: 2),
6.50.47.109.103.109.116 "2/mgmt" (dualvar: 2)
}
Unfortunately this Netdisco installation is behind five tunnels and VPNs
and I can not start the GUI, but I'm pretty confident it would now show
up in the Adresses tab. You can give it a try, the interfaces method in
F5.pm would need to look like this:
# Override L3 interfaces
sub interfaces {
my $f5 = shift;
my $partial = shift;
# F5-BIGIP-SYSTEM-MIB::sysInterfaceName uses an ascii
# string of the interface name as table index,
# add an additional record with the regular ifIndex
my $ifnames = $f5->ifName($partial);
my %i2ind = reverse %$ifnames;
my $if = $f5->i_index($partial);
foreach my $k(keys %$if){
my $ifindex = $i2ind{$if->{$k}};
$if->{$ifindex} = $if->{$k};
}
return $if;
}
Then run netdisco-do discover again. If it works maybe a resident
SNMP::Info crack can give us some advice how to make a proper patch for
this, it would need fixing in some more methods and be implemented
module-wide somewhere (is there anything Munge-like for indexes?) Also
I'm not sure if this breaks anything else and if we should keep the
ascii string indexes around.
I imagine the Palo Alto is something similar but I have no way to try
currently. Might be a case for the hot new snapshot feature
(https://github.com/netdisco/netdisco/wiki/Snapshot)
Cheers
Christian
On 15.11.21 14:55, Jim Araujo wrote:
Sure thing Christian.
Both the below Device Types(BigIP * PaloAlto) seem to have similar
issues with Addresses reporting in the DB, but not the WebGUI.
This pate-code is the -DISQ(first link) and DB commands(second link)
from a BigIP F5 i2800
https://pastecode.io/s/avkzee1x
https://pastecode.io/s/y3ekxc8r
This Paste-code is the -DISQ(first link) and DB commands(second link)
from a PaloAlto PA-5220
https://pastecode.io/s/vagnng2z
https://pastecode.io/s/wvsppfdh
On 11/15/2021 7:17 AM, Christian Ramseyer wrote:
Hi Jim
Ok it wasn't quite clear to me earlier in this thread where you're not
finding the expected IPs. For the addresses tab, the data is collected
in the discover step, and written into the table device_ip. This is
most likely empty for your device, you can check with:
db=> select * from device_ip where ip = 'x.x.x.x';
IIRC this uses IP-MIB, but you can see exactly what OIDs are queried,
and what is returned by running
netdisco-do discover -DISQ -d x.x.x.x
Unfortunately your pastecode snipped expired so I can't tell if there
were any errors there, but this should point the search in the right
direction.
Cheers
Christian
On 12.11.21 22:59, Jim Araujo wrote:
Alrighty. I just erased the db from postgres and add a single device
having the issue. Still occuring only with one device.
https://pastecode.io/s/tuuc90gy
and the address page
On 11/12/2021 4:34 PM, Michael Butash wrote:
Why I mentioned comparing the databases, though not sure how they
link to each other, by a device column or some index, or combination
thereof. If the ssh collected mac table is using a wrong device to
link to snmp data, it'll just treat them as two broken sets of
devices vs. merging and associating them. Could just be something
dumb like ip vs. name vs. whatever you used for the sshcollector
device spec in deployment.yml, but the data sounds just unassociated
for whatever reason.
I had to learn to nose around the psql tables and basic sql queries
manually (select * from $table;), but it was helpful to see how
things associate, and what data was available to build my own custom
reports. I'd suggest a crash course in postgres and sql queries in
your spare time if planning to stick with netdisco over time. I
took a day or two over the weekend to do so and actually make a
custom report or 3, if you know linux decently already it's not bad,
and learning new stuff is never bad in general.
-mb
On Fri, Nov 12, 2021 at 1:53 PM Jim Araujo <jimara...@gmail.com> wrote:
I believe you are on to something with the problem associating an
asset to the same device or different. The Palo is part of a HA
pair so the IP and MAC are used by both according to Netdisco. If
I search the database for a MAC or IP owned by the Palo it shows
up on both units. Would this explain why searching in the the web
gui shows two devices listed with the same IP but the unit itself
says no address found?
-
On 11/12/2021 12:42 AM, Michael Butash wrote:
I would think with snmp in any version working you would get
*most* data back to build a device model, and arp would give you
the rest for mac associations as a router without a cam table
would link device interfaces. Even stupid cisco firewalls that
still don't support cdp/lldp or arp mib via polling usually give
me a proper response enough to build a netdisco device, but not
enough to associate them properly to a switch link in topology
without the mac->ip via arp tables polled with ssh as Christian
mentioned.
Do you get lldp neighbors show on the pan/f5 and switch both in
netdisco? Maybe debug the device discovery to see what it's
polling and results, or compare the data between the sql tables
if not matching up? Seems perhaps something isn't associating
the same device, snmp vs ssh collector, or conflicting? Should it?
Good data to probably add to the site for troubleshooting ssh
features out of this, something I was meaning to try at my last
project but didn't get to for our pans there.
-mb
On Thu, Nov 11, 2021 at 5:04 PM Jim Araujo <jimara...@gmail.com>
wrote:
Hi Christian,
Thanks again for looking into this. Perhaps I am confused. I
thought for
these particular devices SSHCollector performs the task of
getting IP
addresses?
I've setup the stanza for
https://metacpan.org/pod/App::Netdisco::SSHCollector::Platform::PaloAlto
and looking at the code on github, it looks like the
SSHCollector
performs a "set cli scripting-mode on" and then a "show arp
all". I
believe the collector is working as the database command you
had me run
shows the database has the interface IP addresses. Just the
Web GUI is
blank. If I search for an IP owned by the Palo it shows up
under the
switch it is plugged into, but not for the PaloAlto itself.
On 11/11/2021 6:46 PM, Christian Ramseyer wrote:
> Ok in the macsuck part it says that the device does not
advertise
> layer 2 functionality in the SNMP attributes, and so no
macsuck is
> performed.
>
> I think if you just enter the IP or Mac in the search box
on top of
> the UI, you should get an entry similar to the attached
image. But
> since there is no L2 mapping from Mac to Port it will not
show up in
> the ports of the device, it's just a disconnected ARP entry.
>
> I have no Palo Altos to try, in some cases you can get
lucky by just
> enabling layer 2 in the snmp settings, but more likely Palo
Alto does
> not support these MIBs and will not give you any useful
data. Maybe
> somebody else on the list has experience with this brand.
>
> Cheers
> Christian
>
>
> On 11.11.21 17:29, Jim Araujo wrote:
>> Hi Christian, thanks for the troubleshooting steps. I see
that the
>> last query shows the database has the IPv4 addresses in
it, but the
>> Web Gui does not. The troubleshooting log is from a Palo
alto PA-5220.
>>
>> https://pastecode.io/s/0i2kdb37
>>
>>
>> On 11/11/2021 8:08 AM, Christian Ramseyer wrote:
>>> Hi Jim
>>>
>>> The SSH collector only collects ARP tables. There are two
more
>>> pieces that are needed for everything to show up properly
in the web
>>> interface:
>>>
>>> - discovery of the L2 device: netdisco-do -D discover
-d <IP>
>>> this will give the list of interfaces and other hardware
>>> - collection of the mac address tables: netdisco-do -D
macsuck -d <IP>
>>> this will give the interface -> connected MAC addresses
>>>
>>> If you see ARP entries collected that later can not be
found in the
>>> UI, most likely one of these is not working properly with
your gear.
>>>
>>> I usually look into the database using SQL after running
>>> discover/macsuck/arpnip: netdisco-do psql , then:
>>>
>>> select * from device where ip = <ip of l2 or l3 device>;
>>> -- if there's nothing here, discovery or credentials is
the problem
>>> -- all your routers and switches involved should show
up here
>>>
>>> select * from device_port where ip = <ip of l2 device>;
>>> -- if there's nothing here, discovery found no ports:
>>> -- shitty SNMP support or switch is not added to netdisco
>>>
>>> select * from node where switch = <ip of l2 device>;
>>> -- here you should see some connected macs on ports,
otherwise
>>> -- macsuck issue
>>>
>>> select * from node_ip where mac = <a mac found on a port
in prev.
>>> step>;
>>> -- you should find your arp tables here, otherwise arpnip
issue or
>>> -- you're not pointing netdisco at the correct L3 devices
>>>
>>> Maybe that helps narrowing the issue down a bit.
>>>
>>> Cheers
>>> Christian
>>>
>>>
>>>
>>>
>>> On 10.11.21 17:58, Jim Araujo wrote:
>>>> I believe SSHCollector is getting the ARP entries as
when i do a
>>>> '~/bin/netdisco-do arpnip -d 1.2.3.4 -D' shows
'processed 43 ARP
>>>> cache entries. However nothing is listed in the web gui
for this
>>>> node...Am I running into a bug?
>>>>
>>>> https://sourceforge.net/p/netdisco/netdisco2/254/
>>>>
>>>> On 11/10/2021 8:43 AM, Jim Araujo wrote:
>>>>> Hello netdisco team! First off great work on this app,
it is
>>>>> incredible. SNMPv3 seems to work correctly, however
when I create
>>>>> a new stanza for SSH collection for devices like BigIP,
PaloAlto,
>>>>> and NXOS is there a way to test if they are working? On
the web
>>>>> gui these particular devices do not show any
information under
>>>>> Addresses, nor anything under Ports...I am hoping to
get this
>>>>> working as we have a large F5 environment and would be
helpful to
>>>>> be able to search for IPs that the F5 or PaloAlto owns.
>>>>>
>>>>> Thanks again!
>>>>>
-- -Jim
_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
-- -Jim
_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
--
-Jim
_______________________________________________
Netdisco mailing list
netdisco-users@lists.sourceforge.net
https://sourceforge.net/p/netdisco/mailman/netdisco-users/
--
Christian Ramseyer, netnea ag
Network Management. Security. OpenSource.
Phone: +41 79 644 77 64
Teams: ramse...@netnea.onmicrosoft.com
--- End Message ---
_______________________________________________
Netdisco mailing list - Digest Mode
netdisco-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/netdisco-users
