Hi!
<TL;DR>
After a recent network configuration change (added re(4), vether(4)) I'm
experiencing this memory leak from somewhere.
How can I check/query how much memory the kernel (or parts of it) is
using over time, besides running top(1) with system processes shown --
I'm also staring at systat(1)'s `malloc' and `pool' views but I'm not
really sure what I'm (or rather what I should be) looking at.
</TL;DR>
I have an on-board Realtek NIC and an add-on Intel PCIe 4 port NIC.
re0 at pci3 dev 0 function 0 "Realtek 8168" rev 0x0c: RTL8168G/8111G (0x4c00),
msi
em0 at pci1 dev 0 function 0 "Intel I350" rev 0x01: msi
em1 at pci1 dev 0 function 1 "Intel I350" rev 0x01: msi
em2 at pci1 dev 0 function 2 "Intel I350" rev 0x01: msi
em3 at pci1 dev 0 function 3 "Intel I350" rev 0x01: msi
Before I started to experience the leak I had the on-board NIC disabled
in BIOS and only used two ports of the Intel NIC. My setup looked like
this:
== Network
ISP --- pppoe0 <-> em0 -- OpenBSD 6.4 -- em1(dhcpd) --- [switch] LAN
`- athn0 --- LAN
`- athn1 --- LAN
`- bridge0(em1, athn[01])
Internal IP configured on em1.
athn? devices don't have IPs, just bridged to the internal network.
dhcpd(8) is running on em1.
Now I started to use re0 and one more NIC on the Intel card:
ISP --- pppoe0 <-> re0 -- OpenBSD 6.4 -- em0 --- LAN
`- em1 --- LAN
`- em2 --- LAN
`- athn0 --- LAN
`- athn1 --- LAN
`- vether0 (dhcpd)
`- bridge0(em[123], athn[01], vether0)
Now the 4 port Intel NIC basically acts as a switch, the em? interfaces
don't have IP addresses configured, and the athn? interfaces don't have
one either.
I've added a vether(4) interface with an internal IP (same as the em1
have had before) to the bridge, and dhcpd(8) (and everything else that
needs it) now uses that interface.
# ifconfig bridge0
bridge0: flags=41<UP,RUNNING>
index 10 llprio 3
groups: bridge
priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp
designated: id 00:00:00:00:00:00 priority 0
athn0 flags=3<LEARNING,DISCOVER>
port 6 ifpriority 0 ifcost 0
[bunch of `pass/block in/out on athn0 src/dst <MAC>' rules]
athn1 flags=3<LEARNING,DISCOVER>
port 7 ifpriority 0 ifcost 0
[bunch of `pass/block in/out on athn1 src/dst <MAC>' rules]
em0 flags=3<LEARNING,DISCOVER>
port 1 ifpriority 0 ifcost 0
em1 flags=3<LEARNING,DISCOVER>
port 2 ifpriority 0 ifcost 0
em2 flags=3<LEARNING,DISCOVER>
port 3 ifpriority 0 ifcost 0
em3 flags=3<LEARNING,DISCOVER>
port 4 ifpriority 0 ifcost 0
vether0 flags=3<LEARNING,DISCOVER>
port 13 ifpriority 0 ifcost 0
Addresses (max cache: 100, timeout: 240):
[...]
# ifconfig vether0
vether0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
lladdr fe:e1:ba:d0:58:44
description: Internal LAN
index 13 priority 0 llprio 3
groups: vether
media: Ethernet autoselect
status: active
inet 192.168.0.1 netmask 0xffff0000 broadcast 192.168.255.255
inet6 <> prefixlen 64 scopeid 0xd
inet6 <> prefixlen 64 pltime 573642 vltime 573642
# ifconfig em
em0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu
1500
lladdr
description: Internal LAN - Room1
index 1 priority 0 llprio 3
media: Ethernet autoselect (100baseTX full-duplex,rxpause,txpause)
status: active
em1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu
1500
lladdr
description: Internal LAN - Room2
index 2 priority 0 llprio 3
media: Ethernet autoselect (1000baseT
full-duplex,master,rxpause,txpause)
status: active
em2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu
1500
lladdr
description: Internal LAN - Room3
index 3 priority 0 llprio 3
media: Ethernet autoselect (100baseTX full-duplex,rxpause,txpause)
status: active
em3: flags=8902<BROADCAST,PROMISC,SIMPLEX,MULTICAST> mtu 1500
lladdr
index 4 priority 0 llprio 3
media: Ethernet autoselect (none)
status: no carrier
Nothing fancy, just basically a "software switch" for connecting
separate rooms in the house and adding a vether(4) interface for
conveniency.
Also, everything else remain unchanged, i.e.: no additional services
running, no new software has been installed, no introduction of new pf
rules, other than replacing the interface names of course.
I'm using /etc/malloc.conf@ -> S on all my machines but never had any
problems with it (and still don't anywhere else).
== pf(4)
pf(4) is used on the box, and the rules that have been changed are the
ones related to the interfaces mentioned above:
set skip on lo0
set skip on bridge0
pass on vether0 all flags S/SA allow-opts
pass on em0 all flags S/SA allow-opts
pass on em1 all flags S/SA allow-opts
pass on em2 all flags S/SA allow-opts
pass on em3 all flags S/SA allow-opts
pass on athn0 all flags S/SA allow-opts
pass on athn1 all flags S/SA allow-opts
match in on vether0 inet proto tcp from 192.168.0.0/16 port = <port> to any set
(prio(2, 6), queue(ssh_bulk, ssh_rt))
block return on vether0 proto tcp from any to any port = 25
block return on em0 proto tcp from any to any port = 25
block return on em1 proto tcp from any to any port = 25
block return on em2 proto tcp from any to any port = 25
block return on em3 proto tcp from any to any port = 25
block return on athn0 proto tcp from any to any port = 25
block return on athn1 proto tcp from any to any port = 25
Previously I've had `skip' on athn? as well.
(I can upload the complete rule set somewhere if it is of interest)
== Processes
So first I naturally suspected one of the daemons I'm running, and
there's a few of them:
>From base:
apmd
dhcpd
hostapd
nsd
ntpd
pflogd
rad
sensorsd
slaacd
smtpd
sshd
syslogd
unbound
(tmux)
>From ports:
- collectd-5.8.0p2
- dhcpcd-7.1.1p4 (updated to the version in HEAD, this was my first
suspicion, 'cause way back I've had memory leak with an older version)
- miniupnpd-1.9p4
== Monitoring
I'm constantly monitoring the system with collectd/facette, and I've yet
to pinpoint a single process that would consume more and more over time.
Yet, top(1) displayed this over the course of one day and 9 hours re
memory usage:
@2019-04-14T00:00:00
load averages: 0.20, 0.25, 0.24 00:00:00
Memory: Real: 188M/736M act/tot Free: 1168M Cache: 379M Swap: 0K/2189M
@2019-04-15T09:00:01
load averages: 0.42, 0.34, 0.28 09:00:01
Memory: Real: 226M/1209M act/tot Free: 695M Cache: 379M Swap: 0K/2189M
Between 8am and 12pm the same day:
@2019-04-15T08:00:00
load averages: 0.18, 0.24, 0.24 08:00:00
Memory: Real: 225M/1197M act/tot Free: 707M Cache: 379M Swap: 0K/2189M
@2019-04-15T12:00:00
load averages: 0.38, 0.30, 0.26 12:00:00
Memory: Real: 215M/1238M act/tot Free: 666M Cache: 379M Swap: 0K/2189M
Now disregarding that memory usage fluctuates and it won't stay constant
forever, this is still slowly consuming the memory, and unfortunately I
noticed this box was unresponsive two times now before I started to
investigate:
- No network access whatsoever (no icmp replies)
- Connecting to the console, it *seemed* responsive, I could type in
anything into the login screen but after hitting enter after the
username it just hanged there.
- For some strange reason Ctrl+Alt+Esc didn't work, albeit ddb.console=1
== Summary
So maybe I'm now a bit biased but I'm starting to contemplate the idea
that maybe that network topology change, vether(4) or maybe re(4) could
be the culprit.
Could someone recommend a good way to rule out kernel network driver
leak? I was thinking about monitoring <something> and see how memory
consumption changes but I'm not sure where to start.
Or maybe if someone have had troubles using vether(4) in a bridge(4) or
re(4) in the past, I would be very curious to hear that as well!
Thanks in advance for any hints!
Daniel
== dmesg
OpenBSD 6.4 (GENERIC.MP) #9: Tue Mar 26 19:21:43 CET 2019
r...@syspatch-64-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 2059997184 (1964MB)
avail mem = 1988288512 (1896MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec130 (75 entries)
bios0: vendor American Megatrends Inc. version "F2" date 08/06/2015
bios0: Gigabyte Technology Co., Ltd. H81M-DS2
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC FPDT SSDT SSDT MCFG HPET SSDT SSDT
acpi0: wakeup devices PXSX(S4) RP01(S4) PXSX(S4) PXSX(S4) RP03(S4) PXSX(S4)
PXSX(S4) RP05(S4) PXSX(S4) RP06(S4) PXSX(S4) PXSX(S4) GLAN(S4) EHC1(S4)
EHC2(S4) XHC_(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Celeron(R) CPU G1840 @ 2.80GHz, 2793.95 MHz, 06-3c-03
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Celeron(R) CPU G1840 @ 2.80GHz, 2793.55 MHz, 06-3c-03
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xf8000000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 2 (RP01)
acpiprt2 at acpi0: bus 3 (RP03)
acpiprt3 at acpi0: bus 4 (RP05)
acpiprt4 at acpi0: bus 5 (RP06)
acpiprt5 at acpi0: bus 1 (PEG0)
acpiprt6 at acpi0: bus -1 (PEG1)
acpiprt7 at acpi0: bus -1 (PEG2)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C2(500@67 mwait.1@0x10), C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C2(500@67 mwait.1@0x10), C1(1000@1 mwait.1), PSS
acpipwrres0 at acpi0: FN00, resource for FAN0
acpipwrres1 at acpi0: FN01, resource for FAN1
acpipwrres2 at acpi0: FN02, resource for FAN2
acpipwrres3 at acpi0: FN03, resource for FAN3
acpipwrres4 at acpi0: FN04, resource for FAN4
acpitz0 at acpi0: critical temperature is 105 degC
acpitz1 at acpi0: critical temperature is 105 degC
acpicmos0 at acpi0
acpibtn0 at acpi0: PWRB
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD1F
cpu0: Enhanced SpeedStep 2793 MHz: speeds: 2800, 2700, 2500, 2400, 2300, 2100,
2000, 1900, 1700, 1600, 1500, 1300, 1200, 1100, 900, 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Core 4G Host" rev 0x06
ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi
pci1 at ppb0 bus 1
em0 at pci1 dev 0 function 0 "Intel I350" rev 0x01: msi, address
em1 at pci1 dev 0 function 1 "Intel I350" rev 0x01: msi, address
em2 at pci1 dev 0 function 2 "Intel I350" rev 0x01: msi, address
em3 at pci1 dev 0 function 3 "Intel I350" rev 0x01: msi, address
inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics" rev 0x06
drm0 at inteldrm0
inteldrm0: msi
error: [drm:pid0:drm_edid_block_valid] *ERROR* EDID checksum is invalid,
remainder is 39
Raw EDID:
00 ff ff ff ff ff ff 00 fc ad 00 00 00 00 00 00
fe 11 01 03 08 25 1e 78 ca bb 05 a3 56 47 99 24
f4 50 54 2f ce 00 01 01 e7 ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
error: [drm:pid0:drm_edid_block_valid] *ERROR* EDID checksum is invalid,
remainder is 7
Raw EDID:
00 ff ff ff ff ff ff 00 fc ad 00 00 00 00 00 00
de 11 01 03 08 25 1e 78 ea bb 05 a3 56 47 99 24
d4 50 54 2f ce 00 01 01 e7 ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
error: [drm:pid0:drm_edid_block_valid] *ERROR* EDID checksum is invalid,
remainder is 39
Raw EDID:
00 ff ff ff ff ff ff 00 fc ad 00 00 00 00 00 00
de 11 01 03 08 25 1e 78 ea bb 05 a3 56 47 99 24
f4 50 54 2f ce 00 01 01 e7 ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
error: [drm:pid0:drm_edid_block_valid] *ERROR* EDID checksum is invalid,
remainder is 71
Raw EDID:
00 ff ff ff ff ff ff 00 fc ad 00 00 00 00 00 00
fe 11 01 03 08 25 1e 78 ea bb 05 a3 56 47 99 24
f4 50 54 2f ce 00 01 01 e7 ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
drm:pid0:drm_do_get_edid *WARNING* VGA-: EDID block 0 invalid.
inteldrm0: 1024x768, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
azalia0 at pci0 dev 3 function 0 "Intel Core 4G HD Audio" rev 0x06: msi
azalia0: No codecs found
"Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 16
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
addr 1
ppb1 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi
pci2 at ppb1 bus 2
ppb2 at pci0 dev 28 function 2 "Intel 8 Series PCIE" rev 0xd5: msi
pci3 at ppb2 bus 3
re0 at pci3 dev 0 function 0 "Realtek 8168" rev 0x0c: RTL8168G/8111G (0x4c00),
msi, address
rgephy0 at re0 phy 7: RTL8251 PHY, rev. 0
ppb3 at pci0 dev 28 function 4 "Intel 8 Series PCIE" rev 0xd5: msi
pci4 at ppb3 bus 4
athn0 at pci4 dev 0 function 0 "Atheros AR9281" rev 0x01: apic 8 int 16
athn0: AR9280 rev 2 (2T2R), ROM rev 16, address
ppb4 at pci0 dev 28 function 5 "Intel 8 Series PCIE" rev 0xd5: msi
pci5 at ppb4 bus 5
athn1 at pci5 dev 0 function 0 "Atheros AR9281" rev 0x01: apic 8 int 17
athn1: AR9280 rev 2 (2T2R), ROM rev 22, address
ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
addr 1
pcib0 at pci0 dev 31 function 0 "Intel H81 LPC" rev 0x05
ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi, AHCI 1.3
ahci0: port 4: 3.0Gb/s
ahci0: port 5: 3.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 4 lun 0: <ATA, KINGSTON SHFS37A, 608A> SCSI3 0/direct
fixed naa.50026b726304fb22
sd0: 114473MB, 512 bytes/sector, 234441648 sectors, thin
sd1 at scsibus1 targ 5 lun 0: <ATA, KINGSTON SHFS37A, 608A> SCSI3 0/direct
fixed naa.50026b726304fbe2
sd1: 114473MB, 512 bytes/sector, 234441648 sectors, thin
ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: apic 8 int 18
iic0 at ichiic0
spdmem0 at iic0 addr 0x50: 2GB DDR3 SDRAM PC3-10600
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev
2.00/0.05 addr 2
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev
2.00/0.05 addr 2
uhidev0 at uhub3 port 2 configuration 1 interface 0 "SINO WEALTH USB KEYBOARD"
rev 1.10/1.00 addr 3
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at uhub3 port 2 configuration 1 interface 1 "SINO WEALTH USB KEYBOARD"
rev 1.10/1.00 addr 3
uhidev1: iclass 3/0, 5 report ids
uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
uhid1 at uhidev1 reportid 3: input=3, output=0, feature=0
uhid2 at uhidev1 reportid 5: input=0, output=0, feature=5
uhidev2 at uhub3 port 3 configuration 1 interface 0 "American Power Conversion
Back-UPS XS 1400U FW:926.T1 .I USB FW:T1" rev 1.10/1.06 addr 4
uhidev2: iclass 3/0, 123 report ids
upd0 at uhidev2
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
sd2 at scsibus3 targ 1 lun 0: <OPENBSD, SR RAID 1, 006> SCSI2 0/direct fixed
sd2: 114470MB, 512 bytes/sector, 234435953 sectors
root on sd2a (59f8b4a610bc10ff.a) swap on sd2b dump on sd2b
pppoe: GENERIC ERROR: RP-PPPoE: Child pppd process terminated
wskbd1: disconnecting from wsdisplay0
wskbd1 detached
ukbd0 detached
uhidev0 detached
uhid0 detached
uhid1 detached
uhid2 detached
uhidev1 detached
pppoe0: received unexpected PADO
pppoe0: received unexpected PADO
pppoe0: received unexpected PADO
upd0 detached
uhidev2 detached
uhidev0 at uhub3 port 3 configuration 1 interface 0 "American Power Conversion
Back-UPS XS 1400U FW:926.T1 .I USB FW:T1" rev 1.10/1.06 addr 3
uhidev0: iclass 3/0, 123 report ids
upd0 at uhidev0
upd0 detached
uhidev0 detached
uhidev0 at uhub3 port 3 configuration 1 interface 0 "American Power Conversion
Back-UPS XS 1400U FW:926.T1 .I USB FW:T1" rev 1.10/1.06 addr 3
uhidev0: iclass 3/0, 123 report ids
upd0 at uhidev0
upd0 detached
uhidev0 detached
uhidev0 at uhub3 port 3 configuration 1 interface 0 "American Power Conversion
Back-UPS XS 1400U FW:926.T1 .I USB FW:T1" rev 1.10/1.06 addr 3
uhidev0: iclass 3/0, 123 report ids
upd0 at uhidev0
upd0 detached
uhidev0 detached
uhidev0 at uhub3 port 3 configuration 1 interface 0 "American Power Conversion
Back-UPS XS 1400U FW:926.T1 .I USB FW:T1" rev 1.10/1.06 addr 3
uhidev0: iclass 3/0, 123 report ids
upd0 at uhidev0
upd0 detached
uhidev0 detached
uhub3: device problem, disabling port 3
uhub3: port error, restarting port 3
uhidev0 at uhub3 port 3 configuration 1 interface 0 "American Power Conversion
Back-UPS XS 1400U FW:926.T1 .I USB FW:T1" rev 1.10/1.06 addr 3
uhidev0: iclass 3/0, 123 report ids
upd0 at uhidev0
--
LÉVAI Dániel
PGP key ID = 0x83B63A8F
Key fingerprint = DBEC C66B A47A DFA2 792D 650C C69B BE4C 83B6 3A8F
