Great, I'll see about cutting a release soon. On Mon, Mar 28, 2022 at 1:34 PM [email protected] <[email protected]> wrote:
> no stack traces anymore. It's running now for a week. > > again, thanks a lot for the hint. > > Regards > > Olaf > > > On Tuesday, 22 March 2022 at 14:10:29 UTC+1 [email protected] wrote: > >> This snmp_exporter binary was built against master and with go 18.1 and >> gosnmp v1.34.0. >> Looks very promising. It's running for 3h now and no trace at all. I also >> like the new error messages, which make much more sense, instead of the old >> timeout message, because everything was a timeout in the old exporter. >> >> ts=2022-03-22T12:26:40.683Z caller=collector.go:274 level=info >> module=pxgeneric target=10.11.25.10 msg="Error scraping target" err="error >> getting target 10.11.25.10: unknown username" >> ts=2022-03-22T12:33:09.370Z caller=collector.go:274 level=info >> module=pxdlrouternxos target=10.11.25.14 msg="Error scraping target" >> err="error getting target 10.11.25.14: not in time window" >> >> >> Thanks a lot and regards, >> >> Olaf >> >> On Monday, 21 March 2022 at 21:17:24 UTC+1 [email protected] wrote: >> >>> I wonder if this is an instance of >>> https://github.com/gosnmp/gosnmp/pull/354. >>> >>> This is fixed in the Go SNMP library, but we haven't cut a new >>> snmp_exporter build in a while, so it's not in the release. >>> >>> If you could try a fresh build from main to see if it's still a bug. >>> >>> On Fri, Mar 18, 2022 at 11:03 AM [email protected] <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> we are running on 0.20.0 and scraping devices, which are not configured >>>> correctly (probably acl issue on switch side), produce these errors all the >>>> time. This device for example is a Cisco N7k. >>>> >>>> ``` >>>> level=info ts=2022-03-18T08:29:09.347Z caller=collector.go:224 >>>> module=pxdlrouternxos target=10.14.25.10 msg="Error scraping target" >>>> err="error getting target 10.14.25.10: recover: runtime error: invalid >>>> memory address or nil pointer dereference\nStack:goroutine 358625 >>>> [running]:\ >>>> ngithub.com/gosnmp/gosnmp.(*GoSNMP).send.func1(0xc000e155c8)\n\t/go/pkg/mod/github.com/gosnmp/[email protected]/marshal.go:326 >>>> <http://ngithub.com/gosnmp/gosnmp.(*GoSNMP).send.func1(0xc000e155c8)%5Cn%5Ct/go/pkg/mod/github.com/gosnmp/[email protected]/marshal.go:326> >>>> +0xa5\npanic(0x9e4880, 0xe89790)\n\t/usr/local/go/src/runtime/panic.go:969 >>>> +0x1b9\ngithub.com/gosnmp/gosnmp.(*GoSNMP).send(0xc0004f6c60, >>>> 0xc0004c42a0, 0xc0000de601, 0x0, 0xb34ee0, 0xc000b609c0)\n\t/go/pkg/mod/ >>>> github.com/gosnmp/[email protected]/marshal.go:372 >>>> <http://github.com/gosnmp/[email protected]/marshal.go:372> +0x20e\ >>>> ngithub.com/gosnmp/gosnmp.(*GoSNMP).Get(0xc0004f6c60, 0xc00012e300, >>>> 0x6, 0x6, 0x0, 0x0, 0xc000522b10)\n\t/go/pkg/mod/ >>>> github.com/gosnmp/[email protected]/gosnmp.go:363 >>>> <http://github.com/gosnmp/[email protected]/gosnmp.go:363> >>>> +0x159\nmain.ScrapeTarget(0xb40660, 0xc0004b41c0, 0xc00045ca67, 0xe, >>>> 0xc000161770, 0xb34f40, 0xc000522840, 0x0, 0x0, 0x0, >>>> ...)\n\t/app/collector.go:133 +0x5be\nmain.collector.Collect(0xb40660, >>>> 0xc0004b41c0, 0xc00045ca67, 0xe, 0xc000161770, 0xb34f40, 0xc000522840, >>>> 0xc000281020)\n\t/app/collector.go:222 +0xbc\ >>>> ngithub.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()\n\t/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/registry.go:446 >>>> <http://ngithub.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()%5Cn%5Ct/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/registry.go:446> >>>> +0x1a2\ncreated by >>>> github.com/prometheus/client_golang/prometheus.(*Registry).Gather\n\t/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/registry.go:457 >>>> <http://github.com/prometheus/client_golang/prometheus.(*Registry).Gather%5Cn%5Ct/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/registry.go:457> >>>> +0x5ce\n\ngoroutine 1 [IO wait, 216 >>>> minutes]:\ninternal/poll.runtime_pollWait(0x7f0f18245708, 0x72, >>>> 0x0)\n\t/usr/local/go/src/runtime/netpoll.go:222 >>>> +0x55\ninternal/poll.(*pollDesc).wait(0xc0004c6018, 0x72, 0x0, 0x0, >>>> 0xa85b68)\n\t/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 >>>> +0x45\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/local/go/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Accept(0xc0004c6000, >>>> 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, >>>> 0x0)\n\t/usr/local/go/src/internal/poll/fd_unix.go:394 >>>> +0x1fc\nnet.(*netFD).accept(0xc0004c6000, 0x1eadb908595c5b95, 0x0, >>>> 0x0)\n\t/usr/local/go/src/net/fd_unix.go:172 >>>> +0x45\nnet.(*TCPListener).accept(0xc0004a8300, 0x6234102b, 0xc00052dad0, >>>> 0x48e3a6)\n\t/usr/local/go/src/net/tcpsock_posix.go:139 >>>> +0x32\nnet.(*TCPListener).Accept(0xc0004a8300, 0xc00052db20, 0x18, >>>> 0xc000000180, 0x7f8dec)\n\t/usr/local/go/src/net/tcpsock.go:261 >>>> +0x65\nnet/http.(*Server).Serve(0xc0004c4000, 0xb3ee20, 0xc0004a8300, 0x0, >>>> 0x0)\n\t/usr/local/go/src/net/http/server.go:2937 +0x266\ >>>> ngithub.com/prometheus/exporter-toolkit/web.Serve(0xb3ee20, >>>> 0xc0004a8300, 0xc0004c4000, 0x0, 0x0, 0xb34f40, 0xc000286420, 0x0, >>>> 0xe0)\n\t/go/pkg/mod/ >>>> github.com/prometheus/[email protected]/web/tls_config.go:192 >>>> <http://github.com/prometheus/[email protected]/web/tls_config.go:192> >>>> +0x1b0\ >>>> ngithub.com/prometheus/exporter-toolkit/web.ListenAndServe(0xc0004c4000, >>>> 0x0, 0x0, 0xb34f40, 0xc000286420, 0x0, 0x0)\n\t/go/pkg/mod/ >>>> github.com/prometheus/[email protected]/web/tls_config.go:184 >>>> <http://github.com/prometheus/[email protected]/web/tls_config.go:184> >>>> +0xfd\nmain.main()\n\t/app/main.go:248 +0xb71\n\ngoroutine 45 [syscall, >>>> 3871 >>>> minutes]:\nos/signal.signal_recv(0x0)\n\t/usr/local/go/src/runtime/sigqueue.go:147 >>>> +0x9d\nos/signal.loop()\n\t/usr/local/go/src/os/signal/signal_unix.go:23 >>>> +0x25\ncreated by >>>> os/signal.Notify.func1.1\n\t/usr/local/go/src/os/signal/signal.go:150 >>>> +0x45\n\ngoroutine 46 [select, 3871 >>>> minutes]:\nmain.main.func1(0xc00012e360, 0xb34f40, >>>> 0xc000286420)\n\t/app/main.go:179 +0xe5\ncreated by >>>> main.main\n\t/app/main.go:177 +0x7ce\n\ngoroutine 50 [IO >>>> wait]:\ninternal/poll.runtime_pollWait(0x7f0f18245620, 0x72, >>>> 0xb36220)\n\t/usr/local/go/src/runtime/netpoll.go:222 >>>> +0x55\ninternal/poll.(*pollDesc).wait(0xc0004c6098, 0x72, 0xb36200, >>>> 0xe444f0, 0x0)\n\t/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 >>>> +0x45\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/local/go/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc0004c6080, >>>> 0xc0004d8000, 0x1000, 0x1000, 0x0, 0x0, >>>> 0x0)\n\t/usr/local/go/src/internal/poll/fd_unix.go:159 >>>> +0x1a5\nnet.(*netFD).Read(0xc0004c6080, 0xc0004d8000, 0x1000, 0x1000, >>>> 0x92a41b, 0xc00052f7f8, 0x7f2b36)\n\t/usr/local/go/src/net/fd_posix.go:55 >>>> +0x4f\nnet.(*conn).Read(0xc0004ae050, 0xc0004d8000, 0x1000, 0x1000, 0x0, >>>> 0x0, 0x0)\n\t/usr/local/go/src/net/net.go:182 >>>> +0x8e\nnet/http.(*connReader).Read(0xc0004ac750, 0xc0004d8000, 0x1000, >>>> 0x1000, 0x10012864e, 0x0, >>>> 0x7f0f18281db8)\n\t/usr/local/go/src/net/http/server.go:798 >>>> +0x1ad\nbufio.(*Reader).fill(0xc0002806c0)\n\t/usr/local/go/src/bufio/bufio.go:101 >>>> +0x105\nbufio.(*Reader).ReadSlice(0xc0002806c0, 0xc00025460a, >>>> 0x7f0f18281db8, 0xc00052f988, 0x40d950, 0xc00022a100, >>>> 0x100)\n\t/usr/local/go/src/bufio/bufio.go:360 >>>> +0x3d\nbufio.(*Reader).ReadLine(0xc0002806c0, 0xc00022a100, 0x479294, >>>> 0xea09c0, 0x0, 0xa5b4c0, >>>> 0xc000730a80)\n\t/usr/local/go/src/bufio/bufio.go:389 >>>> +0x34\nnet/textproto.(*Reader).readLineSlice(0xc000730a80, 0xc00022a100, >>>> 0x4d788d, 0xc0004c6080, 0x467500, >>>> 0xc001bc3e00)\n\t/usr/local/go/src/net/textproto/reader.go:58 >>>> +0x6c\nnet/textproto.(*Reader).ReadLine(...)\n\t/usr/local/go/src/net/textproto/reader.go:39\nnet/http.readRequest(0xc0002806c0, >>>> 0x0, 0xc00022a100, 0x0, 0x0)\n\t/usr/local/go/src/net/http/request.go:1012 >>>> +0xaa\nnet/http.(*conn).readRequest(0xc00024c0a0, 0xb40660, 0xc0004b4180, >>>> 0x0, 0x0, 0x0)\n\t/usr/local/go/src/net/http/server.go:984 >>>> +0x19a\nnet/http.(*conn).serve(0xc00024c0a0, 0xb40660, >>>> 0xc0004b4180)\n\t/usr/local/go/src/net/http/server.go:1851 +0x705\ncreated >>>> by net/http.(*Server).Serve\n\t/usr/local/go/src/net/http/server.go:2969 >>>> +0x36c\n\ngoroutine 358626 >>>> [semacquire]:\nsync.runtime_Semacquire(0xc000880838)\n\t/usr/local/go/src/runtime/sema.go:56 >>>> +0x45\nsync.(*WaitGroup).Wait(0xc000880830)\n\t/usr/local/go/src/sync/waitgroup.go:130 >>>> +0x65\ >>>> ngithub.com/prometheus/client_golang/prometheus.(*Registry).Gather.func2(0xc000880830, >>>> 0xc000281020, 0xc000281080)\n\t/go/pkg/mod/ >>>> github.com/prometheus/[email protected]/prometheus/registry.go:463 >>>> <http://github.com/prometheus/[email protected]/prometheus/registry.go:463> >>>> +0x2b\ncreated by >>>> github.com/prometheus/client_golang/prometheus.(*Registry).Gather\n\t/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/registry.go:462 >>>> <http://github.com/prometheus/client_golang/prometheus.(*Registry).Gather%5Cn%5Ct/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/registry.go:462> >>>> +0x60d\n\ngoroutine 201493 [select]:\ >>>> ngithub.com/prometheus/client_golang/prometheus.(*Registry).Gather(0xc00041a2d0, >>>> 0x0, 0x0, 0x0, 0x0, 0x0)\n\t/go/pkg/mod/ >>>> github.com/prometheus/[email protected]/prometheus/registry.go:513 >>>> <http://github.com/prometheus/[email protected]/prometheus/registry.go:513> >>>> +0xbbc\ >>>> ngithub.com/prometheus/client_golang/prometheus/promhttp.HandlerFor.func1(0xb3f0a0, >>>> 0xc0004c41c0, 0xc00022a000)\n\t/go/pkg/mod/ >>>> github.com/prometheus/[email protected]/prometheus/promhttp/http.go:126 >>>> <http://github.com/prometheus/[email protected]/prometheus/promhttp/http.go:126> >>>> +0x99\nnet/http.HandlerFunc.ServeHTTP(0xc0004edc70, 0xb3f0a0, 0xc0004c41c0, >>>> 0xc00022a000)\n\t/usr/local/go/src/net/http/server.go:2042 >>>> +0x44\nmain.handler(0xb3f0a0, 0xc0004c41c0, 0xc00022a000, 0xb34f40, >>>> 0xc000286420)\n\t/app/main.go:110 +0x6d7\nmain.main.func2(0xb3f0a0, >>>> 0xc0004c41c0, 0xc00022a000)\n\t/app/main.go:201 >>>> +0x51\nnet/http.HandlerFunc.ServeHTTP(0xc0004a82a0, 0xb3f0a0, 0xc0004c41c0, >>>> 0xc00022a000)\n\t/usr/local/go/src/net/http/server.go:2042 >>>> +0x44\nnet/http.(*ServeMux).ServeHTTP(0xea0040, 0xb3f0a0, 0xc0004c41c0, >>>> 0xc00022a000)\n\t/usr/local/go/src/net/http/server.go:2417 >>>> +0x1ad\nnet/http.serverHandler.ServeHTTP(0xc0004c4000, 0xb3f0a0, >>>> 0xc0004c41c0, 0xc00022a000)\n\t/usr/local/go/src/net/http/server.go:2843 >>>> +0xa3\nnet/http.(*conn).serve(0xc00024c640, 0xb40660, >>>> 0xc0000ca100)\n\t/usr/local/go/src/net/http/server.go:1925 +0x8ad\ncreated >>>> by net/http.(*Server).Serve\n\t/usr/local/go/src/net/http/server.go:2969 >>>> +0x36c\n\ngoroutine 358543 [IO >>>> wait]:\ninternal/poll.runtime_pollWait(0x7f0f18245368, 0x72, >>>> 0xb36220)\n\t/usr/local/go/src/runtime/netpoll.go:222 >>>> +0x55\ninternal/poll.(*pollDesc).wait(0xc000290298, 0x72, 0xb36200, >>>> 0xe444f0, 0x0)\n\t/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 >>>> +0x45\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/local/go/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000290280, >>>> 0xc000ecf5a1, 0x1, 0x1, 0x0, 0x0, >>>> 0x0)\n\t/usr/local/go/src/internal/poll/fd_unix.go:159 >>>> +0x1a5\nnet.(*netFD).Read(0xc000290280, 0xc000ecf5a1, 0x1, 0x1, >>>> 0xc0000ca1d8, 0xc000856f68, >>>> 0x4774bc)\n\t/usr/local/go/src/net/fd_posix.go:55 >>>> +0x4f\nnet.(*conn).Read(0xc00000e040, 0xc000ecf5a1, 0x1, 0x1, 0x0, 0x0, >>>> 0x0)\n\t/usr/local/go/src/net/net.go:182 >>>> +0x8e\nnet/http.(*connReader).backgroundRead(0xc000ecf590)\n\t/usr/local/go/src/net/http/server.go:690 >>>> +0x58\ncreated by >>>> net/http.(*connReader).startBackgroundRead\n\t/usr/local/go/src/net/http/server.go:686 >>>> +0xd5\n\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000" >>>> ``` >>>> >>>> Any idea, why this errors out? Other devices, which cannot be scraped >>>> are not producing these stack traces. >>>> >>>> Thanks a lot and regards, >>>> >>>> Olaf >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Prometheus Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/prometheus-users/7956644c-a2ce-4773-b9e4-b7a197bdcfdan%40googlegroups.com >>>> <https://groups.google.com/d/msgid/prometheus-users/7956644c-a2ce-4773-b9e4-b7a197bdcfdan%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/3f001b5c-a319-4ccb-a390-57e94bf09233n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/3f001b5c-a319-4ccb-a390-57e94bf09233n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmoCt18kMTssD-9SgRd88OkO5URnYkK-8wNSuWVmu9c75g%40mail.gmail.com.
