Multi-vendor (async) gRPC dial-out collector

[Salvatore Cuzzilla]

Hi Everyone,

during the last EuroBSDCon I had a very interesting talking with some of
you regarding subjects like NETCONF/YANG and Network Telemetry & Automation
in general. 


Since I felt there was some interest around these topics I'd like to
share with you a Project I've been working on during the last months.

https://github.com/scuzzilla/mdt-dialout-collector

It's based on gRPC & it can be utilized to collect network data streams
from your devices.

A docker container based on debian (stable) is already available for a
quick try. 


docker pull scuzzilla/mdt-dialout-collector:latest

I'm currently planning to test it on OpenBSD using vmm.  



---
:wq,
Salvatore.

RPKI Lab setup (rpki-client)

[Salvatore Cuzzilla]

Dear All,

I wrote a small article regarding setting up a Lab environment using 
rpki-client to be able to test RPKI against different vendors (eventually 
including OpenBGP).

I thought it might be of your interest: 
https://medium.com/@salvatore.cuzzilla/rpki-my-lab-environment-b23804a278c0

Of course your feedback is most then welcome!


Regards,
Salvatore.

Re: compiling pmacct on obsd6.8

[Salvatore Cuzzilla]

Hi Folks,

small update:
I was recently in touch with the pmacct's Founder (Paolo Lucente) thanks to him 
we can now compile flawlessly the latest version of pmacct without introducing 
additional dependencies/external patches.


Regards,
Salvatore. 

---

January 12, 2021 3:08 PM, "Salvatore Cuzzilla"  wrote:

> I compiled the latest release (v1.7.5) using a recent version of libpcap.
> enough to add "--with-pcap-libs=/path/to/libpcap-1.x.x 
> --with-pcap-includes=/path/to/libpcap-1.x.x"
> to the ./configure. 
> 
> January 12, 2021 1:46 PM, "Stuart Henderson"  wrote:
> 
>> On 2021-01-12, Kapetanakis Giannis  wrote:
>> 
>>> On 12/01/2021 10:25, Stuart Henderson wrote:
>> 
>> On 2021-01-12, Masato Asou  wrote:
>>> Hi,
>>> 
>>> From: Salvatore Cuzzilla 
>>> Date: Mon, 11 Jan 2021 17:40:21 +0100
>>> 
>>> I'm having some troubles with compiling the latest version of pmacct
>>> (https://github.com/pmacct/pmacct) on obsd6.8 .
>>> 
>>> I just wanted to know if someone was successful with it ...
>>> 
>>> You can use /usr/ports/net/pmacct.
>> 
>> That is not the latest version; OpenBSD's libpcap is too old for newer
>> pmacct which uses PCAP_NETMASK_UNKNOWN (added in 1.1.0, released 2010).
>>> pkg version runs fine for me on 6.8
>>> 
>>> # pkg_info pmacct
>>> Information for inst:pmacct-1.7.1
>>> 
>>> G
>> 
>> That is a bit old and missing a lot of useful/interesting things in
>> the newer versions, especially on the BGP side.

Re: compiling pmacct on obsd6.8

[Salvatore Cuzzilla]

I compiled the latest release (v1.7.5) using a recent version of libpcap.
enough to add "--with-pcap-libs=/path/to/libpcap-1.x.x 
--with-pcap-includes=/path/to/libpcap-1.x.x" to the ./configure. 



January 12, 2021 1:46 PM, "Stuart Henderson"  wrote:

> On 2021-01-12, Kapetanakis Giannis  wrote:
> 
>> On 12/01/2021 10:25, Stuart Henderson wrote:
> 
> On 2021-01-12, Masato Asou  wrote:
>> Hi,
>> 
>> From: Salvatore Cuzzilla 
>> Date: Mon, 11 Jan 2021 17:40:21 +0100
>> 
>> I'm having some troubles with compiling the latest version of pmacct
>> (https://github.com/pmacct/pmacct) on obsd6.8 .
>> 
>> I just wanted to know if someone was successful with it ...
>> 
>> You can use /usr/ports/net/pmacct.
> 
> That is not the latest version; OpenBSD's libpcap is too old for newer
> pmacct which uses PCAP_NETMASK_UNKNOWN (added in 1.1.0, released 2010).
>> pkg version runs fine for me on 6.8
>> 
>> # pkg_info pmacct
>> Information for inst:pmacct-1.7.1
>> 
>> G
> 
> That is a bit old and missing a lot of useful/interesting things in
> the newer versions, especially on the BGP side.

Re: compiling pmacct on obsd6.8

[Salvatore Cuzzilla]

thanks for the advice, it helped.


January 12, 2021 5:25 AM, "Masato Asou"  wrote:

> Hi,
> 
> From: Salvatore Cuzzilla 
> Date: Mon, 11 Jan 2021 17:40:21 +0100
> 
>> I'm having some troubles with compiling the latest version of pmacct
>> (https://github.com/pmacct/pmacct) on obsd6.8 .
>> 
>> I just wanted to know if someone was successful with it ...
> 
> You can use /usr/ports/net/pmacct.
> --
> ASOU Masato

compiling pmacct on obsd6.8

[Salvatore Cuzzilla]

Hi Guys,

I'm having some troubles with compiling the latest version of pmacct
(https://github.com/pmacct/pmacct) on obsd6.8 .

I just wanted to know if someone was successful with it ...


---
:wq,
Salvatore.

Wireguard - VPN up after reboot

[Salvatore Cuzzilla]

Hi Everyone,

I'm happily using 'Wireguard' to setup few VPNs.
I store the required configuration within /etc/hostname.wg0 & I startup the 
tunnel with 'doas sh
/etc/netstart wg0'.

Everything is working like expected. 
However, upon system reload the connectivity is lost.
The wg0 interface comes up but the tunnel stays in a sort of 'waiting'
state.

The only way I figure out to bring it up is either re-launching 'doas sh 
/etc/netstart wg0' or
pinging the tunnel default gateway.

Is there any decent/clean way to avoid manual intervention?

---
:wq,
Salvatore.

Re: base LoC & committers

[Salvatore Cuzzilla]

do you know if it's possible to see some statistics about the
committers? like for example number of commits per committer. 

On Tue, 2020-12-08 at 15:53 +0100, Benjamin Baier wrote:
> On Tue, 08 Dec 2020 13:49:13 +0100
> Salvatore Cuzzilla  wrote:
> 
> > Hi Everyone,
> > 
> > just out of curiosity, I was asking myself:
> > 
> > - approx how many LoC do we have in *base*?
> > - & how many committers are actually contributing to it?
> > 
> > when I think about some other OS with a kernel of almost 30M LoC &
> > over
> > 5k committers I go insane :)  
> > 
> > 
> > Regards,
> > Salvatore.
> 
> $ cloc /usr/src
>   111439 text files.
>85841 unique files.  
>55120 files ignored.
> 
> github.com/AlDanial/cloc v 1.86  T=254.29 s (229.3 files/s, 94467.6
> lines/s)
> ---
> 
> Language files  blankcomm
> ent   code
> ---
> 
> C1741212941481491
> 3937181673
> C/C++
> Header 14902 4933731021729   
>  4255540
> C++  10637 483624 511
> 8112771795
> Perl  4309 169414 228
> 936 956256
> Bourne
> Shell  1263  57662  69942
>  434428
> Markdown   279  47833
>   0 407365
> PO
> File129 141599 190451
>  319672
> Python1461  35581  35
> 610 134779
> HTML   259  17553
> 993 128449
> Assembly   969  21343  56
> 839 117720
> yacc93  14004   8
> 880 108162
> reStructuredText   775  49070  43
> 308 106806
> Expect 460  14443  21
> 700  74931
> make  2459  15471   8
> 987  68516
> Windows Module
> Definition  200   6600  3  49
> 202
> m4 177   5669   3
> 351  48578
> CMake  882   5106   3
> 729  36458
> ASP.NET  2   1013
>  18  24717
> TeX 29   3094  12
> 237  21764
> Pascal  58   3289  16
> 255  13924
> Scheme  95   1438
> 146  12907
> XML108828
> 396  10910
> lex 35   1714   1
> 908  10441
> awk 57686   1
> 607   8210
> SWIG67   2752
> 508   7668
> Fortran
> 77 183893   2886 
>   7495
> Oracle
> PL/SQL4180  1
>6945
> Go  26908
> 733   6507
> Objective
> C++   23   1097840   
> 6332
> Objective
> C211   1639629   
> 6041
> YAML   100 75
>  60   5954
> OCaml   59   1366   2
> 512   4083
> Fortran
> 90  73264818 
>   3457
> Korn
> Shell  83900   1118  
>  3381
> JSON41  1
>   0   2651
> SQL  5 77
>  38   2343
> sed 46221
> 593   1848
> CSS   

base LoC & committers

[Salvatore Cuzzilla]

Hi Everyone,

just out of curiosity, I was asking myself:

- approx how many LoC do we have in *base*?
- & how many committers are actually contributing to it?

when I think about some other OS with a kernel of almost 30M LoC & over
5k committers I go insane :)  


Regards,
Salvatore.

Re: obsd 6.7 - TOR relay (non-exit) & /var folder

[Salvatore Cuzzilla]

the issue is temporary “solved":

03:42:36 -ksh ToTo@APU2c4 ~ $ doas cat /etc/tor/torrc | egrep "^Log "
Log debug file /dev/null
Log info file /dev/null
Log notice file /dev/null

it’s confirmed that something is not going well with the logs handling ...



On 25 Jun 2020, at 15:39, Stuart Henderson  wrote:

On 2020/06/25 14:59, Salvatore Cuzzilla wrote:
> 
> Unfortunately the only think i know for sure is that the /var folder is
> constantly loosing free space & When i restart tor it gets back to
> normal. I can't (I don't know how to) figure out the involved files ...
> 
> "du" is not really helping nor "fstat"  ... Is there anything else
> i could test?

du won't show size of an unlinked file.

fstat won't show filenames but will show inode numbes. If it is from a
file that existed at startup and was then moved away, you could capture
inode numbers of all files on the filesystem when starting (find /var
-ls, the first number is the inode number), then compare with the INUM
column in fstat.

Or, if you change logs to syslog, and that fixes the problem, you have
your answer...


> On 25.06.2020 09:29, Stuart Henderson wrote:
>> On 2020-06-24, Salvatore Cuzzilla  wrote:
>>> After few attempts, I can't still don't understand what's going on
>>> it seems that the only way to free up the /var folder is to restart the
>>> tor's daemon.
>>> 
>>> "pkill -HUP -u _tor -U _tor -x tor" didn't help ...
>>> 
>>> Other ideas?
>> 
>> Did you figure out what files are involved?
>> 
>> If it's logs, use syslog instead.
>> 
> 
> ---
> :wq,
> Salvatore.

Re: obsd 6.7 - TOR relay (non-exit) & /var folder

[Salvatore Cuzzilla]

Unfortunately the only think i know for sure is that the /var folder is
constantly loosing free space & When i restart tor it gets back to
normal. I can't (I don't know how to) figure out the involved files ... 


"du" is not really helping nor "fstat"  ... Is there anything else
i could test?

On 25.06.2020 09:29, Stuart Henderson wrote:

On 2020-06-24, Salvatore Cuzzilla  wrote:

After few attempts, I can't still don't understand what's going on
it seems that the only way to free up the /var folder is to restart the
tor's daemon.

"pkill -HUP -u _tor -U _tor -x tor" didn't help ...

Other ideas?


Did you figure out what files are involved?

If it's logs, use syslog instead.



---
:wq,
Salvatore.

Re: obsd 6.7 - TOR relay (non-exit) & /var folder

[Salvatore Cuzzilla]

After few attempts, I can't still don't understand what's going on
it seems that the only way to free up the /var folder is to restart the
tor's daemon.

"pkill -HUP -u _tor -U _tor -x tor" didn't help ...

Other ideas?

On 23.06.2020 11:50, Salvatore Cuzzilla wrote:

Hi Gabriel,

thanks for the hint!

I actually use to "rcctl reload tor" to rotate the logs.
I now switched to "pkill -HUP -u _tor -U _tor -x tor" let's see if it's helping!


Regards,
Salvatore.


June 23, 2020 12:53 PM, "Salvatore Cuzzilla"  wrote:


Hi Folks,

I’m running a TOR node on my [APU2c4 (SSD) + OBSD 6.7]

somehow the TOR process is polluting my /var folder until, after few days, it’s 
fulfilled (~6G).
In the beginning I thought that it was related to the daemon's logs, something 
misconfigured within
newsyslog.conf ... it’s not!

the funny thing is that, as soon as shut the daemon the /var folder is free-up 
back again…

-
12:46:44 -ksh root@APU2c4 /var/tor/diff-cache # df -h | grep /var
/dev/sd0e 6.3G 1.7G 4.4G 28% /var

12:46:55 -ksh root@APU2c4 /var/tor/diff-cache # rcctl stop tor
tor(ok)

12:48:00 -ksh root@APU2c4 /var/tor/diff-cache # df -h | grep /var
/dev/sd0e 6.3G 327M 5.7G 5% /var
12:48:00 -ksh root@APU2c4 /var/tor/diff-cache
-

I’m a bit lost, from where should I start?

Regards,
Salvatore.


---
:wq,
Salvatore.

Re: obsd 6.7 - TOR relay (non-exit) & /var folder

[Salvatore Cuzzilla]

Hi Gabriel,

thanks for the hint!

I actually use to "rcctl reload tor" to rotate the logs.
I now switched to "pkill -HUP -u _tor -U _tor -x tor" let's see if it's helping!


Regards,
Salvatore.


June 23, 2020 12:53 PM, "Salvatore Cuzzilla"  wrote:

> Hi Folks,
> 
> I’m running a TOR node on my [APU2c4 (SSD) + OBSD 6.7]
> 
> somehow the TOR process is polluting my /var folder until, after few days, 
> it’s fulfilled (~6G).
> In the beginning I thought that it was related to the daemon's logs, 
> something misconfigured within
> newsyslog.conf ... it’s not!
> 
> the funny thing is that, as soon as shut the daemon the /var folder is 
> free-up back again…
> 
> -
> 12:46:44 -ksh root@APU2c4 /var/tor/diff-cache # df -h | grep /var
> /dev/sd0e 6.3G 1.7G 4.4G 28% /var
> 
> 12:46:55 -ksh root@APU2c4 /var/tor/diff-cache # rcctl stop tor
> tor(ok)
> 
> 12:48:00 -ksh root@APU2c4 /var/tor/diff-cache # df -h | grep /var
> /dev/sd0e 6.3G 327M 5.7G 5% /var
> 12:48:00 -ksh root@APU2c4 /var/tor/diff-cache 
> -
> 
> I’m a bit lost, from where should I start?
> 
> Regards,
> Salvatore.

obsd 6.7 - TOR relay (non-exit) & /var folder

[Salvatore Cuzzilla]

Hi Folks,

I’m running a TOR node on my [APU2c4 (SSD) + OBSD 6.7]

somehow the TOR process is polluting my /var folder until, after few days, it’s 
fulfilled (~6G).
In the beginning I thought that it was related to the daemon's logs, something 
misconfigured within newsyslog.conf ... it’s not!

the funny thing is that, as soon as shut the daemon the /var folder is free-up 
back again…

-
12:46:44 -ksh root@APU2c4 /var/tor/diff-cache # df -h | grep /var
/dev/sd0e  6.3G1.7G4.4G28%/var

12:46:55 -ksh root@APU2c4 /var/tor/diff-cache # rcctl stop tor
tor(ok)

12:48:00 -ksh root@APU2c4 /var/tor/diff-cache # df -h | grep /var
/dev/sd0e  6.3G327M5.7G 5%/var
12:48:00 -ksh root@APU2c4 /var/tor/diff-cache 
-

I’m a bit lost, from where should I start?


Regards,
Salvatore.

discard me

[Salvatore Cuzzilla]

discard me

openbsd.org down?

[Salvatore Cuzzilla]

Can’t reach openbsd.org  - planned maintenance?

Re: bridge, vether & dhcpd

[Salvatore Cuzzilla]

nope, the L2 if(s) (including bridge) are running only with option ‘up’ within 
hostname.if files
& all the other L3 ifs are with IP statically assigned 

> On 17 Mar 2020, at 09:44, Stefan Sperling  wrote:
> 
> On Tue, Mar 17, 2020 at 08:24:34AM +0100, Salvatore Cuzzilla wrote:
>> Dear all,
>> 
>> is someone using a setup with multiple layer 2 interfaces & a single vether 
>> IP interface (layer 3) bundled all together in a bridge?
>> Well, i’m using this setup too and almost everything is working like 
>> expected. 
>> 
>> However, 
>> I have a couple of hosts  connected to the L2 interfaces & i would like them 
>> to dynamically get an IP (dhcpd instance already up & running)
>> atm, this is not working. I thought about PF , but probably it’s not the 
>> issue …
>> 
>> any advice? configuration examples i can go through?
>> 
>> 
> 
> Is dhclient also running? If so, try to stop dhclient and see if
> it works then.

bridge, vether & dhcpd

[Salvatore Cuzzilla]

Dear all,

is someone using a setup with multiple layer 2 interfaces & a single vether IP 
interface (layer 3) bundled all together in a bridge?
Well, i’m using this setup too and almost everything is working like expected. 

However, 
I have a couple of hosts  connected to the L2 interfaces & i would like them to 
dynamically get an IP (dhcpd instance already up & running)
atm, this is not working. I thought about PF , but probably it’s not the issue …

any advice? configuration examples i can go through?

Re: relayd redirect not working

[Salvatore Cuzzilla]

Ciao Dave,

I'm also playing with relayd as a L7 gateway and as far as I can see from your
config there is no CA and key configured. In order for HTTPS to work relayd
needs to be able to do TLS inspection and of course you should redirect all
your https traffic to port 8443 (using PF for example). If you check the
pf.conf man page under both the sections RELAYS and Examples you should be
able to find a lot of good hints.


Regards,
Salvatore.

> On 12 Mar 2017, at 06:48, Dave Cohen  wrote:
>
> I'm struggling to figure out why network traffic is not making it to a
service I'm running.
>
> What I'm trying to do is serve http and https from a non-standard server.
(Called `caddy`, if you're curious).  I want to run this thing as non-root
user.  I'm not aware of any way to have the non-root user open ports 80 or
443.  Which is great, so long as I can get traffic to those port to be
redirected to my server, which I have listening on 8080 and 8443
respectively.
>
> I prefer the TLS traffic to 443 terminate at my server on 8443.  And I've
been trying to do this with relayd redirects.
>
> Here's what I've tried, in /etc/relayd.conf:
>
> table  {127.0.0.1}
>
> redirect "https" {
>listen on 0.0.0.0 port 443
>forward to  port 8443 check icmp
> }
>
> redirect "http" {
>listen on 0.0.0.0 port 80
>forward to  port 8080 check icmp
> }
>
>
>
> With that configuration, traffic on port 80 works as expected, my server
responds.  But https traffic on port 443, as far as I can tell, never makes it
to my server listening on port 8443.  I'm not sure why the two redirects which
are so similar do not behave the same way.
>
> Possibly, the https redirect needs to use `route to` rather than `forward
to`.  When I tried that, relayd errors with "missing interface to route to".
I couldn't figure out reading `man relayd.conf` how to get past that error.
If anyone has a working example, please share.
>
> My questions for this group are (a) is there a smarter way than what I'm
trying?  And if not (b) what am I doing wrong?  Thanks in advance for any
info!
>
> -Dave