Re: experience setting up a low memory machine
On Wed, Mar 11, 2020 at 6:47 PM Jordan Geoghegan wrote: > > > > On 2020-03-11 00:13, Stuart Longland wrote: > > On 15/2/20 6:43 pm, Dumitru Moldovan wrote: > >> [SNIP] > > [SNIP] > > > > Sometimes it's better to realise when something has past its prime. > > A year or two ago I had OpenBSD working on my iBook with 64MB of RAM, > even got FVWM working on it. For fun and testing purposes, I ran some > small OpenBSD virtual machines with 64MB RAM as well. A few years back I > got OpenBSD to boot with 32MB, but it wasn't particularly usable. I've > found 128MB to be usable for basic terminal work, but you're definitely > correct about 256MB being the bare minimum for anything fancy or GUI > related. > > At work I run OpenBSD 6.1 in a VM for Request Tracker. It has 512MB RAM and it seems that may very well be overkill. At previous jobs I can ManageEngine ServiceDesk Plus and even in Linux you needed 2GB minimum just for it to get out of bed. I plan on rebuilding it with 6.6 (can't update RT because packages are too old in 6.1) and might run it on 256MB for shits and giggles. -- Aaron Mason - Programmer, open source addict I've taken my software vows - for beta or for worse
Re: pf-badhost-0.3 released
On 2020-03-11 12:41, Anders Andersson wrote: On Tue, Mar 10, 2020 at 10:53 PM Jordan Geoghegan wrote: pf-badhost and unbound-adblock are both now at version 0.3, released earlier today. Links to the scripts can be found here: www.geoghegan.ca/pfbadhost.html www.geoghegan.ca/unbound-adblock.html Thanks, this looks very interesting! But maybe you can help answering a question that popped up when I read your page about pf-badhost. You mention that "Subnet aggregation is used to take the address list and "aggregate" the addresses into the smallest possible representation using CIDR blocks.", but I was under the assumption that pf already did this for its tables to speed up lookups. Is there anything preventing the aggregation code to run on every pf table modification? Assuming an already sorted list, it shouldn't take long to merge a new entry. Perhaps I've missed some use of pf tables that makes this impossible or not applicable in the general case. Hi Anders, I am by no means an expert on the nuts and bolts of pf, but I do know that pf stores table data in a radix tree / radix table. By their nature, radix trees ignore exact duplicates, but I'm not exactly sure how they handle the partial overlapping of ranges. This article gives an easy to follow cursory overview of raddix trees if you're interested: https://blog.sqreen.com/demystifying-radix-trees/ As far as I understand, pf makes no modifications to the contents of your tables, all it does is parse the list to confirm the addresses and/or CIDR blocks are valid. When it's looking for matches within ranges, it will look for the most specific match available. For example, if you have a list containing an overlap: ... 192.168.0.0/16 192.168.1.0/22 ... When a packet from 192.168.1.5 arrives and is processed by a rule referencing this table, it will match with 192.168.1.0/22. Even though both entries are valid and match the packet, the /22 is more specific, and thus the one which matches closest. pf may do some magic optimizations under the hood that I'm unaware of, but at the end of the day, it does not modify the actual contents of your table. The use I've found in the subnet aggregation function has been mostly for the purpose of keeping the list clean and tidy. I have a few installations where I have all the lists enabled, including the use of the GeoIP country blacklisting function. On these installations, subnet aggregation can reduce the /etc/pf-badhost.txt file from ~60,000 lines down to ~40,000 lines. For example, when blocking China's netblocks (which pulls an aggregated list of all addresses assigned to China by APNIC, and thus uses massive CIDR blocks of /10's etc), if any addresses from any of the other blocklists come from China, they will be removed from the list as they are already covered by the CIDR block info from APNIC. I run pf-badhost on a bunch of Edgerouter Lites, and I've found them to run better when the lists are tidy. With regards to pf performing aggregation on all tables automatically, it wouldn't make sense to run the full subnet aggregation calculations for every table load or insertion/removal, as it can be quite CPU intensive. It takes less than a second to load the table on a $5 Vultr VPS, it takes 20-70 seconds to run the subnet aggregation (depending on which lists are enabled). On my Edgerouter Pro with all the lists enabled, it takes ~6 minutes. On my Edgerouter Lite it takes ~15 minutes to run (over 2 hours when using the built in Perl-based aggregator). I just run the aggregation function with nice and let it do its thing, its being called by cron in the wee hours, so I'm fine just letting it chug along. Regards, Jordan
Re: Time jumping forward issue under OpenBSD 6.6 VMM
Hi Ian, I think this is a common problem, and has been an issue from time to time on KVM machines also, you can try an alternate counter and see how you get on, if you check out Mischa Peters talk on VMM he has some workarounds that he has deployed in production. but for an NTP server the accuracy of the clock may cause you too much problems.. as I said we had issues before with KVM and the timecounter hardware... but the newer timecounters in recent versions of OpenBSD have worked really well for us. I hope this helps, Tom Smyth On Tue, 3 Mar 2020 at 17:05, Ian Gregory wrote: > > On Tue, 3 Mar 2020 at 15:47, mabi wrote: > > It looks like there is a time issue on that VM although I am running the > > default ntpd of OpenBSD 6.6 and I have added the following parameter into > > my /etc/sysctl.conf on that VM: > > > > kern.timecounter.hardware=tsc > > I've had similar issues with timekeeping within guests of VMM, > although there are improvements in -current with the pvclock time > source. Since the fix below I now see occasional instances of the > clock stepping by a few whole seconds (typically less than 8s) but > it's much less frequent and the magnitude is within the bounds of what > ntpd can correct. > > See > http://openbsd-archive.7691.n7.nabble.com/pvclock-stability-tp376946p377922.html > for some backstory > > (aside: I see similar small occasional clock jumps of an integer > number of seconds on OpenBSD-6.6 guests using tsc running on a VMware > ESXi host) > > Regards > Ian > -- Kindest regards, Tom Smyth.
Re: pf-badhost-0.3 released
On Tue, Mar 10, 2020 at 10:53 PM Jordan Geoghegan wrote: > > pf-badhost and unbound-adblock are both now at version 0.3, released > earlier today. > > Links to the scripts can be found here: > > www.geoghegan.ca/pfbadhost.html > www.geoghegan.ca/unbound-adblock.html Thanks, this looks very interesting! But maybe you can help answering a question that popped up when I read your page about pf-badhost. You mention that "Subnet aggregation is used to take the address list and "aggregate" the addresses into the smallest possible representation using CIDR blocks.", but I was under the assumption that pf already did this for its tables to speed up lookups. Is there anything preventing the aggregation code to run on every pf table modification? Assuming an already sorted list, it shouldn't take long to merge a new entry. Perhaps I've missed some use of pf tables that makes this impossible or not applicable in the general case.
Re: alacritty build
On 03-11 16:51, Wayn0 wrote: > On Wed, 11 Mar 2020 at 06:21, Wayne Oliver wrote: > > On 2020/03/10 21:09, Justin Muir wrote: > > > Just wishing to give alacritty a go so I git'd the source and did a > cargo > > > build. > > > I get this error after a while: > > > LLVM ERROR: out of memory > > > I tried increasing the dataset-cur and -max to 4000M in login.conf, > didn't > > > seem to work. > > I managed to build this on my laptop about a week ago. I'll happily > > check my settings for you when I get home this evening (SAST) if you > > don't come right before then. > part of the staff group, datasizr-cur=4096M max infinity > during build on my dual core i7 4xxx it took 17m to build RAM usage really > going over 800mb > 35mb binary Just curious: what was the size after running "strip" on the binary, or did you? -- Luke Call My thoughts: http://lukecall.net (updated 2020-02-18)
Re: OpenBSD 6.6-current shutter about one time every 1-3 seconds
I use significantly slower hardware than yours, AMD SOC with ordinary 2.5" 7200rpm HDD (bioctl encrypted). The same HDD has been installed on the same platform when OpenBSD was 6.5-current with the same encryption level. Cyrus imapd implementations don't present in my setup. Dovecot+opensmtpd+PostgresSQL are working in production and I don't see any activity of them which can cause stutters. I think it can be USB2/3 issues, but I don't know how to diagnose it. Hope somebody give an advice regarding diagnose on system level of the 6.6 itself and 3rd party software behavior. Peter, can you share which software you started in /etc/rc.conf.local by pkg_scripts="imapd..." I'll try to find some correlation. Martin ‐‐‐ Original Message ‐‐‐ On Wednesday, March 11, 2020 5:40 PM, Peter J. Philipp wrote: > On Wed, Mar 11, 2020 at 05:28:11PM +, Martin wrote: > > > Hello list, > > After upgrade from 6.5 to 6.6-current amd64 (all the latest patches > > installed) system stutters. It affects on all visible and background > > activity. For instance, when typing USB keyboard skip symbols, disk write > > operations bumble as well, voip RTP traffic interrupts for less then a > > second. > > What the best way to determine the cause of this behavior? Currently > > looking some ways to diagnose the problem reason. > > Martin > > Hi Martin, > > I get that too, it started when I moved cyrus imapd to my local system a year > or more ago. I've learned to live with it. To elaborate, something cyrus > imapd does is when a mail comes in via fetchmail that mplayer which plays > flac's or streams from an icecast starts stuttering. The combination of > cyrus, thunderbird, and fetchmail may be to blame too, dunno. It may be > disk related? I have a Samsung SSD. My workstation is a 2014 Xeon E3-1275 > with 32 GB RAM. > > Regards, > -peter
Re: OpenBSD 6.6-current shutter about one time every 1-3 seconds
On Wed, Mar 11, 2020 at 06:12:44PM +, Martin wrote: > Peter, can you share which software you started in /etc/rc.conf.local > by > pkg_scripts="imapd..." > > I'll try to find some correlation. > > Martin pkg_scripts="isc_named cyrus_imapd saslauthd" along with apmd -A, vmd, sndiod -f rsnd/0 -f rsnd/1, portmap, nfsd, mountd, unwind, rad, tftpd, xenodm, dhcpd It's just the workstation I power up every day when I'm home. Hope that helps, -peter
OpenBSD 6.6-current shutter about one time every 1-3 seconds
Hello list, After upgrade from 6.5 to 6.6-current amd64 (all the latest patches installed) system stutters. It affects on all visible and background activity. For instance, when typing USB keyboard skip symbols, disk write operations bumble as well, voip RTP traffic interrupts for less then a second. What the best way to determine the cause of this behavior? Currently looking some ways to diagnose the problem reason. Martin
Re: OpenBSD 6.6-current shutter about one time every 1-3 seconds
On Wed, Mar 11, 2020 at 05:28:11PM +, Martin wrote: > Hello list, > > After upgrade from 6.5 to 6.6-current amd64 (all the latest patches > installed) system stutters. It affects on all visible and background > activity. For instance, when typing USB keyboard skip symbols, disk write > operations bumble as well, voip RTP traffic interrupts for less then a second. > > What the best way to determine the cause of this behavior? Currently looking > some ways to diagnose the problem reason. > > Martin Hi Martin, I get that too, it started when I moved cyrus imapd to my local system a year or more ago. I've learned to live with it. To elaborate, something cyrus imapd does is when a mail comes in via fetchmail that mplayer which plays flac's or streams from an icecast starts stuttering. The combination of cyrus, thunderbird, and fetchmail may be to blame too, dunno. It may be disk related? I have a Samsung SSD. My workstation is a 2014 Xeon E3-1275 with 32 GB RAM. Regards, -peter
Re: pf-badhost-0.3 released
Hi Jordan, Thanks for the good work. Great solution to replace third-party adblockers addons in browsers. Blocked 100% ads in my tests. Regards, -- Fabio Martins > Hey folks, > > Last time I posted about this, I got a fair bit of interest and I've had > quite a few downloads and enquiries about pf-badhost, so I figured I'd > share here that I've updated the script. > > pf-badhost and unbound-adblock are both now at version 0.3, released > earlier today. > > I highly encourage anybody running an older version of these scripts to > update to the latest version, as I have made a number of significant > improvements to the security and robustness of the script. > > Links to the scripts can be found here: > > www.geoghegan.ca/pfbadhost.html > www.geoghegan.ca/unbound-adblock.html > > Regards, > Jordan > > >
Re: alacritty build
On Wed, 11 Mar 2020 at 06:21, Wayne Oliver wrote: > > On 2020/03/10 21:09, Justin Muir wrote: > > Just wishing to give alacritty a go so I git'd the source and did a cargo > > build. > > > > I get this error after a while: > > > > LLVM ERROR: out of memory > > error: Could not compile `alacritty` > > > > I tried increasing the dataset-cur and -max to 4000M in login.conf, didn't > > seem to work. > > > > Any ideas? > > I managed to build this on my laptop about a week ago. I'll happily > check my settings for you when I get home this evening (SAST) if you > don't come right before then. > > part of the staff group, datasizr-cur=4096M max infinity during build on my dual core i7 4xxx it took 17m to build RAM usage really going over 800mb 35mb binary not sure it's worth all of that -- Wayn0
Re: experience setting up a low memory machine
On 2020-03-11 00:13, Stuart Longland wrote: On 15/2/20 6:43 pm, Dumitru Moldovan wrote: Not really, about 21 years ago I was learning to get XFree86 working, to break free from the console on a desktop with 24MB of RAM. It's all relative… I can recall years ago experimenting with operating systems on old machines (even by that day's standards)… trying to get Slackware Linux running in 4MB RAM. Today, I'm trying to cram stuff into 32kB of RAM and thinking how 4MB and a MMU would be luxury! (Even RetroBSD needs 128kB.) I'm sure someone on here will tell me how they used to flip switches on a front panel to fill the whopping 128 bytes of RAM on their 8080 with machine code whilst dreaming of a punched tape loader and a few more DRAM chips. I would suggest any "modern" multi-tasking general purpose OS such as OpenBSD, Linux, etc, you're looking at a minimum of 256MB RAM. In the future, probably consider doubling that… and again. I have an old PII 300MHz laptop with 160MB RAM, and while it boots Linux just fine (I had it running Gentoo doing AX.25 packet radio stuff), firing up a web browser (Firefox) to check the weather is unusably slow. I could throw OpenBSD on it, but not sure that would really make much difference: kernel re-linking is going to be painful on that machine. Sometimes it's better to realise when something has past its prime. A year or two ago I had OpenBSD working on my iBook with 64MB of RAM, even got FVWM working on it. For fun and testing purposes, I ran some small OpenBSD virtual machines with 64MB RAM as well. A few years back I got OpenBSD to boot with 32MB, but it wasn't particularly usable. I've found 128MB to be usable for basic terminal work, but you're definitely correct about 256MB being the bare minimum for anything fancy or GUI related.
Re: experience setting up a low memory machine
On 15/2/20 6:43 pm, Dumitru Moldovan wrote: > Not really, about 21 years ago I was learning to get XFree86 working, > to break free from the console on a desktop with 24MB of RAM. It's all relative… I can recall years ago experimenting with operating systems on old machines (even by that day's standards)… trying to get Slackware Linux running in 4MB RAM. Today, I'm trying to cram stuff into 32kB of RAM and thinking how 4MB and a MMU would be luxury! (Even RetroBSD needs 128kB.) I'm sure someone on here will tell me how they used to flip switches on a front panel to fill the whopping 128 bytes of RAM on their 8080 with machine code whilst dreaming of a punched tape loader and a few more DRAM chips. I would suggest any "modern" multi-tasking general purpose OS such as OpenBSD, Linux, etc, you're looking at a minimum of 256MB RAM. In the future, probably consider doubling that… and again. I have an old PII 300MHz laptop with 160MB RAM, and while it boots Linux just fine (I had it running Gentoo doing AX.25 packet radio stuff), firing up a web browser (Firefox) to check the weather is unusably slow. I could throw OpenBSD on it, but not sure that would really make much difference: kernel re-linking is going to be painful on that machine. Sometimes it's better to realise when something has past its prime. -- Stuart Longland (aka Redhatter, VK4MSL) I haven't lost my mind... ...it's backed up on a tape somewhere.
