Re: mounting audio cd
On 5/31/24 15:46, Harald Arnesen wrote: MIZSEI Zoltán [31/05/2024 20.15]: Interestingly BeOS and Haiku lets you to mount an audio cd, it generates a vfs from the toc and shows the tracks as wav or flac (fixme), it does an automatic conversion behind the courtains if you copy a file from an audio cd. Linux also had such a thing in the past - I can't remember the name of the file-system. fuse(4) + part of a CD player + A Simple Matter of Programming
Re: OpenBSD alternative setup to ZFS on Linux or FreeBSD
On 2023-11-24, Crystal Kolipe wrote: At the end of last year, I did a comprehensive write-up about using blu-ray recordable on OpenBSD, and as part of that I checked around 100 BD-R discs that had been written about 10 years previously and verified as good at the time. Ten years laster, I found exactly ZERO bad discs. All data was readable from every single disc, (and returned the correct checksums). Do you have any data about blu-ray double layer lifetime? thanks!
Re: OpenBSD alternative setup to ZFS on Linux or FreeBSD
On 11/26/23 08:52, Stuart Henderson wrote: Anyone know whether USB BD-R drives are likely to work on OpenBSD? I've used several. XD08UMB-S works for reading - haven't tried writing yet. Earlier ones worked for reading and writing
Re: OpenBSD alternative setup to ZFS on Linux or FreeBSD
On 11/22/23 20:31, j...@bitminer.ca wrote For long-term storage, you have other risks to manage, not the simple technical risk of "will my portable-USB disk be readable in 2038?". Interfaces die - IDE interface cards? Even if you have one the ISA bus might not be available. Parallel SCSI, parallel PCI etc. Archiving data needs regular copy and verify to newer formats/media/attachments and multiple copies stored multiple places. I have files from 1982 which have survived many disasters. Of course, there's one storage medium verified to last for centuries. Good ink on rag paper stored dry. Papyrus is good for millenia. Not entirely a joke.
Re: mount softdep — does it improve the situation for unexpected shutdowns?
I'm going to blame softdep: A long time ago when modifying large numbers of files on a machine with limited memory suddenly: the system became completely unresponsive, the disk light was on solid the poor little disk was rattling continuously. That went on for minutes. The same job on the same machine without softdep ran slowly and with lots of moderate disk rattle but it completed normally. My guess is that softdep ran out of buffers and purged every one of them. All at once. Never enabled softdep after that. Geoff Steckel
Re: desire for journaled filesystem
On 9/5/23 15:41, Rudolf Leitgeb wrote:
On Tue, 2023-09-05 at 14:16 -0400, John Holland wrote:
So this gave me the list of the files with what they seem to be in
groups. I think a lot of them are browser cache, jpegs, pngsI
looked
at some of the gzipped ones and they were web pages and css files.
There are some that don't make sense, for instance #16251989 is
listed
as "ISO Media " and contains binary data and then some HTML.
So: great surprise, most of these lost+found files are browser cache
stuff, which was created just before the big crash. This is the kind of
stuff which hasn't been written to disk yet. It's extremely unlikely,
that you'll find a file in lost+found, which you haven't touched in
jours/days before the crash.
I would suggest, you ignore this lost+found directory for now, and only
if you can't find some important data you can search this directory
based on strings which should be in the missing file.
Maybe that's, why OpenBSD never cared about a journaling FS ...
What I've done to recover filenames, etc.
restore your backup somewhere
doas find -type f -exec cksum {} + > sums.restored
doas find -type f -exec cksum {} + > sums.damaged
munge with sed, awk, uniq & friends
-> list of undamaged files
-> list of possible undamaged files in orphan directories
-> list of new? files & clues to where they lived
then
file on interesting orphans
Not necessarily quick but as Mr. Leitgeb suggests this should give
a very strong clue as to which files are caches and thus ignorable and
where the orphaned files should live
There have been studies about typical lifetimes of files.
IIRC there's a glob of short-lived browser, object, temp etc. files
There's another glob of system, etc files updated regularly but
infrequently.
Most live for weeks to years.
A cron job which does hierarchical dumps daily, weekly, etc. and sent
offline might help.
Re: sndio and bit perfect playback
Signal processing which modifies the data stream -must- (almost) always dither. Without that it introduces -significant and audible- distortion. That has been standard practice in digital audio for more than 50 years. That is why sox dithers by default. Dithering is also known as noise shaping. Its purpose is to make the distortion introduced by signal processing as inaudible as possible. The distortion -cannot- be removed but it can be changed to broadband noise which is much less objectionable. OpenBSD is known for good engineering practice. Since the audio system is set to -force- users to accept whatever signal processing the system applies that processing should follow good practice as well. Other OSes allow unprivileged users to access raw audio devices and bypass any system processing. Users should be given that option. Geoff Steckel On 1/10/23 03:11, Jan Stary wrote: I don't get the point of your message explaning what dither is. My whole point was that one of the sources of the perceived difference between how sox resamples and hows aucat resamples might be dithering, as aucat does not differ (right?).
Re: sndio and bit perfect playback
The math goes back to Shannon and sampling theory: Any time you remove significant digits you remove information. One interpretation is that you introduce noise. Period. The math says so. The math says what the resulting power is. You have the option to determine where the noise goes. If you do nothing the noise is correlated with the input signal. You get what in audio terms is intermodulation distortion. In the simplest case this is harmonic distortion. You can spread the noise by applying digital transforms. You -cannot- remove it entirely. The easiest and most common method is dithering which spreads the power over the (audio) spectrum. That result can be white, pink etc. People are very sensitive to intermodulation distortion. Less sensitive to harmonic distortion. (People who like vacuum tubes and vinyl -like- second harmonic distortion = "rich sound" ) Very less sensitive to random noise. Even less to mostly-random noise outside of 1KHz-5KHz or so. Some of the standard minimal tests of audio equipment translated to the digital domain: Run FFTs with sufficient accuracy on the following (double precision at least for good results): (a) a precise sine wave (32 bits or better) truncated to 24 and 16 bits testing harmonic distortion (usually 1KHz) (b) two sines (usually 1KHz and 7KHz) testing intermodulation distortion (c) (a) and (b) resampled (96 to 48, 48 to 44.1, etc.) and resampled 24 to 16 testing purely digital distortion Decide for yourself if the results are significant for your use. If you read the sox(1) man page you'll notice that it computes in 32 bits and applies dithering -by default- when truncating to the output sample width. You can defeat it if you want and wish to accept the result.
Re: dhclient -d run0
My objection to dhcpleased is not whether the program does useful things. I'm sure it does "what it should". Adding this sentence to the dhcpleased man page would make it clear what it does beyond leasing the IP: "By default, it replaces the DNS server in /etc/resolv.conf and replaces the default route using information from the DHCP server." If it does anything else it's a bug in the documentation or the program. Examine many, many man pages for the traditional suite of Unix programs. They're contracts. Surprises in the .conf leave doubts about all the documentation and the program itself. Now I'll shut up. Geoff Steckel
Re: dhclient -d run0
On 12/21/22 09:05, Crystal Kolipe wrote: On Wed, Dec 21, 2022 at 01:39:47PM +, Rodrigo Readi wrote: The command "dhclient -d run0" with or without "-d" seems to demonize, always, and is till now completely silent. Is this new behaviour normal? How I get it the old way? You might want to look at the commit message for version 1.727 of dhclient.c: http://cvsweb.openbsd.org/src/sbin/dhclient/dhclient.c dhcpleased appears to be a useful & complete replacement for dhclient for most users Right now I run dhclient on my gateway machine. I cut off its fingers when it tries to e.g. mess with routes or resolv.conf. I disable dhcpleased because I don't know -exactly- what it tries to do. Replacing resolv.conf and configuring routes can be disabled in dhcpleased.conf. Are routes and resolv.conf the only things dhcpleased modifies beyond configuring the interface with the leased IP? For instance, pair of sentences in dhcpleased(8) to the effect: "dhcpleased inserts the DNS server information from the DHCP server at the top of the existing /etc/resolv.conf. It configures the default route via the interface being configured." Something like that is implied by using DHCP. Are other routes changed or deleted? Exactly what does the updated resolv.conf contain? DHCP can configure other services. Does dhcpleased do any of that? thanks Geoff Steckel
Re: scp doesn't work properly when the file name begins with a dash h
On 12/15/22 18:59, Mik J wrote: Hello, I have a file named like this -hh-6CP0_3Xf9nreW45XSBGrstMsovnIX6tTe45Enk4 and when I do a scp i have this output scp: unknown option -- h I feel like there's a limitation because "scp * destination" shouldn't output an error, * is considered as files not options What do you think about it ? In 1974, one said cp ./-hello foo\*hello Though using '*' in a filename was considered impolite. Students often tried to hide files from machine operators so there were often strange filenames including, for instance, backspaces. Geoff Steckel
IPv6: ifconfig & rad messages fight disabling routes
Short form: I'm using rad to update local machines with the IPv6 address prefix currently assigned by Verizon. It runs on my firewall/external router. The router advertisement destructively changes the route and interface of an address on that machine. dhcpcd gets /56 from Verizon fiber service. manually ifconfig interface1 inet6 ::9/56 netstat -rn -f inet6 shows :: and ::9 routed via interface1 everything works. rad message sent out from interface1: /56 valid 1000 preferred 1 netstat now shows routed via lo0 ::9 is still on interface1 That address is now (mostly) unusable Other addresses assigned to that interface work. Is this expected? Possible workarounds: blocking loopback of router advertisements (preferred) dhcp6 server for guest machines & fixed assignments for servers/test machine (inconvenient or worse) Monitor prefix for changes - hope that's not often I have to assume the mechanism which processes RAs follows the RFC and uses the link level address of the sender. I hypothesize it sees lo0 as the sender and Bad Things happen. ding:236$ doas ifconfig cnmac0 inet6 2600:4040:::7 prefixlen 56 ding:237$ netstat -rn -f inet6 2600:4040::::/56 2600:4040::xxx::7 UCn 0 0 - 4 cnmac0 2600:4040::::7 f0:9f:c2:cf:9e:e7 UHLl 0 0 - 1 cnmac0 --- rad message 22:06:10.869060 fe80::f29f:c2ff:fecf:9ee7 > ff02::1: icmp6: router advertisement (chlim=64, pref=medium, router_ltime=1800, reachable_time=0, retrans_time=0) (prefix info: LA valid_ltime=7200, preferred_ltime=7200, prefix=2600:4040::xxx::/56) (prefix info: LA valid_ltime=2592000, preferred_ltime=604800, prefix=fde3:3eec:4f3c:29dc::/64) (rdnss: lifetime=2000s, addr=fde3:3eec:4f3c:29dc::7)(dnssl: opt_len=3) [icmp6 cksum ok] (len 128, hlim 255) 2600:4040::xxx::/56 ::1 UGR 0 718 32768 56 lo0 2600:4040::xxx::7 f0:9f:c2:cf:9e:e7 UHLl 0 16 - 1 cnmac0 There's a off-by-4 in RAD display in tcpdump. Will post that later. I added the extra detail in that as well - will post if anyone things it's useful. geoff steckel
Re: sndio and bit perfect playback
On 10/14/22 05:21, Alexandre Ratchov wrote: On Thu, Oct 13, 2022 at 05:20:49PM -0400, Geoff Steckel wrote: If those don't work it's a (fixable) bug/not-yet-implemented. I've tried those settings with ambiguous results but not failure. My usb dacs don't have visible indicators & I don't have a USB protocol sniffer. Running audioctl during playback reveals the device sample rate. Running audioctl shows what the system thinks the device rate is. In my experience resampling quality in any particular implementation is not guaranteed and can introduce significant artifacts. Declaring a particular implementation "good enough" without knowing more seems premature. Here are the measures of the aliasing noise using sine sweeps. Check the figure for the 44.1kHz to 48kH conversion, the sndiod column: https://arverb.com/pub/src/ I did simple A/B tests with music from CDs and my ears couldn't hear the aliasing noise. Try it. Good a/b >x< tests for audio require extreme care to get accurate results. Simple sine sweeps don't show IM distortion well. In most cases numerically equal amounts of IM distortion are far more easily noticed than harmonic distortions or simple noise (white, pink, etc.) Sometimes you just don't want to think about it (ex., when you debug audio stuff), so resampling off-line (or switching the device rate) still makes sense in certain cases. This is the classic "why would you ever want to do that?" "Just as good" is an opinion. Other OSs can and do provide controls which allow setting the device sample rate to whatever the device can do. This user wants that to work. This means compiling a kernel with AUDIO_DEBUG Real Soon Now and inserting a few more DPRINTFs.
Re: sndio and bit perfect playback
On 10/13/22 16:14, Alexandre Ratchov wrote: On Thu, Oct 13, 2022 at 03:11:50AM +, s...@skolma.com wrote: in summary, audio works.. just not bit-perfectly :) does anyone know if SNDIO supports such mode ? and how i might configure it. bit-perfect is practical for one thing only: avoid questionings about whether the processing adds audible noise & distortion. I've tryed various hacks, including bypassing sndiod and neither was very practical. IMHO, the sndiod resampler covers 99% of the cases. To handle the remaining 1%, I just resample the files off-line. audio/sox is excellent for that. So, I'd suggest you to add "-e s24" to sndiod_flags and resample off-line when needed. HTH There's code in the kernel down through uaudio.c to set sample rates. audioctl accepts a "rate" argument sndiod accepts a "rate" argument. The code in those and libsndio looks reasonable. If those don't work it's a (fixable) bug/not-yet-implemented. I've tried those settings with ambiguous results but not failure. My usb dacs don't have visible indicators & I don't have a USB protocol sniffer. In my experience resampling quality in any particular implementation is not guaranteed and can introduce significant artifacts. Declaring a particular implementation "good enough" without knowing more seems premature. geoff steckel
Re: recommended partitions to backup with dump
On 8/24/22 13:28, Shadrock Uhuru wrote: hi everyone after losing a considerable amount of data that i had accumulated over the last year or so by trying to remove a directory called '~' that i had created by mistake in a sub directory of my home directory with rm -rf ~ which of course started to eat through my home directory with a vengence, i managed to stop it before it went to far, i didn't have any recent backups, needless to say i've learning my lesson about having a good policy of regular backups. what are the recommended partition to backup if 1 i want to do a fresh reinstall e.g. to move to a larger hard drive. 2 for a disaster recovery like what i experienced above. i will be using ville walveranta's autodump 1.5a script which does a full dump on sundays and incremental dumps during the week, i already have /home /etc and /root set for backup, are there any other partitions i should bear in mind ? shadrock I agree with "know exactly what you need" or "save everything" If backups are cheap in time and money just save everything as often as you can - daily if you do a lot online. Rebuilding from incremental dumps can be painful. Even if dumping everything I'd consider leaving out any directory with "cache" in its name That can potentially save many gigs of storage and hours of backup time. Thunderbird and firefox usually have multiple gig databases. If backups are expensive and you're being very, very selective, I'd add to your list: the output of pkg_info /usr/local, /var/mail potentially /usr/src, /var/www, /var/unbound, /var/nsd any directories I added or I'm not sure about any system directories where I've modified files Any volumes not part of the base system have their own dump schedule. hth Geoff Steckel
Re: Firefox and stuttering USB audio
On 6/1/22 4:34 PM, Courtney wrote: I have not found it to be an issue with the number of tabs being open, but rather anything that spikes the processor causing these interruptions. Oddly, even on my 8 core box, just having 1 or 2 cores spiking to 100% (which FF does on demanding sites) causes these interruptions the most. I have also simply had firefox idling with one tab open on the new tab page causing these interruptions sometimes even. I have a buddy using Arch Linux and said even now with newer versions of ff he's been having some strange behavior too. When audio stutters on my machines I can hear the disk drive rattling. All machines have at least 4 CPUs and sufficient memory. Adding nice(1) to the program playing helps but doesn't eliminate the problem. When sndiod gets in the way things are worse. Adding nice to it helps but again doesn't eliminate the problem. Moving the mouse on a slower system causes stuttering. I haven't paid attention when X isn't running. Firefox, Chrome, Thunderbird etc. constantly execute syscalls and very frequently do disk I/O. I -suspect- that some lock blocks the CPU playing the audio when other programs execute syscalls. dt/bt probably could test this but I haven't studied them yet nor have I studied kernel locking in any detail (big and subtle area). I have also seen pretty certain disk I/O delays when audio is playing and other programs compete. The BSDs have CPU priority scheduling. I don't think any have I/O priority scheduling. That problem isn't simple. Ubuntu linux successfully runs mixxx which plays & records multichannel audio in real time with no stutteron a 1.6 GHz Celeron. even through [adjectives deleted] pulseaudio. This is while OBS Studio is webcasting USB video. Keystroke/mouseclick to audio start/stop is always well under 100ms, probably 30 ms or less Getting that to work probably requires significant twisty kernel & program optimization. Geoff Steckel
Re: how to partition routing namespace
On 10/20/2015 10:19 PM, Chris Cappuccio wrote: Geoff Steckel [g...@oat.com] wrote: I'm using sixxx.net as an IPv6 tunnel gateway. They gave me 2001:::0111::0002 as my tunnel endpoint and 2001:::0111::1 as their end and router address. They gave me 2001:::8111::/64 for my address space. Note that the tunnel endpoint addresses are globally routeable. The desired behavior is to partition the network space inside the machine into the gateway section and the rest of the machine >> as if they were connected by a pair of interfaces and a cable << where the interfaces had addresses in 2001...8111 so that locally generated packets would go out with that source address. If the tunnel endpoint x:0111::0002 is globally routeable, why do you care about the machine's own traffic not appearing from that address? None the less, if you must have traffic appear from x:8111::/64, can't you just use that on your gif interface? As gif is a point-to- point interface, there is no need for both participants to be within the same subnet. Of course, if you do this, you can't then apply the x:8111::/64 address to your ethernet interface facing your LAN, which is where it was meant to go, and why it all works this way anyways. If you really must have both x:8111::/64 on the LAN and on the gif interface, you could specify a /128 address for the gif interface and only use one of your x:8111::/64 addresses away from your LAN interface. Thre is no ARP so even if the remote router knows your gif interface as x:0111::0002 and routes to it, you can still use whatever address you want. But I don't really understand why you would want to do this, unless this tunnel router is the only machine you care to IPv6 on. Chris There are a number of reasons 0111::2 is not useful to me. On reading the latest if_bridge.c it looks like it will cross routing domains. No domain information is passed with the packet. A lot of it got rewritten between 5.7 and 5.8 # desired global address ifconfig vether1 inet6 2001:::8111::8 # synthetic router address ifconfig vether2 inet6 2001:::8111::9 rdomain 1 # a synthetic wire between vether1 and 2 ifconfig bridge1 add vether1 ifconfig bridge1 add vether2 # system net routing to tunnel section route add -ipv6 default 2001:::8111:9 # tunnel section routing to external router route -T 1 add -ipv6 default 2001:::0111::1 (modulo typos & misplaced arguments) I'll load -current on a machine, set up the configuration as above & connect it to another machine via a tunnel. If it doesn't work, it ought to. And I'll try to fix it. My pf.conf might be comprehensible then. Many thanks to the people who greatly improved if_bridge.c Geoff Steckel
how to partition routing namespace
If someone has published a solution, please hand me a clue-by-4 I'm running 5.7. If anyone would like a dmesg, etc, I'd be glad to provide. I don't **think** that's relevant here. I'm using sixxx.net as an IPv6 tunnel gateway. They gave me 2001:::0111::0002 as my tunnel endpoint and 2001:::0111::1 as their end and router address. They gave me 2001:::8111::/64 for my address space. Note that the tunnel endpoint addresses are globally routeable. So... if I say "route add -inet6 default ...0111::1", then the source address of any IPv6 connection from this machine defaults to 0111::2 This isn't useful. I must use an address in 8xxx::/64 for functions on the gateway machine. Adding another machine is not possible due to power and money constraints. A $50 machine with two interfaces drawing 10W would solve this but they're hard to find. Maybe when the port to arm is stabler... even so The desired behavior is to partition the network space inside the machine into the gateway section and the rest of the machine >> as if they were connected by a pair of interfaces and a cable << where the interfaces had addresses in 2001...8111 so that locally generated packets would go out with that source address. I'm currently using two rdomains, two routing tables, and a messy pf.conf using things like (approximately) ifconfig gif0 [ipv6 endpoints] rdomain 1 route -inet6 default ::1 route -T 1 -inet6 default 2001...:0111::1 ifconfig lo1 -inet6 ::2/64 rdomain 1 pass inet6 from any to ! \ route-to lo0 rtable 1 pass on gif0 inet6 from ! \ to route to lo1 rtable 0 Occasionally this gets into gif loops and I'm not sure that packets aren't being silently dropped. Is there a simpler method? The far end of the tunnel to sixxx has no hardware address, so I haven't figured out how to do obscene things to use that as a gateway address. Suggestions, upgrade to 5.8 or current or RTFM appreciated. Geoff Steckel
Re: cdio(1) cdrip (error correction / why WAVE).
On 08/23/2015 10:31 AM, n...@nawi.is wrote: Hello ! I have here many audio cd's which I plan to rip - ripping with cdio(1)'s cdrip works without problems. Now I ask myself, whether cdio(1) does some error correction for scratched discs or, if the drive produces some errors (like other tools promotes) ? The reason why I ask is, I want to rip and don't want to control each disc after ripping (nothing like could another tool had more or better functions). Or maybe this function is overrated (for example if a lower speed is used to rip). Audio CDs were designed with simple error correction assuming the drive would interpolate data to conceal any missed samples. Therefore ripping good data is difficult and error prone. User intervention is often necessary. I use cdparanoia to rip. In my experience it is the only program which reliably returns the same data from a disc read on multiple occasions. It attempts to defeat all error correction and concealment in the drive. It then does all its own error correction and reports all instances of non-perfect reads. You will have to check the status output by cdparanoia for every disc. Even using it, while ripping badly scratched discs I sometimes have to slow to x4 or x2. It's possible you could write a script which checked the status and re-ran cdparanoia at slower and slower speeds. It is important to clean the discs very carefully before ripping. The quality of the drive is very important. I use Samsung (Toshiba-Samsung) or Pioneer drives. Also, drives wear out. After reading about 1000 discs each, two drives failed without any obvious error indications. I had to re-rip 100 discs on new drives. Only curios, why is WAVE used to save the tracks and not AIFF (code, license, portability ... ) ? No complain / flame because the target format will be FLAC as it looks now. Thanks for answers. WAV is the format M$ used for uncompressed audio. AIFF was much less widely implemented. I use sox for all my format conversions. Simple quick. It accepts and writes many formats and does all the bulk processing I need. I hope this helps. Geoff Steckel
Re: cdio(1) cdrip (error correction / why WAVE).
On 08/23/2015 05:03 PM, Christian Weisgerber wrote: On 2015-08-23, n...@nawi.is n...@nawi.is wrote: (like other tools promotes) ? The reason why I ask is, I want to rip and don't want to control each disc after ripping (nothing like could another tool had more or better functions). Or maybe this function is overrated There are tools in ports, notably audio/cdparanoia, that claim to go through great effort to add extra-robust data verification, synchronization, error handling and scratch reconstruction capability. That was important in the 1990s, but I don't know if it actually makes any sense or even works with modern drives. (Some) Modern drives are much better than older ones. However, the extremely variable quality of CD pressing and the widespread use of CD-R to distribute short runs instead of paying for a master has made the job of reading the disc harder. Damage to discs from air pollution and salt spray is still a problem. Cdparanoia reads the raw data stream before high-level error correction in the drive. That's available on (almost) every modern drive. The actual RF stream as eight-to-14 modulated bits is not available on any drive I've heard of. The decoded subframes are available on some drives but cdparanoia doesn't process them. By performing the low level processing it can presumably do somewhat better error correction than the drives. Most importantly, it can report cases where the drive would mask errors by replacing the bad data with interpolated (stuff). It also rereads any sector in error and does overlapped reads to be sure no data has been lost between reads. In some cases, drives duplicate data which is also detected. I strongly recommend cdparanoia as the best cd-audio reading program I've found. There may be ones (as above) which reach down yet another level. I've found that using a good drive (good manufacture and not used to death) only 1 out of 750 of my random collection of discs can't be read successfully. Geoff Steckel
Re: new (nasty) spam pattern
On 07/30/2015 11:14 AM, Seth wrote: On Thu, 30 Jul 2015 08:09:38 -0700, Seth l...@sysfu.com wrote: Sorry, forgot the link to greyscanner post [3] http://www.mail-archive.com/misc@openbsd.org/msg116961.html DNSBL is very powerful: post:gwes:4612$ grep -c listed /var/log/maillog 501 post:gwes:grep -c -n sent /var/log/maillog 7 I have to run sendmail until spamd or smtpd can do DNSBL No other solution works nearly as well.
Re: Audio Boost for Sndio
Some sound cards have two volume controls: one is for the specific source and the other is for the whole card. Both must be at 100% for maximum output. On 07/23/2015 06:55 AM, ropers wrote: I'm talking out my arse here, but: To me, your submission vaguely reminds me of the CD Loudness War https://en.wikipedia.org/wiki/Loudness_war. It sounds to me as if your hardware may be inherently a bit too quiet, but to an extent it's possible to compensate for that by pre-processing the signal in a similar way Loudness War CD vendors did when producing their master – but this reduces dynamic range. It may well be that those Windows drivers are doing just that, to compensate for buggy, craptastic audio hardware. But again, I really don't know; I just thought I'd mention this since nobody else has. On 11 July 2015 at 17:30, tekk t...@parlementum.net wrote: On 07/11/2015 08:24 AM, Jan Stary wrote: On Jul 10 19:15:31, h...@stare.cz wrote: On Jul 10 06:01:17, t...@parlementum.net wrote: I'm having a bit of trouble with audio on my 5.7 box (Thinkpad T430.) Audio is just a bit too quiet to be comfortable even when I have everything maxed out. I had a similar problem on Linux Are you sure the audio hardware is actually capable of playing louder than it does? How exactly are you playing what? I'm pretty sure. I mainly see it when playing youtube videos via mpv, https://www.youtube.com/watch?v=d3IidGmVLo4 was giving me trouble for example. I know for sure that the hardware is capable of being much louder since I'm able to play it at a good volume in Windows and Linux (both Pulseaudio and ALSA, after I add a boost device to ALSA.)
Re: Any books about OpenBSD ARM programming?
On 06/24/2015 11:26 AM, Piotr Kubaj wrote: Hi all, I'm mainly a FreeBSD user but want to learn OpenBSD. I'm also interested in basic electronics, like programming own thermometer. That's why I want to install OpenBSD on my BeagleBone Black and write some simple programs using I/O pins. Are there any tutorials on this? I have found some books about FreeBSD kernel programming, but none for OpenBSD. Thanks for your help. [demime 1.01d removed an attachment of type application/pgp-signature which had a name of signature.asc] For programming I/O pins there is probably a driver already written. User processes could do what you want. If you want to write a parallel pin driver, a general familiarity with kernel concepts is probably enough. Then copy something like the lpt driver. The McKusick books are a reasonable introduction to the kernel as it was some decades ago. The concepts haven't changed. The System V book also. General familiarity with concepts like address spaces, interrupts, process contexts, memory management, etc. helps a lot. Once you have that basis reading the man pages and the code is a lot easier. FreeBSD and OpenBSD have diverged but not so far as to make the conceptual bases incompatible. Linux went its own way from the beginning and it isn't close to BSD. Geoff Steckel
Re: icmp6 get dropped on gif tunnel
On 03/27/2015 01:31 PM, Bastien Durel wrote: Hello. I have an openbsd router with 2 upstreams (one pppoe (pppoe0 on sis1), one ipoe (sis0)). I have a sixxs(6-in-4) tunnel (gif0). If the gif tunnel is on one of my providers (pppoe0), it works well. gif0: flags=8051UP,POINTOPOINT,RUNNING,MULTICAST mtu 1280 description: Sixxs priority: 0 groups: gif egress tunnel: inet 109.190.17.241 - 212.100.184.146 inet6 fe80::200:24ff:fecf:42ac%gif0 - prefixlen 64 scopeid 0xc inet6 2001:6f8:202:19c::2 - 2001:6f8:202:19c::1 prefixlen 128 the 2001:6f8:3c8::/48 subnet which is routed via this tunnel This provider gives me native Ipv6, so the tunnel is pretty useless, and I want to put it on the other provider, which doesn't. But when I move it on the other provider, the tunnel basicly works (I can ping an inside box (2001:6f8:3c8:42:xxx) from the outside), but the router does not answer to ping, on the tunnel endpoint Ipv6 (2001:6f8:202:19c::2) nor on any other interface (in 2001:6f8:3c8::/48). Then sixxs count it as down, and will disable it if nothing is done. I can ping from router to remote tunnel endpoint (2001:6f8:202:19c::1), but remote tunnel endpoint does not get any answer when it ping my router endpoint. nor does can I ping it from outside. If I tcpdump gif0, I can see icmpv6 in and out. Does you have any clue ? Thanks, I've seen a similar problem with traceroute: ping from inside to outside IPv6 host works. Traceroute packets leaving gif0 are visible leaving via ipv4 interface. Traceroute ICMP6 packets returned are visible entering gif0 but aren't visible in pf (at least what I've tried) pass in log on gif0 any doesn't give me anything. I may misunderstand log vs rule matching. Is there a rule which will guarantee that a packet will be logged no matter what happens to it later in pf processing? The IPv6 packets cross routing domains to get to/from gif0. I could set up a test net (4 machines) to debug this if I had better knowledge (a) about logging as above (b) where to look in the code to put information gathering code. I suspect some sort of mismatch in the state matching code but that's because I can't think of anywhere else. If anyone has a little time to suggest places to look I'd appreciate it. If sending to tech@ be helpful I'll do that. thanks Geoff Steckel
Re: httpd: multiple addresses for one server
On 01/03/2015 08:42 AM, Reyk Floeter wrote: On Thu, Jan 01, 2015 at 11:54:46PM -0500, Geoff Steckel wrote: Is there any way todo the equivalent of: server an.example.com listen on 192.168.2.99 listen on 2001.fefe.1.1::99 ?? It appears that the code in parse.y explicitly forbids this and the data structures for a server don't *seem* to have more than one slot for an address. Is there another way to achieve this effect? From one comment in the checkins, it looks like server an.example.com listen on 192.168.2.99 . server an.example.com listen on 2001.fefe.1.1::99 would work. Duplicating the entire server description is difficult to maintain. Is someone planning to work in this area soon? thanks Geoff Steckel I used include directives to avoid duplications (see previous reply) but the following diff allows to add aliases and multiple listen statements. Reyk [...diff omitted...] 1000 thanks for an almost instantaneous and complete extension!! This makes httpd a complete replacement for apache in my host. Geoff Steckel
httpd: multiple addresses for one server
Is there any way todo the equivalent of: server an.example.com listen on 192.168.2.99 listen on 2001.fefe.1.1::99 ?? It appears that the code in parse.y explicitly forbids this and the data structures for a server don't *seem* to have more than one slot for an address. Is there another way to achieve this effect? From one comment in the checkins, it looks like server an.example.com listen on 192.168.2.99 . server an.example.com listen on 2001.fefe.1.1::99 would work. Duplicating the entire server description is difficult to maintain. Is someone planning to work in this area soon? thanks Geoff Steckel
rdomain != 0 lo0 in table
When an interface is given an IP6 address in anew rdomain, lo0 is named in various routes when that table is queried via netstat -r -f inet Does the pseudo-interface lo0 actually exist in multiple routing tables simultaneously, or does the name 'lo0' signify an otherwise anonymous point to hang the extra routes defined when any address is configured in a rdomain? - river:5626$ sudo ifconfig re2 rdomain 3 river:5627$ netstat -rn -f inet6 -T 3 Routing tables river:5629$ sudo ifconfig re2 inet6 2002:1234:4567::0 prefixlen 64 rdomain 3 river:5630$ netstat -rn -f inet6 -T 3 Routing tables Internet6: DestinationGateway Flags Refs Use Mtu Prio Iface 2002:1234:4567:: 00:30:18:ab:ee:49 HL 00 - 4 lo0 2002:1234:4567::/64link#3 C 00 - 4 re2 fe80::%re2/64 link#3 C 00 - 4 re2 fe80::230:18ff:feab:ee49%re2 00:30:18:ab:ee:49 HL 00 - 4 lo0 ff01::%re2/32 link#3 C 00 - 4 re2 ff02::%re2/32 link#3 C 00 - 4 re2 -- In a more complex case, lo0 exists with a route but was never configured in this rdomain. This is part of my implementation of an ip6 tunnel endpoint. The endpoint IP6 address must not be used as a source address from my net so that address is in a separate routing domain. pf transfers packets between the domains. -- netstat -rn -f inet6 -T 1 Routing tables Internet6: DestinationGateway Flags Refs Use Mtu Prio Iface default2001:4830::xxx::1 UGS0 427 - 8 gif0 ::1link#7 UHL00 - 4 lo0 2001:4830::xxx::1 2001:4830::xxx::2 UH 1 61 - 4 gif0 2001:4830::xxx::2 link#10 UHL00 - 4 lo0 fe80::%lo1/64 fe80::1%lo1 U 00 - 4 lo1 fe80::1%lo1link#7 UHL00 - 4 lo0 fe80::%gif0/64 link#10 UC 00 - 4 gif0 fe80::230:18ff:feab:ee47%gif0 link#10 UHL00 - 4 lo0 ff01::%lo1/32 fe80::1%lo1 UC 00 - 4 lo1 ff01::%gif0/32 link#10 UC 00 - 4 gif0 ff02::%lo1/32 fe80::1%lo1 UC 00 - 4 lo1 ff02::%gif0/32 link#10 UC 00 - 4 gif0 If this is not the intended behavior, I can dig into the code - this would be in the ip6 route add code? If it should be sent there I'll copy this to tech@ thanks Geoff Steckel
Re: Help, please, understanding AHCI error on amd64
On 08/29/2014 06:15 PM, Chris Cappuccio wrote: Evan Root [cellarr...@gmail.com] wrote: It seems that after reading the backblaze and google papers about drive reliability that there is no statistically obvious difference. It's too close to call. Both papers end with hedges and further questions. Even if enterprise drives are more reliable it isn't by more than a single percent and it isn't consistent enough to matter either. The difference in firmware between consumer and enterprise drives can be papered over with a good RAID controller. The kernel might need to timeout or do something. But the problem should be attacked there instead of saying oh just buy the 5x cost drive. It depends a lot on how you intend to use them: how annoyed you get if you have to replace one, how annoyed you get if the array goes off-line or goes into degraded mode when a drive performs its most extensive bad block recovery, how annoyed you get when the RAID controller mistakenly takes a drive off line, and how much money and effort you're willing to put into the case in which you mount the drives. You could set the RAID timer to wait for multiple minutes before declared the drive offline. That would effectively put the RAID in degraded mode during that time - writing would be disabled. You might be able to disable the drive's extended error recovery with an internal parameter setting, requiring the RAID controller to do bad block remapping but the drive will never take a long time to respond. That would minimize the time that the RAID was in degraded mode or off line. If you did neither, then when a drive did extended error recovery, the controller would declare the drive off-line. You would have to intervene and figure out whether the drive was really bad or that you just needed to reset the controller's status. The controller firmware is unlikely to do that for you. You could install the drives in a well-cooled vibration-isolating case. That would minimize seek errors caused by vibration. excess spindle bearing wear, and premature failure due to overheating. The consumer drives are not designed to read, write, and seek continuously. Consumer multiple-drive bays and boxes usually have no vibration isolation and poor cooling. Given the above precautions, a consumer drive could work very well. I had several very premature drive failures using consumer drives in a consumer multi-bay case. Since then I've always mounted drives with considerable space between them and have had no failures. YMMV Geoff Steckel
Re: Help, please, understanding AHCI error on amd64
On 08/29/2014 04:04 PM, Evan Root wrote: It seems that after reading the backblaze and google papers about drive reliability that there is no statistically obvious difference. It's too close to call. Both papers end with hedges and further questions. Even if enterprise drives are more reliable it isn't by more than a single percent and it isn't consistent enough to matter either. Evan Root, CCNA On Thu, Aug 28, 2014 at 1:40 PM, Chris Cappuccio ch...@nmedia.net wrote: Christian Weisgerber [na...@mips.inka.de] wrote: Now, the real question is whether enterprise drives actually *are* more reliable than consumer drives. For regular hard disks, the answer is definitely, no. What I extract from the various verbiage is: Under benign conditions, i.e. no significant vibration and good cooling, the two are virtually equally reliable. Under harsh conditions, specifically high vibration and high heat due to constant activity, the desktop drives are likely to fail quickly and the enterprise drives will continue to operate. The various grades of WD disks are labeled as to how many disks will be in the chassis. IIRC something like 1 or 2, up to 5, and lots for the 3 grades. This corresponds to the amount of vibration hardening. Now, with good cooling and some shock mounting which does vibration isolation but holds the drives so that they don't move during seeks (conflicting requirements, ingenuity required) desktop drives might be usably reliable. So comparing reliability statistics is only meaningful if the case, cabinet, and cooling are equivalent. My guess is that the enterprise drives are aimed at data centers which pack drives into the smallest possible space and can't afford to pull them to replace them. They would pay the 3X premium.
Re: Help, please, understanding AHCI error on amd64
On 08/27/2014 01:03 PM, Christian Weisgerber wrote: Paul de Weerd: | Here's a bold suggestion: Don't buy consumer drives. The guys that buy LOTS disagree. https://www.backblaze.com/blog/what-hard-drive-should-i-buy/ Oh, I know. That's a different operations model, though. When you have LOTS of drives, failures will inevitably become commonplace, so your whole storage architecture will be built to deal transparently with lost drives, and once that is in place, a higher rate of failure may be an acceptable trade-off. This is very different from an environment with few drives, where any single failure will be a pain to deal with. The saying goes that nobody wants backup, everybody wants restore, but I'd really prefer not having to restore either. Now, the real question is whether enterprise drives actually *are* more reliable than consumer drives. This paper: http://download.intel.com/support/motherboards/server/sb/enterprise_class_versus_desktop_class_hard_drives_.pdf describes the different features and intended uses of enterprise drives vs. desktop drives. The hardware requirements for a (good) enterprise drive far exceed those of a desktop drive. Software has to use those features as well. Most important to users: different error recovery philosophy! Desktop: do whatever is necessary to correct a read error, no matter how long it takes. The software is not time sensitive and may not be able to recover from a single sector error. Enterprise: disk must stay on line. Perform simple error recovery and depend on higher level software to repair or replace the bad sector. So unless an enterprise drive is in aRAID system with intelligent sector recovery including bad block remapping, the much higher price may not buy better usability. As usual YMMV.
Re: Left side and bottom of boot text console off of screen now
On 03/08/2014 10:17 PM, Nick Holland wrote: On 03/08/14 09:26, Chris Bennett wrote: On Sat, Mar 08, 2014 at 09:06:54AM -0500, Nick Holland wrote: On 03/08/14 08:51, Chris Bennett wrote: As of this update, I have had these two portions of the screen move off of visible area. this update ... from what? I'm going to assume from a pre-radeondrm version. Yes, from quite a while back. Only get internet here now through my new smartphone. ++ | | | | | | | |text console | | | | | | | | | | | | | | | | +-|--+ ++ Everything else is working fine. Chris Bennett Radeon DRM...looks like a desktop, so I'm guessing you have a VGA connected LCD monitor. No, HDMI note the amount of guessing I'm doing here. radeondrm runs the video in a graphics mode it didn't used to run in, so you will probably have to re-adjust your monitor. Fill the screen (top might do it sufficiently), and hit the auto adjust button, tweek if needed with the manual adjustments. It is a TV that accepts VGA, HDMI, etc. There are no tweek adjustments. oof. I got one of those, an old Dell thing. Accepts lots of things, does none of them well, it seems. I'm still thinking you have a monitor problem more than a computer problem, though certainly not supposed to happen with DVI. 'course, I've had that discussion with my old Dell thing, and it was wholly unimpressed with my logic. Is your monitor really 1920x1080? that's what radeondrm thinks it is. That's kinda what my bluray player thought of my monitor, come to think of it (it isn't), which is also shifted off to one side, iirc. I only use this console for sysmerge, but that would now be very difficult to use, since I couldn't see the options. Chris ok, sounds like you have painful connectivity and a cranky monitor (or a video combination with a bug), so for the moment, I'd suggest just disabling the radeondrm driver via ukc, and it will revert to the old style text mode, which will probably work Just Fine for you. boot boot -c bla bla bla ukc disable radeondrm *nnn radeondrm disabled ukc quit [happy (hopefully) boot] IF you have another monitor of any kind of any attachment, I'd like to verify your problem persists with it or goes away (without the UKC hack, of course) Nick. This is a common problem, even with DVI. If the monitor and video card/driver/whatever disagree about when the blanking intervals start or end, stuff will be chopped off one side or top/bottom of the display. Usually, with 24x80 text, only one character is chopped off the left side. If the characters are smaller, one can lose so many that the console is useless. Using something other than 24x80 as the default before the system setup is complete is a harmful regression. Geoff Steckel
Re: NAT reliability in light of recent checksum changes
On 01/27/2014 08:07 PM, Giancarlo Razzolini wrote: Em 27-01-2014 19:05, Why 42? The lists account. escreveu: FWIW, you don't have to out in the sticks (the backwoods?) to have a network problem: http://mina.naguib.ca/blog/2012/10/22/the-little-ssh-that-sometimes-couldnt.html However, as I understand it, in this case the TCP checksumming worked and protected the application from the corrupted data. Cheers, Robb. I wasn't exactly in the woods, but I had a 600Kbps unreliable ADSL connection that would send the packets. But the latency and corruption was so severe that TLS handshakes would take too long. And even if complete, the connection wouldn't sustain itself. Anyway, the UDP vpn improved things quite a bit. This due, well, to UDP of course, and to the dynamic compression, reducing the amount of data sent to the wire. This case you pointed, the TCP checksumming was doing it's job. Successfully protecting the application. This kind of things, where bits randomly flip, proves that computer science can be anything but an EXACT science. That's one of the reasons why the machines will (hopefully) always need humans. Cheers, To add to the preceeding... One client of mine used a CVS repository via coast-to-coast NFS. Somewhere in the deeps, the UDP checksum was set to 0 (no checksum). Somewhere else, one bit in each packet was corrupted. If the UDP checksum had been present we would have seen the bad data a lot sooner. We had to go back at least a month, sometimes more, to find good data, and then recreate all the edits. This scenario shows a danger of silently passing corrupt packets. It would be good if when data protected by a checksum is modified, the current checksum is validated and some appropriate? action is done (drop? produce invalid new checksum?) when proceeding. Geoff Steckel
Re: Is Ext2 stable enough for normal use?
On 01/02/2014 02:25 PM, Evan Root wrote: Just so you all know, This thread makes me want to try out Ext on Openbsd reeeaalll bad. I might just be able to provide real answers about how it works here in a couple weeks. Evan Root, CCNA On Sat, Dec 21, 2013 at 1:13 AM, Maurice McCarthy m...@mythic-beasts.comwrote: On Thu, Dec 19, 2013 at 05:04:32AM -0600 or thereabouts, Shawn K. Quinn wrote: I know for a fact that on GNU/Linux, NTFS performance is terrible, especially on larger files. True story: once I tried backing up something as a large .zip file to NTFS on a GNU/Linux system. The ETA would start off saying something reasonable like 3 hours then, three hours later, it would be about 8 hours and it would keep going up from there. Shawn Quinn, I take it all back. The performance on ntfs in Ubuntu 13.04 is atrocious with large files. The tar archive of video that I mentioned before had _not finished after 5:01 hours but took another 5 hours to complete. So to verify I tried a plain copy command between two internal hardisks. After the first 7G transfer had decreased to under 200K per sec. My home PC has 8G of RAM. Regards Maurice Last time I tried to read an ext3 (with journal disabled) IIRC it would mount but no other operation worked. That was 5.2 or so. FS was written by Linux 3.2. In return, of course, that Linux wouldn't mount an OpenBSD FFS. Geoff Steckel
Re: nsd sendto failure - how to debug?
On 12/23/2013 11:50 AM, Adam Thompson wrote: On 13-12-22 06:44 AM, Stuart Henderson wrote: I'm seeing lots of nsd[11026]: error: sendto failed: No route to host You may need to raise net.inet.udp.sendspace Done. Raised from ~9000ish (default) to 41600. Errors still occurring periodically, no perceptible change in rate. If that was the problem in the first place, wouldn't the error be different (ENOBUFS instead of EHOSTUNREACH)? If it's of any interest, the error often occurs in bursts (n=4 within syslog's last message repeated /n/ times window). You might get a hint from netstat statistics by capturing them at intervals and seeing if there's anything unique about periods with errors. The fact that the problem is intermittent strongly suggests a cacheing problem somewhere. No route to host, of course, suggests that the arp cache is not working for you for some reason, routing tables are messed up, or that dns lookups are returning nonsense or unusable addresses for destination names. If the program will log errors (including at minimum the destination address) or if you could add syslog or local disk file logs of errors including destination name, IP, size, and timestamp, this might help to go further. Errors for local destinations suggest arp problems. Errors for remote destinations suggest dns or some form of route flapping. If your network has any sort of dynamic routing (for instance, multiple outgoing interfaces with failover) that's easy to misconfigure. Geoff Steckel
Re: LSI SAS 9211
On 12/04/2013 02:11 PM, Chris Cappuccio wrote: Does anyone use these LSI MPT2 cards? The LSI MPT and MPT2 cards disable the write cache on the drives. With MPT, you can run lsiutil under a linux usb flash and enable the disk write cache with some minor hassle. Once the setting is flashed to the card, it works nicely. With MPT2, there is no lsiutil and the sas2ircu program doesn't give you any of the flexibility of lsiutil. A project I was working on did. I still have the cards. Geoff Steckel
Re: FAQ 7.3
On 11/21/2013 02:31 PM, Miod Vallat wrote: What should i do to have scrollback again? Scrollback is currently not supported when running frame buffer display drivers. I am not aware of plans to work on restoring this feature (although it is probably somewhere on my todolist). Btw, to mitigate this fact, is there maybe a mode to determine the geometry of cli framebuffer, like 80x50 or 100x40 etc? Not yet. However, there is work in progress to allow for the console font metrics to be changed at runtime, which would in turn allow different resolutions for the textmode emulation. Soon to hit a source tree near you. Miod KMS is a very good thing for X company. I'm disappointed that another very useful feature (scrollback) got lost along the way. When things go wrong, especially during stressful operations like reinstall and upgrade configuration files, 24x80 is IMnsHO inadequate and scrollback is really, really useful. At those times tmux or other layers are not easily available - /usr may not be mountable yet, the net is almost certainly off because pf hasn't been configured correctly yet, and it's quite likely there's no other machine around to use for a serial console. Please keep us posted on the font metric changes. 50x would be a lot better but still very much less than the current 100 or more scrollback lines. How early in the boot process would the font metric change capabilities be accessible? Could a boot-time or config option work? I'd be very glad to test and help debug anything in this area. thanks Geoff Steckel
Re: OpenBSD crypto and NSA/Bruce Schneier
On 09/11/2013 05:42 AM, Rudolf Leitgeb wrote: Second, low hanging fruit. Contrary to what some hysterical reports may claim, and some violations of rules aside, NSA is mostly after bad guys, some of which know quite well what they are doing. These bad guys will not necessarily be kind enough to present NSA with unpatched Windows desktops. why bother with us ? people are most generally NOT careful. So, hey, what if you can't break in OpenBSD ? This is not a marketing operation run by NSA which can claim success if they catch the 90% dumbest. Quite to the contrary, they should be most interested in the most sophisticated ones, and why wouldn't bad guys use OpenBSD if they had the impression it was more secure? As I have mentioned before: what good is perfect security in an OS if you have no control over the hardware? Put some back doors into the CPU or the networking hardware and OpenSSH will fall. There is really no point in trying to outwit three letter agencies with our laptops. Disk drives are (presumably) trivial to take over. They have firmware and mechanisms to use alternate physical blocks for a given logical block. Scenario: Reset - request for block 0 within a timeout window - substitute alternate boot record subsequent interesting code. Modern drives contain enough spare sectors to have acomplete software universe hidden in them. no reset or timeout - request for block 0 -return good data Very hard to detect without a reasonably high level of suspicion and a properly set up test jig. The conditions for substituting interesting data could be made arbitrarily complexand/or sophisticated, including scanning data read and written for patterns. Almost anything with microcodeor firmware can be subverted with very few traces. That means network interfaces, CPUs, disk controllers, USB interfaces, . Oh yes - cars trucks. Geoff Steckel
Re: 10GbE (Intel X540) performance on OpenBSD 5.3
On 08/07/2013 12:55 PM, Maxim Khitrov wrote: I found a number of recommendations for the things to keep an eye on, but nothing that gave me any ideas on what else to try for improving the performance. Specifically, I looked at netstat -m on both systems, where everything was well below the limits (500 mbufs in use during the test). I see about 8200/8700 (ix0/total) interrupts in systat with 1500 MTU. CPU usage in top is split between two cores, one at ~80% interrupt and the other at ~80% system. Most of the time all four cores are at least 10% idle (hyper-threading is disabled in BIOS). netstat -i shows no errors for ix0 and sysctl net.inet.ip.ifq.drops is at 0 on both systems. First, 80% interrupt time says that there's really no more time to process packets. AFAIK, OpenBSD only processes interrupts on CPU0. If 127.0.0.1 only got 12Gb/S, that strongly suggests that it can't process fast enough to handle a real 10Gb/S load. Second, 10Gb/S is the hardware bit rate. The only way to see that is to send one infinite-length packet. Every packet has sync, preamble, and trailer bit times which reduce the time for sending data payload. Third, does your test program account for IP and TCP/UDP headers? If it doesn't, then that's even more overhead deducted from the available payload. Fourth, can the interface card really send at line rate? Many 100K and 1G cards can't. I've never used a 10G card. Fifth, if your test program segfaults at any time, it can't be trusted. Sixth, there were out-of-order packets. This could mean packet loss, either because either sender or receiver cards were confused/overloaded/something or the receiver CPU went deaf long enough to drop packets. I used to work on high performance networks (at least, high performance for the time). All of the above were real problems I encountered. Geoff Steckel
Re: Hardware backdoors in Lenovo?
On 07/26/2013 04:56 PM, Dmitrij D. Czarkoff wrote: On Fri, Jul 26, 2013 at 08:36:17PM +, Christian Weisgerber wrote: (2) Since the NSA has preferential access to all sorts of vulnerabilities (if not outright backdoors) in IT equipment exported by American companies, it stands to reason that they are scared shitless of the reverse scenario. In fact Chinese hardware could be banned just because of theoretic future security risk. That's not to mention the fact that it may be banned because the US backdoors can't be planted any more - workstations for security-concious environments cost quite a lot, and banning some company from this market would make a good point in negotiating such delicate matters. (3) There is an ever-increasing amount of code running outside the control of the operating system. Have you looked at the remote management options of a plain office PC lately? CPU microcode updates from the BIOS? And what *does* all that SMM code do? It's all completely trustworthy and bug free, I'm sure. FWIW the network cards' firmware would serve a better place for backdoor - they interfere with network and do some cryptography the OS relies upon. Don't forget disk drives. Hmmm, I've been reset, and we'rereading block 1. Let's give him hidden block 1.With a little tinkering,multiarchitecture takeovers. Geoff Steckel
Re: SSI
On 09/27/2012 05:16 PM, m brandenberg wrote: On Thu, 27 Sep 2012, David Coppa wrote: For starters, what is SSI? As many TLAs go, it can mean multiple things. I won't try to guess what you want. Here's my guess: http://en.wikipedia.org/wiki/Single-system_image I... I... I can hear Theo's eyes roll from *3000* miles away! -- Monty Brandenberg What problem is this single image intended to solve? There are better ways to do whatever it is. There are research papers showing that the cost of transferring a process from machine to machine greatly exceeds the incremental increase in CPU availability. Sharing memory across a network, like threads, is almost always the wrong approach to data coherency. Sharing I/O devices across a network has mostly been accomplished, though NFS is a bad example. Synchronizing the mess is, in the general case, impossible. Basically, SSI is one of those traps (like threads, again) that might appeal to the naive but dies horribly once one really looks at the idea. Geoff Steckel
Re: Reading a damaged disk
On 09/24/2012 03:30 PM, STeve Andre' wrote: Is there a way, with something like the scsi(1) command, to tell the sd driver to ignore the error below? I've some data on this disk that I'd like to get back. I realize that all hell may break loose trying something like this but I'd like to try.. Any other ideas on how to get the disk stable enough to look at a few files would be appreciated. I have a backup but wouldn't mind the latest data. tnx, STeve Andre' Sep 24 15:22:18 paladin /bsd: umass0 at uhub0 Sep 24 15:22:18 paladin /bsd: port 1 configuration 1 interface 0 JMicron USB to ATA/ATAPI Bridge rev 2.00/1.00 addr 2 Sep 24 15:22:18 paladin /bsd: umass0: using SCSI over Bulk-Only Sep 24 15:22:18 paladin /bsd: scsibus4 at umass0: 2 targets, initiator 0 Sep 24 15:22:18 paladin /bsd: sd3 at scsibus4 targ 1 lun 0: WDC WD25, 00JS-75NCB3, SCSI2 0/direct fixed serial.152d2352DCA4469971FF Sep 24 15:22:18 paladin /bsd: sd3: 238418MB, 512 bytes/sector, 488281250 sectors Sep 24 15:22:18 paladin /bsd: sd3(umass0:1:0): Check Condition (error 0x70) on opcode 0x28 Sep 24 15:22:18 paladin /bsd: SENSE KEY: Media Error Sep 24 15:22:18 paladin /bsd: ASC/ASCQ: Unrecovered Read Error Sep 24 15:22:18 paladin /bsd: sd3(umass0:1:0): Check Condition (error 0x70) on opcode 0x28 Sep 24 15:22:18 paladin /bsd: SENSE KEY: Media Error Sep 24 15:22:18 paladin /bsd: ASC/ASCQ: Unrecovered Read Error Sep 24 15:22:18 paladin /bsd: sd3(umass0:1:0): Check Condition (error 0x70) on opcode 0x28 Sep 24 15:22:18 paladin /bsd: SENSE KEY: Media Error Sep 24 15:22:18 paladin /bsd: ASC/ASCQ: Unrecovered Read Error It's unlikely that anything would help. This looks like reading block 0 fails. If the disk goes click-click-click, nothing will help. Attempting to mount any partition will probably fail. Try: dd if=/dev/rsd3c of=/dev/null count=1 skip=5 If this command succeeds, then you have a chance of finding something. You have two options: search through the raw disk for possibly useful data or attempt to make a partial disk image on a new drive, filling in the correct fdisk and disklabel data. In either case, don't attempt any operation on block 0 or anything in the first (say) 1000 blocks. If you get errors from attempting to read blocks far away from 0, it's likely that the disk is now a paperweight or a source of two powerful magnets. Geoff Steckel
Re: strange system resource limits
On 08/10/2012 06:33 PM, Vijay Sankar wrote: Quoting Friedrich Locke friedrich.lo...@gmail.com: Hi, i have setted my system resources for a given user via login.conf, but after the user login the ulimit -a returns different values. Here is my login.conf entry: general:\ :coredumpsize=infinity:\ :cputime=infinity:\ :datasize=infinity:\ :filesize=infinity:\ :stacksize=infinity:\ :maxproc=infinity:\ :memorylocked=infinity:\ :memoryuse=infinity:\ :openfiles=infinity:\ :vmemoryuse=infinity:\ :auth=krb5-or-pwd:\ :ignorenologin:\ :localcipher=blowfish,6:\ :ypcipher=old:\ :priority=-5:\ :ftp-chroot:\ :tc=default: But when i log in, what i get for ulimit is: sioux@gustav$ ulimit -a time(cpu-seconds)unlimited file(blocks) unlimited coredump(blocks) unlimited data(kbytes) 8388608 stack(kbytes)32768 lockedmem(kbytes)unlimited memory(kbytes) unlimited nofiles(descriptors) 7030 processes1310 My doubt is why data and stack limits are not infinity ? Thanks in advance. I think this could be because the developers do not want datasize or stack to be unlimited :) I do recall reading somewhere in the lists that the maximum amount of virtual memory that can be allocated by a process using malloc is 8GB and is set by MAXDSIZ (in vmparam.h). Hopefully I am not giving you a totally silly answer and someone more knowledgeable will answer your question correctly. Vijay Sankar, M.Eng., P.Eng. ForeTell Technologies Limited vsan...@foretell.ca - This message was sent using ForeTell-POST 4.9 There's a physical limit to memory available on a machine: RAM + swap space. It is impossible to grant a program more memory than that. Using swap space incurs a very large performance decrease. It is advantageous to all users of a system for programs to request as little memory as is reasonable. For various reasons, an arbitrary upper limit is set even if more RAM and swap space is available. The OS architects set that limit based on their best judgement of the tradeoff between program size and overall system utility. Avoiding paging of code and stack avoids truly painful bad performance. Also, the kernel is mapped into the address space of all user programs. On 32-bit machines, this limits the maximum possible user program address space to about 2G. On 64-bit machines the constraint depends on how many bits of address space are actually implemented and other considerations. Limiting stack space to 32M is fairly reasonable - recursion to huge depth may be theoretically wonderful but rarely useful in real life, and allocating multi-megabyte stack variables is likely a programming error. One might argue for a somewhat larger maximum for truly twisty programs, but I haven't seen many programs that need over 8M. When main memory becomes 10X larger and CPU speeds 10X higher changing the limit might be considered. The above is from observing various OSes externals and internals. The kernel group obviously know far more about how the limits are chosen. Geoff Steckel
Re: softraid 5 current state?
On 08/06/2012 10:42 PM, Nick Holland wrote: On 08/06/12 17:22, Geoff Steckel wrote: Does anyone know what the current state of softraid 5 is? The man page says rebuild and scrub are not supported. The last checkin was about 6 months ago. sounds like your question is answered. Scrub and rebuild are critical for RAID5, if that wasn't obvious... Play, write code, don't put into production yet. Any information would be appreciated. I've got 3 or 4 terabytes that need a reliable home. And yes, RAID is no substitute for backups. One place I worked put 4 drives in a case with fans for 1. RAID go bye-bye. Sometimes, the lack of ability to use your first choice of a design causes you to look closer, think harder, and often you come up with a better solution. For the amount of data you are talking, and the lack of other key words like access time and such, I'm guessing you are looking at music, video and picture-type files. Mostly static stuff. If your issue is not losing data, and your data is mostly static, get a few 2-3TB disks, break them up into 1TB partitions. Fill a chunk, SHA256 all the files on that chunk, mark it read only. Fill next chunk, SHA256 all the files, mark it read only, etc. As the chunks are filling, rsync them to another disk, preferably in another machine. Your actively filling chunk, maybe you want to make that RAID1 until it is full, then copy it off to two separate chunks, and start over. Periodically, re-run your SHA256's against your RO files, looking for changed data...and fix (from the other copy) if found. Note: this can give you an actual backup of your data. Good as a one month rotation with monthly pulls? Of course not, but beats the heck out of RAID(anything). Why chunks (partitions) of 1TB rather than one Huge Disk? Several reasons: * Encourages you to lock file systems and mount them only as read-only. * Encourages you to PLAN for filled file systems. This file system WILL fill in the near future. You will have to do something different in the near future. Plan for it now. * Makes upgrading storage easier: * Install new disk. * Point new files to go to new disk. * if new disk is significantly bigger than old disk: * at leisure, copy chunks from old disk to new disk. * Verify successful copy * remove old disk. (note: 1TB takes a while to move. I don't care how you do it) * Beats the heck out of copying all data from old to new system and being down until it is done!! * RO partitions contain and minimize some kinds of disasters. I did this some years back on an e-mail archive (actually, I used a number of small arrays, rather than individual disks). I must say, there was no question in my mind after running it through a number of technology improvements and other events, several small partitions beat the heck out of one big array. Blew out a big chunk of the storage at one point...no big deal, was restoring from (a snoot-load of) DVDs while it was gathering more data at the same time -- downtime measured in a small number of hours (and no lost data). In my day job, I do have the opportunity to use ZFS and other volume managers and fancy file systems. For the most part, they just cover for bad (or no) system design rather than solving problems that can't be solved better in other ways. Not that I haven't had them help me out (maybe even haul my ass out of the fire), but usually the message should be, your design sucked, you didn't know what you were doing, maybe you should start over. Nick. Yes, this is all very sensible and I've done things that way before. I've already split out the (almost) static data onto multiple drives which has saved me several times. Even so, that data sometimes gets modified so it has to be carefully watched. Unfortunately, the bulk of the data are not entirely or predictably RO, the rate of filling is unpredictable, and segmenting it into chunks would make it largely unusable. I'd have to rebalance fairly frequently and accidentally running out of space on a chunk would be very painful. With my current 2TB file system, I have to plan fairly carefully to avoid saving too many intermediate steps. The main failure mode I've seen is drive (not file system) failure so splitting a drive into multiple file systems hasn't been useful. There's a very large premium for me to use something which doesn't require installing and removing internal drives. Good SATA/SAS drive bays are very expensive. I've used external USB drives which are messy, slow, and less reliable than I'd like, but I already use a number of them and cycle through them as backups. A RAID 1 volume for online use looks like the best available solution to prevent loss of current data (since last backup). Thanks very much for your thoughts! Geoff
softraid 5 current state?
Does anyone know what the current state of softraid 5 is? The man page says rebuild and scrub are not supported. The last checkin was about 6 months ago. Any information would be appreciated. I've got 3 or 4 terabytes that need a reliable home. And yes, RAID is no substitute for backups. One place I worked put 4 drives in a case with fans for 1. RAID go bye-bye. Geoff Steckel
Re: Kernel Level Audio Next Generation
On 08/05/2012 06:02 AM, Alexandre Ratchov wrote: On Tue, Jul 31, 2012 at 10:46:57PM +0300, Alexey Suslikov wrote: Hello misc. http://klang.eudyptula.org/ very interesting ideas. Just curious, why they didn't even try to evaluate OpenBSD sndio. An overall approach to the problem is interesting thing too Q: Why a audio system in the kernel? A: Because it's the only reasonable thing to do. IMHO it's not important; both kernel-mode and user-mode approaches can work, neither seems better. Both have their advantages. kernel-mode is thought to be better with respect to underruns, but that's not 100% true. Trying to explain. Underruns occur whenever an element in the audio processing chain doesn't complete its work before the deadline imposed by the hardware. This chain includes the audio player, which necessarily runs in user mode. Thus we have to get right audio programs (user-mode) first to avoid underruns. If you're able to write audio programs that don't underrun, then you're also able to write an audio daemon that doesn't underrun. No need to put the audio system in the kernel in this case. If you're unable to write audio programs that don't underrun, audio will stutter, and putting the audio system in the kernel won't fix stuttering, so no need to put the audio system in the kernel either. AFAICS, the main advantage of a kernel-mode implementation is to avoid the overhead of extra context switches. In the current sndio implementation the overhead of context switches is negligible compared to audio processing. The choice of moving parts of the OpenBSD audio sub-system out of the kernel is mostly for practical reasons; having less code with kernel privileges seems sane as well; A kind of if a program doesn't need kernel privileges then put it in user mode principle. What people think? Maybe we should write an article for wikipedia to make sndio more visible to rest of the world? Sure -- Alexandre Yes, no, maybe, your mileage may vary... There are two separate issues here: throughput/bandwidth and latency. Confusing these is a very common misconception. You can have all the bandwidth (CPU disk) in the world, but if program latency (buffer size / sample rate), you will have over/underruns. If (buffer size / sample rate) is too large, then it is possible that there will be large latencies when starting stopping, etc. To have a system which is agile and does not over/underrun, the hardware should be controlled by a pair of functionalities: 1) a kernel function or a process running at real-time guaranteed latency priority which directly controls the hardware. 2) a user process running at a lower priority but one higher than processes doing interruptible long-running tasks. Its latency must be controlled as well as possible. Functionality #1 must never use a blocking system call except a poll/select or something similar. It should use a hardware buffer sized for the best compromise between desired responsiveness and CPU usage tending the buffers. This process/kernel function should then talk to functionality #2 using buffers sized for the greater latency of the non-real-time process. Functionality #2 can control the hardware quickly via the real-time function yet it can buffer large quantities of data for efficient use of its resources and to compensate for large latency. This process must run at a priority sufficiently high to take precedence over updating the screen or other indefinitely CPU-intensive tasks. It does format conversion, resampling, and other tasks which consume relatively large amounts of CPU or memory. It is often helpful to store audio on a dedicated disk to reduce or eliminate queueing delays. I've implemented such an architecture on a 1 MIPS machine handling 16 in and 16 out channels simultaneously. The bottleneck was the disk drive. Applications of this type almost always work best when engineered throughout with reducing latency as the top priority. Current CPUs and disks are capable of handling a huge number of audio channels if properly used. I.e. get the video out of the way, ensure low interrupt latency by eliminating all kernel spins or indefinitely long loops with interrupts blocked, etc. Geoff Steckel
Re: Does OpenBSD have any plan to support Netmap framework?
On 07/13/2012 05:13 PM, Ted Unangst wrote: On Fri, Jul 13, 2012 at 16:06, Andres Perera wrote: you did! you explicitly said that it would be advantageous for programs looking to perform analysis on captured packets. for those programs, it turns out the placement of the filter doesn't matter Sure it matters. Simple example: count packets by /24. Very simple, but you can't do it with bpf. You have to pull all the packets to userland, and bpf is kind of slow at that. Not sure why I wanted to wade into this, but nobody's going to force you to use netmap. A perhaps silly question: in order to achieve the throughput described, is the application in a hard loop monitoring the ring status? If so, the statistic is of limited applicability. Geoff Steckel
Re: cpu choice for firewall
On 06/28/2012 01:50 PM, Joe S wrote: I'm looking to build a new mini-itx firewall based on OpenBSD and would like to get some advice on CPU selection. I've seen multiple statements on this list that indicate CPU cache and CPU speed are the most important factors. Sorry if this is a silly question, but which cache is most useful for what I'm trying to do? L1, L2, or L3? What's more important from a CPU point of view? I don't have a specific amount of throughput that I'm targeting. I'm very curious as to what kind of differences I'm likely to see. By the way, the two CPU's I'm looking at are: Intel Atom D2500 (on Intel D2500CC motherboard) Frequency (MHz): 1867 L1 cache: 32 KB (code) / 24 KB (data) L2 cache: 1024 KB L3 cache: none Intel G620 (on Intel S1200KP motherboard) Frequency (MHz): 2600 L1 cache: 64 KB (code) / 64 KB (data) L2 cache: 512 KB L3 cache: 3072 KB The cache numbers are very different on these CPUs. (both boards are mini-itx and have dual intel gigabit nics) A 100MHz I486 handled 1.544Mbits/sec using about 5% of the CPU. Running IPSEC added another 10% - no hardware acceleration available. On my 5/15Mbit link using a 1GHz VIA chip, it's hard to get CPU utilization over 15%. Latency is 150-300 microseconds. A Geode at 300MHz could probably handle 1Mbit. It all depends on your interfaces, your traffic and whatever else the machine is doing.
Re: Seagate Expansion 3T disk works via USB but not via SATA
On 06/21/2012 06:03 PM, Kenneth R Westerback wrote: On Fri, Jun 22, 2012 at 12:26:23AM +1000, Jonathan Gray wrote: On Thu, Jun 21, 2012 at 11:54:55PM +1000, Jonathan Gray wrote: It seems the lba48 capacity values being pulled out aren't sane for whatever reason. Can you try switch the controller into ahci mode via the bios? Looking at this again, it seems there is no support for 4k sectors with wd(4) only sd(4). So until someone with a 4k sector disk can change that code you'll have to switch the controller to ahci mode. Dammit. The plan was for wd(4) to die before disks got that big. Sigh. Ken The ST421/506 interface will never die. The Signetics 8X300 processor's ghost lives! The 3090 emulated the 3070 emulated the 370 emulated the 7094 emulated the 1401 emulating the 401 (IIRC). Somebody probably runs a 3090 simulator on a AMD64 emulating a 486 emulating a 386 emulating an 8085. Or an 8008. I don't think a 4004 could stand in for a 3090. Geoff
Re: Large (3TB) HDD support
On 06/01/2012 03:41 PM, Theo de Raadt wrote: 2012/6/1 Tyler Morgantyl...@tradetech.net: http://www.openbsd.org/faq/faq14.html#LargeDrive That doesn't mention GPT, which is the problem with drives2TB. https://en.wikipedia.org/wiki/GUID_Partition_Table Can OpenBSD already boot from a 4TB drive on an UEFI system? Try to buy systems that don't rely on UEFI. In the next few years, prepare to buy systems and find out they require UEFI, and then demand a refund. Prepare for it to get even worse than that. If you buy a UEFI-capable system today, you are giving power to the people who will (in the future) make UEFI-only systems which are incapable of running OpenBSD. UEFI arrived with all sorts of promises of making machines better, but is being turned into something completely nefarious. On the surface, UEFI seems to allow the manufacturer and others to insert any amount of black-box malware during boot. That's enough to make me shudder. Obviously, opportunities abound for that code to prevent unauthorized O/Ses from running or subvert them once running. Are there other and larger issues? On the other hand, GPT by itself appears useful. Does it also contain boobytraps? Thanks! Geoff Steckel
Re: Large (3TB) HDD support
On 06/01/2012 04:55 PM, Theo de Raadt wrote: On the other hand, GPT by itself appears useful. What is useful about GPT? *EVERY USER* has the following simple requirements: 1. I have a machine. 2. I want to install an operating system on it (or, have an operating system installed from the factory) What am I missing -- what is useful about GPT? I guess I mean -- what is *MORE* useful about GPT, compared to the other options already available? I guess I should be even more clear: Does GPT provide any functionality that *EVERY USER* needs? No, it does not. I agree completely that any OS can write any kind of disk label (or none) that it needs. As you say, in the very, very large majority of cases no FDISK or GPT or anything else external to the loaded OS is needed. The apparent advantage of GPT over FDISK partitions is that it can describe partitions 2TB for systems hosting multiple OSes. That's all I meant. Sorry that it wasn't clear. Geoff Steckel
Re: Unbound
On 05/22/2012 08:50 AM, Stuart Henderson wrote: On 2012-05-21, Geoff Steckelg...@oat.com wrote: On 05/20/2012 10:49 PM, Nick Holland wrote: On 05/20/12 17:49, David Diggles wrote: Ok, I am interested in opinions on why one should migrate from BIND to unbound? 1) It is unlikely there will be any more updates to BIND9 in OpenBSD base install. 2) It is even more unlikely that the comedyfest that BIND10 appears to be added to OpenBSD base install (*snicker* python?? *snicker*) 3) BIND sucks. Degree of suckage has varied from release to release, but it has consistently remained a bad idea implemented poorly. 4) Unbound NSD sucks less. Nick. My site needs both split horizon and pretty complete authoritative support. Does anyone have suggestions about BIND replacement(s) for this scenario? Right now BIND works for me (for some value of works.) One machine serving as: 1) primary nameserver for multiple domains 2) secondary nameserver for multiple domains 3) internal nameserver for domains in (1) with additional records 4) internal nameserver for internal domains If there is a discussion of this in an archive some place I'll look for it. I didn't see much useful searching for split horizon and unbound. thanks! Geoff Steckel I think this type of setup is the simplest way to handle this with a combination of nsd + unbound: - Local clients point to an IP address running unbound - NS records point to an address running nsd for the authoritative zones - For the authoritative zones, unbound does a normal lookup for the main records but has additional local-data lines for the additional records in (3). - Unbound has local-zone / local-data lines which are very straightforward when (4) is a reasonably simple case (e.g. override a few A records, add some others, etc) OR - Unbound has stub-zone lines if (4) requires a full authoritative server (DNSSEC etc) IIRC, if you are providing reverse DNS for RFC1918 zones you need to override the default static zones in unbound. If you require that the internal domains cannot be resolved from external addresses (e.g. if someone points directly at the authoritative server and knows the domain name), at the moment you would need a separate copy of nsd bound to a different address/port and firewall it off or bind to a local-only address. I wonder though, there is already of course a per-zone acl setting for zone transfers, so perhaps it might not be considered too much feature-creep to add a per-zone query ACL to NSD... Of course continuing to use BIND is another option. Thanks very much! I think using NSD for the outward facing authoritative service makes sense. Retaining BIND is probably best for the internal service since I see no way to add the local domains, etc. to unbound/nsd while retaining recursive search of external servers. It may be possible. I just can't see how to do it without truly complex setup. thanks again, Geoff Steckel
upgrade: 3-way sysmerge a possibility?
I'm afraid I must be very dense or make very unusual configuration changes since sysmerge-ing my systems takes at absolute minimum two hours each and the gateway and major servers take more. It is true that some of that time is spent preserving changes to configuration files for various installed packages which is an orthogonal problem. A great deal of the time is spent figuring out which changes are noise. Cross-checking to be sure that the changes are correct takes even longer. Eliminating duplicated changes (i.e. changes I installed which are included as standard in the next release) is also tricky. The noise presented by sysmerge is a major reason I find it very difficult to upgrade my systems as often as I would like. I usually reinstall about every three releases since that is far less work than three upgrades. Of course, reinstallation has its own perils. In the best of all possible worlds, sysmerge would use diff3 or a similar tool. It would take some previous release, the new release, and the modified /etc and /var so that the administrator could see which were his changes and which were changes between releases. It could use a set of diffs between released versions as input so that the administrator did not need to save the unmodified previous /etc /var. One possible variant of this could create a reinstall tarball which would allow an administrator to extract all relevant changes from an obsolete installation to apply to a fresh installation of the most recent version. I looked at this some time ago. I didn't pursue it because of lack of time and wanting to see if sysmerge would do what I needed. I didn't want to propose something that sysmerge did already. A good 3-way editing tool that could be used on a regular terminal window didn't seem to be easily found, either. Another aspect I explored was how to handle the general case of derived configuration files such as sendmail.cf - there is no useful standard place where these are generated (/usr/share/sendmail is not a good place). sysmerge has special cases for passwd, etc, but those are hard-coded. If there is any consensus that this would be useful to anyone else, I would be glad to contribute to any project of this sort if one exists, either to test or to work on any part of it. Geoff Steckel
Re: Unbound
On 05/20/2012 10:49 PM, Nick Holland wrote: On 05/20/12 17:49, David Diggles wrote: Ok, I am interested in opinions on why one should migrate from BIND to unbound? 1) It is unlikely there will be any more updates to BIND9 in OpenBSD base install. 2) It is even more unlikely that the comedyfest that BIND10 appears to be added to OpenBSD base install (*snicker* python?? *snicker*) 3) BIND sucks. Degree of suckage has varied from release to release, but it has consistently remained a bad idea implemented poorly. 4) Unbound NSD sucks less. Nick. My site needs both split horizon and pretty complete authoritative support. Does anyone have suggestions about BIND replacement(s) for this scenario? Right now BIND works for me (for some value of works.) One machine serving as: 1) primary nameserver for multiple domains 2) secondary nameserver for multiple domains 3) internal nameserver for domains in (1) with additional records 4) internal nameserver for internal domains If there is a discussion of this in an archive some place I'll look for it. I didn't see much useful searching for split horizon and unbound. thanks! Geoff Steckel
Re: a live cd/dvd?
[lots of text snipped] I was looking at laptops recently. I took 2 linux CDs, an OpenBSD install CD, and a USB stick with OpenBSD on it. I got a lot more useful information about hardware compatibility from the OpenBSDs than the Linux CDs because OpenBSD didn't try to bring up anything graphical at the beginning. The tools on the OpenBSD install disk were (just barely) sufficient to do what I needed. I didn't use the stick because the USB ports on the store systems weren't easily accessible. I've also rescued unbootable systems with the OpenBSD install disk. Live CDs take forever to boot and run because seeking on a CD is very slow. The install CD came up a great deal faster because it didn't try to set up a fancy environment. If one really wanted to make an OpenBSD live DVD, one might (this has *not* been tested): Install onto a clean disk with everything on one partition. Add 2 entries to / (/mem_var, /mem_etc) Add 3 entries to /dev for memory file systems. Edit /etc/fstab to point /tmp, /var, and /etc to those. Add some code to the beginning of /etc/rc to: create the 3 memory file systems mount /mem_etc and /mem_var copy /etc to one and /var to another unmount the copies Create a DVD with a boot sector from the above. Presumably one could write a script to do this procedure and apply it to any release. I don't intend to write such a script. Someone who wanted to do this would need to know the purpose of /etc/rc and shell programming. That person would not need to know any kernel internals. All the necessary tools have sufficient manual pages. I'm quite sure I missed something. init should continue to read the buried /etc/rc... or at least about 40 releases ago that's what would happen. This begs the questions of networking, setting up X, etc. This doesn't rate a FAQ entry. It does show you can do this with the tools supplied and it's not rocket science.
Re: acer aspire one D270
On 05/03/2012 07:04 PM, frantisek holop wrote: hi there, here is the dmesg for a spanking new acer aspire one D270 netbook. installation was easy and fast. devices not supported (yet): intel integrated video, BCM4313 (probably gonna replace with an iwn) there is a curious ehci0 timeout.. the 6 cell battery is supposed to give 8h, so would be a nice long haul netbook :] OpenBSD 5.1-current (GENERIC.MP) #253: Thu Apr 26 01:45:24 MDT 2012 dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP . Broadcom BCM4313 rev 0x01 at pci2 dev 0 function 0 not configured I have an HP Pavilion-1 4010dm-US with that chip in it. Even the Penguin doesn't have a working driver. The only thing I was able to make work on it was the Broadcom binary blob. Your plan for another interface sounds wise. Geoff Steckel
Re: Question on LPD and OpenBSD printing
On 04/04/2012 06:10 PM, Girish Venkatachalam wrote: Nothing. Okay I am giving up now. -Girish -- G3 Tech Networking appliance company web: http://g3tech.in mail: gir...@g3tech.in telnetprinter_ip_address 9100 %!PS (hi\n) print flush What does it do? If it echoes hi, then postscript works. end with a controld then close the connection Another test would be telnetprinter_ip_address 9100 %!PS 100 300 moveto /TimesRoman findfont 24 scalefont selectfont (Testing 1 2 3 4) show showpage controld That should print a page with Testing 1 2 3 4 in the middle. Does the printer have a built in web server for configuration? Are the correct ports and emulations enabled? This is unlikely to be a problem with lpr if you have configured /etc/printcap according to the example included in it. Are you sending postscript or HPGL to the printer? If you are sending plain text it is very unlikely you will see anything useful. Use a2ps (for example - there are other programs which do the same) to format plain text into postscript. Geoff Steckel
Re: the aucat recording studio - stereo panning
On 02/10/2012 05:04 AM, Jan Stary wrote: On Feb 09 18:30:01, Geoff Steckel wrote: Has 'sox' been mentioned in this context? For many simple filters, source-sox-aucat works well for me. Command line only. Yes, I'd very much like to stay on the command line for this. Would you please care to share the specifics of how you do it? Thank you very much Jan Check sox.sourceforge.net - the documentation there is good. Geoff
Re: the aucat recording studio - stereo panning
On 02/09/2012 06:05 PM, Alexandre H wrote: You'll face other problems preventing you from doing everything with aucat. First, there's no reverb, which is necessary to create the spacial feel, volume changes are too abrupt (cause small clicks) and not real-time. Implementing pan, effects and smooth parameter changes would bloat aucat/sndiod. IMO the way to go is to handle processing in small programs (with a simple record-process-play loop) and keep sndiod only for routing the signal to the hardware or other programs. Currently that's the way I handle some effects, I write small programs that apply effects on the record stream to send the result in real-time on the play stream. Then I use -mmon to record the result in a file. Not very flexible, but good enough to test the concept. If I understand what you are doing, perhaps you have another way. Noatun doesn't make the deal ? You play audio file with it and add the filter Arts::Synth_FREEVERB. It has the essential control knobs for reverb. Kaffeine has other filters for audio. Noatun Kaffeine work well with filters, parameters can be adjusted in real-time. And it should be possible to record the audio output in a file. And perhaps you will want to write new filters for them ;). Has 'sox' been mentioned in this context? For many simple filters, source-sox-aucat works well for me. Command line only.
python3 deleted from ports
python 3 was moved to lang/python from lang/python3 in ports. That probably is a good idea. Unfortunately, and to my *great* inconvenience, lang/python3 was removed from ports and packages before 5.1 will be released. I had begun developing an application using python3 and had to reinstall. It was unsettling to find that I couldn't get the package. The old ports/python3 won't compile and install due to a problem with libreadline. I would be glad to provide the context to anyone who is interested. Several compiles fail (non-fatally, unfortunately) and finally ldd: libreadline.a is not an ELF executable is the fatal problem when trying to link with c++ during make install. I understand that this would be an extremely low priority item since presumably when 5.1 is released python 3 will be available again. I've tried patching all sorts of files, symbolic links in /usr/include, retrofitting .mk files, and so forth and so on from the CURRENT ports and from the 5.0 ports to no avail. Yes, I know this is crossposted. I'm not subscribed to ports@ I'll look at the archived ports@ every day for a week or three. I searched everywhere I could get to for a reason why the previously apparently working port and package were removed before the new one was available. If there is a reason why the 5.0 version is unusable could someone please inform me. Otherwise, if anyone has a working python3 amd64 install package for 5.0, would you please let me know and put it someplace I can download it? Thanks!
Re: Disk blocking and unacceptable wait times with Areca ARC 1210
On 01/11/2012 05:12 PM, Ted Unangst wrote: On Wed, Jan 11, 2012, Chris Cappuccio wrote: If only one disk is affected at a time, 5.0 is the fastest, and has the most trouble with responsiveness while being fast, this is likely to be improved by a fair I/O scheduler. There is a generic framework in place now for schedulers to get plugged in I don't think anybody has actually written it yet. There's also an issue with dirty buffers getting eaten up, but that is prominent on slow devices, and you'd be WAITing in buf_needva in that case. I don't think needva has been totally ruled out from what I've seen, though it's less likely. My other guess is that the raid card itself prioritizes writes over reads leading to a backlog of read requests. I didn't follow the thread all the way back, so forgive me if this has been covered. I'm betting that the disk subsystem RAID controller combination are choking on queued metadata writes. Some of the questions are aimed at the user, and some at people who know the system code. User: Is the file system mounted with soft updates? Would the writes of the bit maps, inode and indirect blocks have piled up? Does turning off soft updates help? What is the block/cluster size? What is the stripe size and RAID configuration? RAIDs are really slow doing the required read-modify-write on small writes. The cacheing algorithm(s) in the cluster may be interfering with the metadata writes. When reading the file the first time when no metadata is cached, does the delay occur? If the file is opened in update mode so that no new allocation is done, does the delay occur? A trivial program might have to be written (C, Python, Perl, LISP, COBOL, whatever). Developers: Would the filesystem code write logically contiguous data blocks out of order? If so, that could trigger read-modify-writes as well. Has the soft update code changed to accumulate more metadata in core? I don't know if there's any utility which can capture data about the types of data in the disk queues. That would rule this out. Again, if this has been covered, just ignore me. Geoff Steckel
Re: copyout from physio, why?
On 12/28/2011 09:08 PM, gwes wrote:
I hope someone can shed some light on this.
I'm running 5.0-current on an AMD64 with 4GB of physical memory.
Reading large chunks (64K or multiples) from /dev/rsd0c using
the AMD chipset SATA controller and a modern 1G drive:
time dd if=/dev/rsd3c of=/dev/null bs=128k count=1
1+0 records in
1+0 records out
131072 bytes transferred in 11.348 secs (115498831 bytes/sec)
0.0u 2.0s 0:11.34 17.9% 0+0k 0+1io 0pf+0w
Profiling the kernel shows that copyout() is being called from
physio() via uvm_vsunlock_device() for every MAXPHYS byte chunk.
On first inspection, physio calls uvm_vslock_device(...,map)
which checks to see if all pages in the request satisfy
PADDR_IS_DMA_REACHABLE(). If so, it returns NULL in map.
After strategy() returns, map is sent to uvm_vsunlock_device,
which calls copyout() if map != NULL.
There's a comment on uvm_vslock_device saying it always returns
something in *retp, but the code seems to indicate otherwise.
PADDR_IS_DMA_REACHABLE checks against dma_constraint, which
is 0..0x which should allow all memory 4G to be used
for DMA.
What have I missed?
I believe that the copyout() shouldn't happen.
I'm trying to run multiple 140MB/sec drives simultaneously and
the copyout() is a killer - it's eating more of the
system memory and CPU bandwidth than I'd like.
thanks
Geoff Steckel
Responding to myself - on further examination, if the system has
only 2G, DMA is done to user pages as expected without copy{in,out}.
Once past the evil 3G boundary, page physical addresses are
= 0x1, and the code hardwires a limit several places.
The ahci (at least) driver chokes and hangs if fed a physical address
over 4G.
I'd really like to help someone fix this or if given more clues on where
to look, I'll dig some more. The work I'm trying to do will probably
survive quite well in 2G.
Re: 1 Mpps router and OpenBSD?
I haven't looked for years, but in the dim dark past many NIC drivers made non-optimal hardware accesses which slowed them down quite a bit. Profiling helped find some of them. Exactly how many cycles of stall happen during a bus (PCI, PCIe, ISA, VME) reference depends on the CPU, but a wild guess is 600 or more on a 3GHz CPU. A write reference can cause barrier stalls as well. I was working with FreeBSD at the time. From casual glances the OpenBSD drivers *were* very similar in their bus access patterns. Typically a bus access was done in a per-packet loop which could be hoisted out of the loop. Some reads could be moved so that the data wasn't used for a while getting some parallelism. The standard optimizations for slow references. One or two truly egregious cases were inline spin waits for talking through the NIC to the net modem chip(s). Even longer ago on a 4.4BSD-based MP system, allowing interrupts on CPUs other than 0, great care taken to lock the absolute minimum data, etc. helped a lot. If interrupts had to be taken on CPU0, a task queue to transfer responsibility to the next free processor without an expensive CPU-CPU interrupt helped a great deal. If those have been fixed please accept my apologies to the developers. As I said, I haven't looked at NIC code since 3.9 or before. On 10/27/2011 03:16 PM, tx wrote: OK. I'm about special network tweaks or something like this. For example, for FreeBSD there is a sysctl tweaks that give zero CPU load on 100 Mbps traffic with PRO1000/PT NIC, and about 10% CPU load without these tweaks on same hardware and in same conditions.
Re: Question about apmd power savings
On 10/18/2011 02:53 PM, Joe S wrote: This isn't a problem and I'm not complaining, I'm just a bit curious as apmd didn't save me as much power as I hoped for. I noticed that apmd couldn't throttle my cpu in 4.9-RELEASE (amd64). However, since March 2011, -CURRENT recognizes the K10 cpus, so I wanted to try it out apmd on my HP Microserver. So I upgraded my system from 4.9-RELEASE to a recent -CURRENT snapshot. When I run apmd -C hw.setperf gets set to 0 and the hw.cpuspeed gets set to 800 MHz. That was expected. I attached a Kill-a-Watt meter to my system to see what kind of power savings I would experience with apmd enabled. I think my expectations were a bit too high. I was only saving .5 watts when my cpu was throttled down to 800 MHz from 1.3 GHz. I haven't seen many posts on this subject, so I was wondering if the power savings of .5 watts sounds normal, or if something is wrong on my end. Were you running a CPU-intensive workload on the CPU(s)? Changing the clock speed of an idle chip won't change the power usage very much in absolute terms. If the CPU has multiple cores, exercising them all at once may maximize power usage so check with all of them running hard. Memory chips also use more power when cycled, but if the CPU is stalled waiting for memory it may use less power, so the interaction is not a priori well defined. Geoff Steckel
Re: OBSD 4.7 and Via C7 motherboards problem
I've got a C7 board running 4.7 as my firewall. The configuration is a lot more baroque than yours... A couple of thoughts: Your pf.conf should only hold state on one side. Multiple conflicting state table entries for the same connection ensure flaky failures. I use quick wherever possible to eliminate hidden dependencies label entries on pf.conf rules can help show unexpected paths when testing, do before and after runs of netstat -ss pfctl -s labels pfctl -s state and diff them to check where packets are going Also tcpdump of pflog I.E. pass out quick log on $ext_if from ! ($ext_if) to any nat-to \ ($ext_if:0) label nat-rule pass out quick log on $ext_if all label ext-out pass out quick log on $int_if all flags any no-state label int-out pass in quick log on $ext_if all label ext-in pass in quick log on $int_if all flags any no-state label int-in This should show where things go. Geoff Steckel curmudgeon for hire My system: $ dmesg OpenBSD 4.7 (GENERIC) #558: Wed Mar 17 20:46:15 MDT 2010 dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC cpu0: VIA Esther processor 1500MHz (CentaurHauls 686-class) 1.51 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,APIC,SEP,MTRR,PGE,CMOV,PAT,CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,TM,SBF,SSE3,EST,TM2 real mem = 1005023232 (958MB) avail mem = 965070848 (920MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 05/16/06, BIOS32 rev. 0 @ 0xfb570, SMBIOS rev. 2.3 @ 0xf (34 entries) bios0: vendor Phoenix Technologies, LTD version 6.00 PG date 05/16/2006 apm0 at bios0: Power Management spec V1.2 (slowidle) apm0: AC on, battery charge unknown acpi at bios0 function 0x0 not configured pcibios0 at bios0: rev 2.1 @ 0xf/0xdc84 pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfdbb0/208 (11 entries) pcibios0: bad IRQ table checksum pcibios0: PCI BIOS has 11 Interrupt Routing table entries pcibios0: PCI Exclusive IRQs: 5 10 11 pcibios0: PCI Interrupt Router at 000:17:0 (VIA VT8237 ISA rev 0x00) pcibios0: PCI bus #1 is the last bus bios0: ROM list: 0xc/0xfe00 0xd/0x5000! cpu0 at mainbus0: (uniprocessor) cpu0: RNG AES AES-CTR SHA1 SHA256 RSA cpu0: unknown Enhanced SpeedStep CPU, msr 0x08100f1308000f13 cpu0: using only highest and lowest power states cpu0: Enhanced SpeedStep 1501 MHz: speeds: 1500, 800 MHz pci0 at mainbus0 bus 0: configuration mode 1 (bios) pchb0 at pci0 dev 0 function 0 VIA CN700 Host rev 0x00 viaagp0 at pchb0: v3 agp0 at viaagp0: aperture at 0xe800, size 0x1000 pchb1 at pci0 dev 0 function 1 VIA CN700 Host rev 0x00 pchb2 at pci0 dev 0 function 2 VIA CN700 Host rev 0x00 pchb3 at pci0 dev 0 function 3 VIA PT890 Host rev 0x00 pchb4 at pci0 dev 0 function 4 VIA CN700 Host rev 0x00 pchb5 at pci0 dev 0 function 7 VIA CN700 Host rev 0x00 ppb0 at pci0 dev 1 function 0 VIA VT8377 AGP rev 0x00 pci1 at ppb0 bus 1 vga1 at pci1 dev 0 function 0 VIA S3 Unichrome PRO IGP rev 0x01 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) skc0 at pci0 dev 8 function 0 D-Link Systems DGE-530T A1 rev 0x11, Yukon (0x1): irq 11 sk0 at skc0 port A: address 00:0d:88:c8:2b:c8 eephy0 at sk0 phy 0: 88E1011 Gigabit PHY, rev. 3 VIA VT6306 FireWire rev 0x80 at pci0 dev 10 function 0 not configured re0 at pci0 dev 11 function 0 Realtek 8169 rev 0x10: RTL8169/8110SCd (0x1800), irq 5, address 00:30:18:a8:10:76 rgephy0 at re0 phy 7: RTL8169S/8110S PHY, rev. 2 pciide0 at pci0 dev 15 function 0 VIA VT6420 SATA rev 0x80: DMA pciide0: using irq 11 for native-PCI interrupt wd0 at pciide0 channel 1 drive 0: HTS541080G9SA00 wd0: 16-sector PIO, LBA48, 76319MB, 156301488 sectors wd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 5 pciide1 at pci0 dev 15 function 1 VIA VT82C571 IDE rev 0x06: ATA133, channel 0 configured to compatibility, channel 1 configured to compatibility pciide1: channel 0 ignored (disabled) atapiscsi0 at pciide1 channel 1 drive 0 scsibus0 at atapiscsi0: 2 targets cd0 at scsibus0 targ 0 lun 0: TSSTcorp, CDW/DVD SH-M522C, TS01 ATAPI 5/cdrom removable cd0(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 2 uhci0 at pci0 dev 16 function 0 VIA VT83C572 USB rev 0x81: irq 10 uhci1 at pci0 dev 16 function 1 VIA VT83C572 USB rev 0x81: irq 10 uhci2 at pci0 dev 16 function 2 VIA VT83C572 USB rev 0x81: irq 11 uhci3 at pci0 dev 16 function 3 VIA VT83C572 USB rev 0x81: irq 11 ehci0 at pci0 dev 16 function 4 VIA VT6202 USB rev 0x86: irq 11 ehci0: timed out waiting for BIOS usb0 at ehci0: USB revision 2.0 uhub0 at usb0 VIA EHCI root hub rev 2.00/1.00 addr 1 viapm0 at pci0 dev 17 function 0 VIA VT8237 ISA rev 0x00 iic0 at viapm0 spdmem0 at iic0 addr 0x50: 1GB DDR2 SDRAM non-parity PC2-6400CL5 auvia0 at pci0 dev 17 function 5 VIA VT8233 AC97 rev 0x60: irq 11 ac97: codec id 0x56494170 (VIA Technologies VT1617) ac97: codec features headphone, 18 bit DAC, 18 bit ADC, KS Waves 3D audio0 at auvia0 vr0 at pci0 dev 18 function 0 VIA RhineII-2 rev 0x78: irq 10, address 00:30:18:a2:dd:0f ukphy0 at vr0
Re: OBSD 4.7 and Via C7 motherboards problem
On 08/08/2010 03:28 PM, Henning Brauer wrote: * Geoff Steckelg...@oat.com [2010-08-08 20:29]: Your pf.conf should only hold state on one side. Multiple conflicting state table entries for the same connection ensure flaky failures. that is wrong in so many ways. first, should only hold state on one side is bullshit advice. holding state on both sides is absolutely fine. wether it is a good idea depends on a number of factors. it never really hurts. second, these state table entries will never ever collide. i may recommend a read here: http://bulabula.org/papers/2009/eurobsdcon-faster_packets/ especially slides 40 to 50 I'm saying what has worked for me. The state code has changed a lot since I did my last big set of tests. If states are truly unified between input and output interfaces, then the correct objection is: States found on any interface are reused quickly on all interfaces The documentation is not terribly clear about that. . I'm still a bit dubious about handling late FINs and other legal packets which the older PF code needed extra help to dispose correctly. Getting back to the original question, perhaps skip on $int would simplify debugging even further? geoff steckel curmudgeon for hire
Re: HPING or equiv
Ok so here's the problem if I do a 'hping -c 1 -i u100 -1 xx.xx.xx.xx' I generate a rather unimpressive 50pps. Issuing the same command on a gentoo box (sorry) I get 9000+ pps. time sudo ping -f ping PING ping.oat.com (198.5.5.10): 56 data bytes --- ping.oat.com ping statistics --- 12180 packets transmitted, 12180 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 0.083/0.141/8.073/0.132 ms 0.0u 0.6s 0:02.75 24.3% 0+0k 0+1io 0pf+0w This is on a very wimpy box ( 2G). Maybe your network interface is no good? If it's an ne2000 type, it won't work worth used tissue paper. geoff steckel
Re: Hardware recommendation for firewalls (more than 4 NICs)
Jason Dixon wrote: On Fri, Jul 11, 2008 at 06:47:13PM -0300, Mart?n Coco wrote: Hi misc, I'm currently looking for hardware alternatives for firewalls that should have more than four NICs. Why could you possibly need 6 physical interfaces? Even if you have a failover pair of firewalls and switches, with a dedicated pfsync interface, you could get by easily with three interfaces. The first two interfaces are trunked, one to each switch. Use vlan(4) interfaces with carp(4) on top of that. Your third interface would crossover between firewalls for private pfsync traffic. H. Why would you ever want to do that? - really not a good thing to say to someone... Saying that means you lack respect for the person or lack imagination. What are you using them for is a better response. I've frequently used 5 ports on my firewall for multiple isolated subnets. I've had very good luck with any of a number of 4-port cards. Unfortunately, the good ones are no longer made. I'm using a 4-sf card which is available on the surplus market for $40 or so. The sf chip occasionally stops transmitting (maybe 2 or three times a week) but the driver (with the latest fixes) catches it. The 4-dc card is better but harder to find. Is there a requirement for low power or small form factor? geoff steckel
Re: Hardware recommendation for firewalls (more than 4 NICs)
I knew it was a matter of time before the vlan insecurity bullshit hit the fan. RTFA. Who says anything about blindly trusting switches? If you can't correctly configure VLANs on your switches, and filter on vlan(4) interfaces in PF, you shouldn't be administering production networks. There's nothing functionally different between: I've developed networks with over a dozen routed VLAN segments on a single physical GbE link. With carp(4) interfaces on top. It's easy. In fact, it's a hell of a lot less error- and failure-prone than managing 5 interfaces. If you're not going to use the features that came with those $5k switches you just bought, you might as well stick with $100 Netgears from Best Buy. Oh dear gracious goodness me. $5K switches Can I sell you a few? Or tell me what brand you buy so I can buy stock? And who is your power company so I can buy stock? And who is your landlord so I can buy shares? I'm sorry, but my application doesn't seem to bear any resemblance to yours. Certainly my constraints are very different. Oh well Please research the archives and contemplate my rant on engineering a few months ago. I'll shut up now.
Re: UDP reception with minimal overhead
Markus wrote: Good evening, I'm setting off for writing prototype code for an imaging application. For this reason, I'm in need of an extremely fast way to transport large amounts of UDP data to a userland application. A socket implementation does not perform very well, which is the reason why I'm looking for a better solution to minimize processing time. My initial idea was to already filter the traffic with bpf and only hand the datagrams (or parts of them) I really want out of kernel space. Hi, Could you forward or post the profile data that shows the hot spot(s) in the kernel? It would be very instructive. thanks geoff steckel
Re: UDP reception with minimal overhead
Markus wrote: On Wed, 18 15:13 , Geoff Steckel wrote: Markus wrote: Good evening, I'm setting off for writing prototype code for an imaging application. For this reason, I'm in need of an extremely fast way to transport large amounts of UDP data to a userland application. A socket implementation does not perform very well, which is the reason why I'm looking for a better solution to minimize processing time. My initial idea was to already filter the traffic with bpf and only hand the datagrams (or parts of them) I really want out of kernel space. Hi, Could you forward or post the profile data that shows the hot spot(s) in the kernel? It would be very instructive. thanks geoff steckel Hi Geoff, this sounds as if you would prefer sockets if possible. Is there a reason why you would reject using bpf directly? A couple of reasons, really... Most important, and overshadowing all the other considerations: It is very likely that the application will incur more overhead creating graphics than it will receiving data from the network. Graphics are very compute-intensive. Therefore, optimizing the network part of the application is very likely a waste of effort. If you have data showing that the graphics are cheap and the network is expensive, then optimizing the network makes sense. Otherwise, my advice would be to build the application as simply and cleanly as possible, profile it, and work on the hot spots which are probably in the graphics code. Other reasons, less important: Sockets are far more portable than BPF, even from OS version to version, much less from OS to OS. A long time ago and far away, I did a -lot- of performance tuning with a group making network appliances. We found that the device drivers were as likely to be the bottleneck as was the socket code. It's moderately unlikely that there would be a large performance advantage to BPF, though I haven't tested it recently. The one place where BPF might be better is the ability to receive more than one packet per kernel/user transition. And if there is a really bad hot spot or spots in the socket code, it would be a big favor to everyone if it were exposed for discussion and possible improvement. I guess for testing the socket approach you could pretty much take any UDP client and throw datagrams on it over a gigabit link (load will be around 30 fps with 3 MB each image frame + headers). The client system does no disk I/O btw, it's just getting the data from the socket and, in its simplest form, hands it over to a display. Unfortunately I got neither some decent computer hardware here at my flat nor a device that could pump out the amount of packets I want to produce via eg. tcpreplay (or simply a camera). I will however try to get some data before I'm heading off for travelling 4 weeks this weekend if you're interested in it. Thanks very much! I would note that even if you do not have hardware sufficient to saturate a system, even 10-20% of saturation would give useful information. Now, as a very extreme example of optimization: If the socket approach doesn't work well for you, and you are feeling very masochistic, have infinite development time, and are willing to survive large heavy objects thrown at you by all sane developers, there is a (insert LARGE skull-and-crossbones here) utterly non-portable and unclean method, which I have tried and can work: map the kernel socket buffer space into the application take data directly from the kernel memory MBUF chains add an IOCTL to acknowledge reading and wait for more data for persons totally beyond reason: add a semaphore-like object in writable memory shared with the kernel which signals data consumed and data available without blocking or forcing a transition to kernel mode from user mode This can eliminate -all- kernel/user data copying, a large number of context switches especially under heavy load, and is about as efficient as possible. This approach can (approximately) halve the kernel overhead for an application processing packets. It is only useful if the amount of CPU per packet is almost nothing. I hope this is helpful. geoff steckel
Re: remove any unwanted devices from the kernel.
Sometimes it matters to be small and sometimes fast. That is a decision made by the kernel hacker. Joe user does not make these decisions because he/she does not understand the overall impact. As someone else who writes code for this fine os would say: removing drivers is pure masturbation. Oh dear I think that ye all do protest far too much. Threats of unspecified system instability are hard to believe. I strongly suggest that anyone desiring to optimize things should be pointed at tools to monitor and profile system performance before doing any changes at all, not just kernel mangling. Hacking out chunks of kernel code because it isn't used is clearly a bad idea. On the other hand, removing unused I/O device drivers has been done for many decades in many O/Ss. It's not a task for the uninitiated but it should not be fraught with danger. I suggest that there are reasonable cases where a non-core-team person would correctly want to remove unused drivers. For systems which must boot very quickly, removing unused drivers whose probe routines cause significant timeouts can make a big difference. Sometimes timeouts are the only way to check for an I/O device behind a blind interface. For instance, checking a floppy drive's seek time is a significant wait. For systems which are intended to run with little memory or which are straining at architectural limits, 100K here and 100K there can make quite a big difference in what applications can run. Many drivers are over 150K when linked. When a megabyte or two counts, removing 10 drivers could make a big difference. If the kernel code is well structured, the following must be true: Removing a driver which is essential to normal operation must cause the kernel compile or link stage to fail. Secret, undocumented dependencies (neither in the man pages or in the source code comments) between apparently unrelated modules are very serious bugs. - The x86 architecture and I/O devices are seriously flawed. There are cases where it is not obvious which driver is the correct one for a peripheral, and the order in which devices are probed is critical. Self identifying devices such as on the PCI bus are mostly immune from this problem, and the kernel is quite good about announcing a PCI device for which no driver is found. Still, ISA devices can be a minefield. While removing an ISA driver has potential to recover the most space or time, it's the hardest to verify. Unfortunately the biggest payback is often from removing drivers intended for systems not made in 10 years or more. - As an aside, I've been wondering what the heck named is doing to initialize itself. It does many thousands of disk accesses for no visible benefit and takes a very long time to do them. curmudgeonly geoff steckel
Re: remove any unwanted devices from the kernel.
On Fri, 6 Jun 2008 10:14:55 -0400 Ted Unangst [EMAIL PROTECTED] wrote On 6/6/08, Geoff Steckel [EMAIL PROTECTED] wrote: For systems which must boot very quickly, removing unused drivers whose probe routines cause significant timeouts can make a big difference. Sometimes timeouts are the only way to check for an I/O device behind a blind interface. For instance, checking a floppy drive's seek time is a significant wait. introducing config. You're compiling your own kernel, so you must have read the man page for config, right? So you know there's no reason to actually compile anything, right? Yes, there is. Applying config multiple times to a built system is much harder to maintain over versions and releases than tracking cvs of GENERIC, files.xxx, etc. and applying diff3/merging, which is scriptable. For systems which are intended to run with little memory or which are straining at architectural limits, 100K here and 100K there can make quite a big difference in what applications can run. Many drivers are over 150K when linked. When a megabyte or two counts, removing 10 drivers could make a big difference. Given the cost of hardware today, the fraction of total memory that one megabyte represents, and the nature of any job that needs all your memory, by the time you finish compiling that kernel it's going to need two megabytes more, not just one. And then what? Hm... Maybe you should look up the concept transaction cost. Changing hardware, especially multiple installations, can be expensive, disallowed, or impossible. In those cases the cost of adding memory is essentially infinite. Some platforms cannot add memory at any cost. The advantage of being able to run even for a few months more on existing hardware can be very large. The advantage of using a smaller hardware platform and saving (say) two watts of power can multiply to hundreds of dollars or even the possibility of a project at all. If the kernel code is well structured, the following must be true: Removing a driver which is essential to normal operation must cause the kernel compile or link stage to fail. What compiler are you using that can detect your system's hardware for you? I think we had already determined that the user removing drivers is reasonably well informed, especially about the details of the platforms to use the kernel. I'm speaking to any entanglements between drivers or with apparently hardware-independent functionality, etc. geoff steckel
small, random essay on performance tuning, was: remove....
The people reading the faq are not the people who need custom kernels. Those people *know* what they need and are not deterred. But as always, when we try to help the userbase by offering the advice they need, someone needs to chime in and muddy the waters. So now some dude is going to read your email and conclude that he can get better mysql speed on his 1gb box by shaving a driver. You're not doing anyone any favors. And no, we cannot assume that people correctly self identify when they are special. Ignorant people don't know they are ignorant. But at least some of them read the faq. So at least let them get the best answer for them. I agree that a user without a good reason shouldn't mess with kernel modification. The default settings as provided are very usable in -almost- every situation. A factual essay with some detail about better approaches to improving system performance might be the following. Of course, it should be edited appropriately if it were to be used, examples added or deleted, and any good warnings added to the what not to do section. (this is cc: to tech@ as a possible extension of the FAQ, etc, if deemed useful) html body h2Improving system performance/h2 pThe best approach is: h3Inspect your system/h3 Intelligently observe what your system is doing using the many monitoring and profiling utilities provided with the system to see where time is being spent and what resources are insufficient. Examples of these utilities are: ps, vmstat, iostat, gprof, etc. h3Correct wasteful applications/h3 pIn most cases, the system monitoring utilities will show a hot spot - one program is using far more resources than it should. Very often the same work can be done using a better algorithm, a more efficient utility, or work can be removed without affecting the desired results. h3Correct kernel usage/h3 p The kernel itself is almost never the problem: if it appears so, the applications you are running are issuing large numbers of syscalls or other behavior which causes the kernel to do work. Correct that behavior before blaming the kernel. codektrace/code is an example of a monitoring program which will record all interaction between a program and the kernel. h3Upgrade I/O devices/h3 p If your applications are being starved and monitoring shows either large interrupt times in the kernel or large idle times when applications should be running, look at your peripherals. Old or inefficient I/O devices can make your system run slowly because the kernel must do very large amounts of work to make them run or user applications must wait a very long time for data. Examples are: NE2000-style or other non-DMA network interfaces, slow disks or disk controllers which don't do DMA, USB 1.0 devices, etc., etc. A newer network card or disk is often the most cost-effective way to increase performance by large amounts for very little money. h3Improve memory use or add memory/h3 p If your system is paging excessively you need to use memory better. p Most modern systems have many megabytes of memory. The kernel only needs a few [a minimum number, like 8?] megabytes to run. All the other memory is available for applications and disk buffers. If memory demand is too high, profile and monitor your system for applications hogging memory. h3User environment tuning/h3 pThere are many ways to make applications run faster which are not necessarily obvious. These are examples which have been useful in the past: pIf programs referencing temporary files are slow, placing code/tmp/code and code/var/tmp/code on drives other than the one with user data being used can sometimes give large speedups. Directing small, transient temporary files to a memory file system, or piping data between applications on multiprocessor or multicore systems can sometimes greatly increase performance. pIf disk I/O is a bottleneck reading very large files, increasing the cluster size of the file system where the large files reside can improve performance, as long as there is sufficient physical memory to hold large disk buffers and the increase in wasted space for small files is acceptable. This requires backing up the file system, destroying it, and recreating it with the new parameters. Be absolutely sure data are backed up and recoverable before doing this! h3Kernel tuning/h3 pThere are a number of kernel parameters which can be tuned if your monitoring shows very specific performance bottlenecks. For instance, network performance with long fast pipes can sometimes be improved by increasing the default socket buffer allocations. Other networking parameters such as TCP SACK window size can be tuned if you are very familiar with TCP low-level details. NFS performance can sometimes be improved by increasing the number of NFS I/O threads. p These should not be done unless you can show why with network statistics and tcpdump
fsck_ffs algorithms
For really large file systems or for small memory machines, the algorithms in fsck_ffs will inevitably run out of memory. It appears that there are two possibilities to deal with this: 1) multiple sub-passes over the file system, dealing with parts of the data each time 2) modifying the ffs on-disk structure to add enough space reserved for fsck to store needed information during its run There may be more, but I believe that in the long run something on the order of 1 or 2 will be necessary. I have some ideas about how to implement either one if there's any interest. Otherwise it'll wait until the indefinite future. geoff steckel
fsck_ffs for large file systems
This is a patch to 4.3 release fsck_ffs to reduce the block map
memory usage in almost all cases. It uses a sparse representation
where regions of all zeros or all ones require no memory.
In the worst case (every region contains both ones and zeros)
it increases memory usage by less than 2%. In the best case
it reduces memory usage for the block map by 98% or so.
CPU usage is increased slightly. Since fsck is very disk-bound
I believe this will not be a problem.
This is a VERY preliminary version. It has been tested on the
few large filesystems ( 30G) I have. I do not assert that
it is acceptable for production in either format or content.
It contains debug code, #if 0, #if 1, and other constructs
which would not be present in a production version.
I would like people to try it and see how badly it fails.
Please send me any failure information and I will attempt to fix the problem.
Thanks very much.
geoff steckel
diff -Pupr /deep/4.3/src/sbin/fsck_ffs/Makefile fsck_ffs/Makefile
--- /deep/4.3/src/sbin/fsck_ffs/MakefileSun Sep 21 07:36:37 1997
+++ fsck_ffs/Makefile Mon May 19 15:08:41 2008
@@ -3,7 +3,7 @@
PROG= fsck_ffs
MAN= fsck_ffs.8
SRCS= dir.c inode.c main.c pass1.c pass1b.c pass2.c pass3.c pass4.c \
- pass5.c fsutil.c setup.c utilities.c ffs_subr.c ffs_tables.c
+ pass5.c fsutil.c setup.c utilities.c ffs_subr.c ffs_tables.c blockmap.c
.PATH: ${.CURDIR}/../../sys/ufs/ffs ${.CURDIR}/../fsck
CFLAGS+= -I${.CURDIR}/../fsck
diff -Pupr /deep/4.3/src/sbin/fsck_ffs/blockmap.c fsck_ffs/blockmap.c
--- /deep/4.3/src/sbin/fsck_ffs/blockmap.c Wed Dec 31 19:00:00 1969
+++ fsck_ffs/blockmap.c Mon May 19 17:45:51 2008
@@ -0,0 +1,133 @@
+/*
+ * Copyright (c) 2008 Geoff Steckel. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ * 3. The names of the contributors
+ *may be used to endorse or promote products derived from this software
+ *without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#define DKTYPENAMES
+#include sys/param.h
+#include sys/time.h
+#include ufs/ufs/dinode.h
+#include ufs/ffs/fs.h
+#include sys/stat.h
+#include sys/ioctl.h
+#include sys/disklabel.h
+
+#include errno.h
+#include fcntl.h
+#include stdio.h
+#include stdlib.h
+#include string.h
+#include ctype.h
+
+#include fsck.h
+#include extern.h
+#include fsutil.h
+
+#define ERR1 tsetbit couldn't enter a new item in blockmap\n
+#define ERR2 tclrbit couldn't find an item in blockmap\n
+
+void
+blkmapset(struct blockmapchunk **blockmap, daddr64_t blkno)
+{
+ struct blockmapchunk *thischunk;
+ daddr64_t chunkno;
+ unsigned int ofsinchunk;
+
+ if (blkno = maxfsblock + 7) {
+ printf(blkmapset called with block %lld out of range\n,
blkno);
+ return;
+ }
+ chunkno = blkno / BLKMAPCHUNK;
+ ofsinchunk = blkno % BLKMAPCHUNK;
+ thischunk = blockmap[chunkno];
+ if ( ! thischunk) {
+ thischunk = calloc(1, sizeof *thischunk);
+ if ( ! thischunk) {
+ printf(blkmapset can't alloc for block %lld\n, blkno);
+ return;
+ }
+ blockmap[chunkno] = thischunk;
+ }
+ setbit(thischunk-bmc_bits, ofsinchunk);
+ thischunk-bmc_count++;
+ if (thischunk-bmc_count == BLKMAPCHUNK) {
+ free(thischunk);
+ blockmap[chunkno] = alloneschunk;
+ }
+}
+
+int
+blkmaptest(struct blockmapchunk **blockmap, daddr64_t blkno)
+{
+ struct blockmapchunk *thischunk;
+ daddr64_t chunkno;
+ unsigned int ofsinchunk;
+
+ if (blkno = maxfsblock + 7) {
+ printf(blkmaptest called with block %lld out of range\n,
blkno);
+ return 0
AMD cpuid decode question
I'm leaving for a vacation in the morning so this is in lieu of anything more systematic like filing bug reports. I have just spent a couple of days trying to get a AMD Athlon5600X2 on an Elitegroup AMD690GM-M2 working. Getting the MP kernel to boot took a lot of debugging. a) The arch/i386/machdep.c cpuid decode in identifycpu() tries to be all things to all cpus. It doesn't follow the spec in AMD docs 25481 and 33610 particularly decoding the extended family and extended model fields which become significant when the family field == 0xF. First, does this matter? Probably not right now, as long as the amd_family6_setup is adequate for all newer CPUs. The algorithms for assigning ICUs to CPUs might suffer, but I really don't know. The documents noted above list the cpuid signatures of all recent chips, so a simple enumeration would probably suffice with defaults for the major families. b) x86 BIOSes are quirky at best. The BIOS on the board mentioned 1) puts the RDSP pointer in type 2 BIOS space (reserved) not in type 3 BIOS space (ACPI) so the search algorithm in acpi_machdep.c doesn't find it causing a crash later. Luckily acpidump(8) does a brute force search and does find it. It appears that the behavior of the BIOS is legal. 2) the RDSP block may advertise itself as revision 2 but have a NULL xdsp physical pointer (64 bit) while having a correct 32 bit pointer. The code assumes that a revision 2 RDSP must have a valid 64 bit pointer which causes a crash. c) acpi.c assumes that there must be an RDSP block so it uses whatever NULL happens to result from the acpi_machdep scan. d) i386/bios.c smbios_find_table's cookie code for finding cached entries can't work right - 0xfff mask instead of 0xff. The encoding (+1 or +2 base?) is unclear. It's likely that the cacheing feature is never used. e) There are a number of almost-duplicate temporarily map a block of physical space into kernel space routines scattered about multiple architectures and multiple places in some architectures which probably should be (garbage) collected. If these are known issues there's no point in me going any further. If any of these are new issues I'll file a bug fix after I get back. I'll be glad to supply diffs now. They are very full of #ifdef BIOS_DEBUG printfs. Amazing what you'll find out when you ask the machine to tell you what's happening. -- don't read any further if you are easily annoyed -- I realize that a lot of the code I was fighting with is inherited from other projects and was not written by OpenBSD people, so this is not a criticism of the people or the project. This experience has validated once more some rules developed over my 40+ years of programming: All should never happen failures must have printfs at minimum to clue the helpless about where the can't happen happened. No unusual or unexpected failure may be silent. A lack of checks for null pointers before use has a high debugging cost. As the certified geniuses who invented the ARPAnet said: be extremely precise sending out be extremely accepting receiving BUT validate, validate, validate Combining cacheing, 'find first', and 'find next' code in the same routine is in my very rigid and uncompromising mind a fatal design error of the genus conflation of similar but incompatible goals. geoff steckel Omnivore Technology
any mvme68k users? want lpt?
Are there any people using OpenBSD on MVME68K platforms? I just was tinkering with the parallel port on a MVME167 and got it working. If there's any interest, I'll think about constructing something - no guarantees how soon. geoff steckel
Re: jetway board sensors (Fintek F71805F)
Theo de Raadt wrote: You really should show a dmesg of your machine. sure: Jan 10 21:54:31 lib /bsd: OpenBSD 4.2-current (fins) #11: Thu Jan 10 21:29:15 EST 2008 Jan 10 21:54:31 lib /bsd: [EMAIL PROTECTED]:/doot/4.2snap/src/sys/arch/i386/compile/fins Jan 10 21:54:31 lib /bsd: cpu0: VIA Esther processor 1500MHz (CentaurHauls 686-class) 1.51 GHz Jan 10 21:54:31 lib /bsd: cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,APIC,SEP,MTRR,PGE,CMOV,PAT,CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,TM,SBF,SSE3,EST,TM2 Jan 10 21:54:31 lib /bsd: cpu0: RNG AES AES-CTR SHA1 SHA256 RSA Jan 10 21:54:31 lib /bsd: real mem = 1005023232 (958MB) Jan 10 21:54:31 lib /bsd: avail mem = 963747840 (919MB) Jan 10 21:54:31 lib /bsd: mainbus0 at root Jan 10 21:54:31 lib /bsd: bios0 at mainbus0: AT/286+ BIOS, date 05/16/06, BIOS32 rev. 0 @ 0xfb570, SMBIOS rev. 2.3 @ 0xf (34 entries) Jan 10 21:54:31 lib /bsd: bios0: vendor Phoenix Technologies, LTD version 6.00 PG date 05/16/2006 Jan 10 21:54:31 lib /bsd: apm0 at bios0: Power Management spec V1.2 (slowidle) Jan 10 21:54:31 lib /bsd: apm0: AC on, battery charge unknown Jan 10 21:54:31 lib /bsd: acpi at bios0 function 0x0 not configured Jan 10 21:54:31 lib /bsd: pcibios0 at bios0: rev 2.1 @ 0xf/0xdc84 Jan 10 21:54:31 lib /bsd: pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfdbb0/208 (11 entries) Jan 10 21:54:31 lib /bsd: pcibios0: bad IRQ table checksum Jan 10 21:54:31 lib /bsd: pcibios0: PCI BIOS has 11 Interrupt Routing table entries Jan 10 21:54:31 lib /bsd: pcibios0: PCI Exclusive IRQs: 5 10 11 Jan 10 21:54:31 lib /bsd: pcibios0: PCI Interrupt Router at 000:17:0 (VIA VT8237 ISA rev 0x00) Jan 10 21:54:31 lib /bsd: pcibios0: PCI bus #1 is the last bus Jan 10 21:54:31 lib /bsd: bios0: ROM list: 0xc/0xfe00 0xd/0x5000! Jan 10 21:54:31 lib /bsd: cpu0 at mainbus0 Jan 10 21:54:31 lib /bsd: cpu0: unknown Enhanced SpeedStep CPU, msr 0x08100f1308000f13 Jan 10 21:54:31 lib /bsd: cpu0: using only highest and lowest power states Jan 10 21:54:31 lib /bsd: cpu0: Enhanced SpeedStep 1500 MHz (1004 mV): speeds: 1500, 800 MHz Jan 10 21:54:31 lib /bsd: pci0 at mainbus0 bus 0: configuration mode 1 (no bios) Jan 10 21:54:31 lib /bsd: pchb0 at pci0 dev 0 function 0 VIA CN700 Host rev 0x00 Jan 10 21:54:31 lib /bsd: agp0 at pchb0: v3, aperture at 0xe800, size 0x1000 Jan 10 21:54:31 lib /bsd: pchb1 at pci0 dev 0 function 1 VIA CN700 Host rev 0x00 Jan 10 21:54:31 lib /bsd: pchb2 at pci0 dev 0 function 2 VIA CN700 Host rev 0x00 Jan 10 21:54:31 lib /bsd: pchb3 at pci0 dev 0 function 3 VIA PT890 Host rev 0x00 Jan 10 21:54:31 lib /bsd: pchb4 at pci0 dev 0 function 4 VIA CN700 Host rev 0x00 Jan 10 21:54:31 lib /bsd: pchb5 at pci0 dev 0 function 7 VIA CN700 Host rev 0x00 Jan 10 21:54:31 lib /bsd: ppb0 at pci0 dev 1 function 0 VIA VT8377 AGP rev 0x00 Jan 10 21:54:31 lib /bsd: pci1 at ppb0 bus 1 Jan 10 21:54:31 lib /bsd: vga1 at pci1 dev 0 function 0 VIA S3 Unichrome PRO IGP rev 0x01 Jan 10 21:54:31 lib /bsd: wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) Jan 10 21:54:31 lib /bsd: wsdisplay0: screen 1-5 added (80x25, vt100 emulation) Jan 10 21:54:31 lib /bsd: re0 at pci0 dev 8 function 0 Realtek 8169 rev 0x10: RTL8169S (0x0400), irq 11, address 00:08:54:d1:c7:eb Jan 10 21:54:31 lib /bsd: rgephy0 at re0 phy 7: RTL8169S/8110S PHY, rev. 0 Jan 10 21:54:31 lib /bsd: VIA VT6306 FireWire rev 0x80 at pci0 dev 10 function 0 not configured Jan 10 21:54:31 lib /bsd: pciide0 at pci0 dev 15 function 0 VIA VT6420 SATA rev 0x80: DMA Jan 10 21:54:31 lib /bsd: pciide0: using irq 11 for native-PCI interrupt Jan 10 21:54:31 lib /bsd: wd0 at pciide0 channel 1 drive 0: Maxtor 6G160E0 Jan 10 21:54:31 lib /bsd: wd0: 16-sector PIO, LBA48, 152627MB, 312581808 sectors Jan 10 21:54:31 lib /bsd: wd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 5 Jan 10 21:54:31 lib /bsd: pciide1 at pci0 dev 15 function 1 VIA VT82C571 IDE rev 0x06: ATA133, channel 0 configured to compatibility, channel 1 configured to compatibility Jan 10 21:54:31 lib /bsd: wd1 at pciide1 channel 0 drive 0: ST380011A Jan 10 21:54:31 lib /bsd: wd1: 16-sector PIO, LBA48, 76319MB, 156301488 sectors Jan 10 21:54:31 lib /bsd: wd1(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 5 Jan 10 21:54:31 lib /bsd: pciide1: channel 1 disabled (no drives) Jan 10 21:54:31 lib /bsd: uhci0 at pci0 dev 16 function 0 VIA VT83C572 USB rev 0x81: irq 10 Jan 10 21:54:31 lib /bsd: uhci1 at pci0 dev 16 function 1 VIA VT83C572 USB rev 0x81: irq 10 Jan 10 21:54:31 lib /bsd: uhci2 at pci0 dev 16 function 2 VIA VT83C572 USB rev 0x81: irq 11 Jan 10 21:54:31 lib /bsd: uhci3 at pci0 dev 16 function 3 VIA VT83C572 USB rev 0x81: irq 11 Jan 10 21:54:31 lib /bsd: ehci0 at pci0 dev 16 function 4 VIA VT6202 USB rev 0x86: irq 5 Jan 10 21:54:31 lib /bsd: ehci0: timed out waiting for BIOS Jan 10 21:54:31 lib /bsd: usb0 at ehci0: USB revision 2.0 Jan 10 21:54:31 lib /bsd: uhub0 at usb0 VIA EHCI root hub rev 2.00/1.00 addr 1 Jan 10 21:54:31 lib /bsd: viapm0 at
jetway board sensors (Fintek F71805F)
Mr. Bihlmaier mentioned that there is no support for the sensors on the Jetway J7F2 boards. I have written a driver for the Fintek F71805F found on some of those boards. It is a modification of the LM78 driver (lm78.c) a href=http://www.oat.com/fintek;here/a. Several people have used it in 4.2. Since lm78.c hasn't changed for 4.3, this shouldn't need to either. I do not assert that the code is in a format acceptable to the OpenBSD team. It appears to work and have no significant failings beyond those already present in lm78.c
Re: jetway board sensors (Fintek F71805F)
bofh wrote: On Wed, Mar 12, 2008 at 8:45 PM, Geoff Steckel [EMAIL PROTECTED] wrote: Mr. Bihlmaier mentioned that there is no support for the sensors on the Jetway J7F2 boards. I have written a driver for the Fintek F71805F found on some of those boards. It is a modification of the LM78 driver (lm78.c) a href=http://www.oat.com/fintek;here/a. Several people have used it in 4.2. Since lm78.c hasn't changed for 4.3, this shouldn't need to either. I do not assert that the code is in a format acceptable to the OpenBSD team. It appears to work and have no significant failings beyond those already present in lm78.c It's hard to look at codes that are 404 compliant... :) Hmmm... that's true :-( try a href=http://www.oat.com/ot/fintek/;http://www.oat.com/ot/fintek//a instead. That might get something in the 200s.
Re: take threads off the table
Artur Grabowski wrote: Geoff Steckel [EMAIL PROTECTED] writes: Any argument to experience must be from similar actual implementations using threads and another model, such as multiple processes with interprocess communications. Sure. I'll pick up the challenge. At work we have a server that uses around 4GB RAM and runs on an 4 cpu machine. It serves millions of tcp connections per hour. sharing the memory without sharing pointer values is too inefficient since a big amount of the memory used is a pre-computed cache of most common query results. The service needs 4x4GB of RAM on the machine to be able to reload the data efficiently without hitting disk, since hitting the disk kills performance in critical moments and leads to inconsistencies between the four machines that run identical instances of this service. Therefore: - fork would not work because cache would not be shared and this would lead to too big cache miss ratio. - adding more RAM won't work because it would spend rack real estate and power and cooling budget which we can't do. - adding more machines will not solve the problem for the same reasons as RAM. - reducing the data set will not work because we kinda like to make lots of money, not just a little money. - partitioning the data does not work good because it causes a too high cost in performance and memory consumption. What works is threads. We've had one thread related bug in the past year. Art, It sounds like your application is pretty reasonable. The benefits of much cash, the restrictions on what hardware can be used, and your willingness to keep the project under control make a big difference in the cost-benefit balance. I can think of one thing that might have made a difference: it's possible under most unix-style OSs to share memory at a fixed address I'm not entirely sure that how much of your database stays in the cache except possibly some of the root, but I hope you've got the tools to know that. Still, you're pushing the envelope very hard to get as much performance and you -need- the performance, and even a percent or two of performance matters your application is SIMD-like in the large you've considered the tradeoffs and accept the risk for the benefits And I infer from what you say: It sounds like most queries are read-only, so they do not affect any shared state, therefore locking issues are relatively few. It also sounds like the application itself is relatively static (or at least the query engine is). The programming team is relatively static due to large $$$ rewards I'm assuming that the query engine is well separated in the code from code which changes due to changes in the data being served All of this taken together puts this into an area where I'm willing to agree that threads are an acceptable solution if not a desirable one. If any of the points above were different (complex state changes, didn't need 100+%, not read-only, not static code, many hands changing on the engine code) I'd disagree. On a very superficial consideration of what you've said, I suspect I could get a multiprocess solution to come within a few percent of the threaded one, but you say you need that last few percent. There are a lot of possible memory architecture issues (4 x 4 GB memory gets me wondering about its exact physical layout and bus architecture). A form of pipelined processing might also partition well, but I don't know any details of what you're doing. Depending very much on the exact situation offloading the TCP handshaking onto the processors in GBit network cards ---might--- work - there are a lot of possible gotchas but ---if--- the cards are fast enough and have enough on-card memory, the payoff could be large of course, then the network cards would have all the threads in them! Good luck, and thanks for the useful example! geoff steckel
OT: supposed advantages of threads
This is my last posting on this, take heart. The threads advocates have never specified any advantages of a program written using that model (multiple execution points in a single image) over a multiple process model, assuming that parallelism is useful. If the purported advantage is access to shared data structures without explicit access mechanisms, let's say I strongly disagree that that is an advantage. It is a whole set of fatal bugs waiting to happen. Please enlighten me if there are any -other- qualities of this model which are supposed to be advantageous to the people paying for and using the programs. I count faster development as an advantage, increased maintenance (bugs) as a disadvantage. The second strongly outweighs the first. geoff steckel
Re: OT: supposed advantages of threads
David Higgs wrote: I'm not an advocate of threads, simply playing devil's advocate because you patently refuse to believe this is anything but a mutually exclusive proposition. Is it not possible to decompose a complex program into a well-secured multi-process program, where one process leverages threads to perform something simple, massively SIMD parallel, and easy to verify? Yes, it is possible, and the SIMD case is the one where I would gladly admit the possibility that threads might be both useful and safe. That process would have to be well isolated and well defined as a byproduct of the decomposition. I didn't want to bring up SIMD because it is rare that people apply it strictly enough to be safe in this context. Also, given NUMA, the SIMD model is not necessarily the most efficient. The limited number of numerical problems I've dealt with could have the data set factored such that sharing was not necessary to achieve good throughput, but I can easily believe that there are many useful problems where this cannot be done. As I've said before, cost vs. benefit. The benefit in this case (large amounts of data shared amongst a large number of CPUs where otherwise a prohibitive amount of IPC would be necessary) and a low cost as a result of the symmetry of the problem could make it a good solution, if and only if the other parts of the program were suitably decomposed. Thank you for bringing up this very specific point. geoff steckel
Re: OT: supposed advantages of threads
Gregg Reynolds wrote: On 2/18/08, Geoff Steckel [EMAIL PROTECTED] wrote: This is my last posting on this, take heart. Please enlighten me if there are any -other- http://acmqueue.com/modules.php?name=Contentpa=list_pages_issuesissue_id=26 See especially Software and the Concurrency Revolution. An articulate and very readable discussion of why it's hard to do threading /with current techniques/. But they stop short of banning the practice. Thanks for the reference. It is well worth noting how much space the article Software and the Concurrency Revolution puts towards discussing locking and other methods of data access sequencing and control. Controlling data access sharing is hard, and uncontrolled access sharing is predictably bad. I will admit skimming the issue in order to reply quickly. I did not see any discussion (though it should be and probably is in the text, just not in what I looked at) of the impact of complex memory architectures on efficiency and concurrency. That architecture can drastically change the optimum access paradigm for best speed. Also, something I also didn't see (though it probably is mentioned) hardware vendors are seriously considering asymmetrical multicore processors where there are significant differences in capabilities among the CPUs on a chip. This also can drive program architecture towards using as little memory space sharing as possible for highest speed in some cases. geoff steckel
Re: take threads off the table
Gregg Reynolds wrote: On 2/17/08, Marc Balmer [EMAIL PROTECTED] wrote: Geoff Steckel wrote: Threads or any other form of uncontrolled resource sharing are very bad ideas. that might be true for those that don't understand threads. for other it can be highly benefitial. Indeed, threads are bad strikes me as just plain silly. In fact, it's not even a technical issue; anybody who thinks it is is in for a rude surprise (like, zero market share) in a few short years. It's a threads is a particular programming model of multiple execution contexts in a (mostly) shared memory and (mostly) shared resource environment which is not cost-effective for producing reliable software. It is much easier to produce a reliable multithreaded application using multiple processes and an appropriate communication method for sharing the specific data needed. Any counterargument must speak to advantages the threads model has over multiple processes or other protected multiple execution models versus the increased cost of construction and maintenance of a reliable program using the threads model. Any argument to experience must be from similar actual implementations using threads and another model, such as multiple processes with interprocess communications. A pandemic computer virus from Redmond exhibits bad interprocess switching behavior. That does not change the argument. geoff steckel
Re: take threads off the table
David Higgs wrote: On Feb 17, 2008 8:01 PM, Geoff Steckel [EMAIL PROTECTED] wrote: Gregg Reynolds wrote: On 2/17/08, Marc Balmer [EMAIL PROTECTED] wrote: Geoff Steckel wrote: Threads or any other form of uncontrolled resource sharing are very bad ideas. that might be true for those that don't understand threads. for other it can be highly benefitial. Indeed, threads are bad strikes me as just plain silly. In fact, it's not even a technical issue; anybody who thinks it is is in for a rude surprise (like, zero market share) in a few short years. It's a threads is a particular programming model of multiple execution contexts in a (mostly) shared memory and (mostly) shared resource environment which is not cost-effective for producing reliable software. Assuming that a software program is not system-critical or requires high security, and it benefits greatly from a shared memory/resource model, I fail to see why threading can not be cost-effective. May I interpret this as saying Threads are cost-effective in programs which don't have to be reliable? If so, what you're saying is irrelevant to the argument I'm making. As an aside, I prefer all programs I use to be as reliable as possible. It is much easier to produce a reliable multithreaded application using multiple processes and an appropriate communication method for sharing the specific data needed. Easier is hard to quantify when writing software. I've written several programs with MPI that could have just as easily been written with threads or fork()ed processes. IMHO, some tasks lend themselves more easily towards threading than others, and others to multiple processes. The only programs which I can see which could possibly justify threads are strict SIMD. Any other cases where threads are proposed as an optimal solution strongly suggest a very sloppy partitioning of data or functionality. Any counterargument must speak to advantages the threads model has over multiple processes or other protected multiple execution models versus the increased cost of construction and maintenance of a reliable program using the threads model. Any argument to experience must be from similar actual implementations using threads and another model, such as multiple processes with interprocess communications. Threads are hard to secure, this is commonly accepted as true. It does not logically follow that OpenBSD should entirely disallow the use of threads (or any other valid programming model) in software. That would significantly hinder adoption of OpenBSD for general workstation use. I would disallow userland threads in any program which is critical to security. The availability of threads to userland programs shouldn't hurt any program that doesn't use them. OpenBSD developers are unlikely to introduce threaded features to userland tools without a clean bill of health from code audits, even when the changes are trivial. Using 'goto' and 'snprintf' in C software are also considered potentially harmful and easy to misuse. But OpenBSD does not limit their use; they CAN be used correctly and without error. The same applies to threads: only when appropriate. Since you (presumably) use OpenBSD, I hope you trust developers to decide when threading is and is not appropriate. In C, goto cannot be used non-locally. C is vulnerable to buffer overruns. That requires vigilance. However, the scrutiny needed to ensure snprintf and the like are not a problem is also local. Threading is a non-local vulnerability. I trust the paranoia of the developers. It is well justified.
Re: Performance Issues of Intel Quad Port NIC
Jonathan Steel [EMAIL PROTECTED] wrote: Hi Everyone We recently purchased an Intel PRO/1000 GT Quad Port network card and decided to run some stress tests to make sure it could maintain gigabit connections on all four ports at the same time. I ran the test with five computers and iperf-1.7.0 (newest version of iperf has speed issues with OpenBSD). The computer with the NIC ran the iperf server in upd mode and the four other machines ran as iperf upd clients. The first test was using the GENERIC kernel 2 ports could sustain 900Mbit/s connections with about 30% packet drop rate 3 ports could sustain 900Mbit/s connections with about 80% packet drop rate 4 ports could sustain 900Mbit/s connections with about 95-99% packet drop rate The second test was using the GENERIC.MP kernel 2 ports could sustain 900Mbit/s connections with no dropped packets 3 ports could sustain 900Mbit/s connections with about 10% packet drop rate 4 ports could sustain 900Mbit/s connections with about 30-50% packet drop rate It would be really helpful if you gave us the before and after output of: netstat -ss -p ip netstat -ss -p tcp netstat -in netstat -m sysctl net and ran iostat 1 for a bit during your tests. netstat -ss tells things about queue drops netstat -in tells packet counts netstat -m tells things about mbuf usage sysctl net tells things about tcp and ip settings and some duplicate data about queues, etc. iostat 1 tells about CPU usage For instance, I suspect that the IP input queue limit is too low, as someone else mentioned. Using the PIC for interrupt vectoring is indeed slow. But we can't tell without at least some of the numbers and status above. geoff steckel
sf(4) - aic6915.c patch for infinite device timeout messages
I've observed that the sf(4) driver sometimes emits device timeout messages every few seconds forever.It appears that after some errors, the driver resets the chip without resetting the counter which triggers the timeout message. The following patch clears the output in progress counter in the code which stops output. in sf_stop: cvs diff -c aic6915.c Index: aic6915.c === RCS file: /cvs/src/sys/dev/ic/aic6915.c,v retrieving revision 1.3 diff -c -r1.3 aic6915.c *** aic6915.c 15 Dec 2006 15:28:27 - 1.3 --- aic6915.c 16 Jan 2008 14:35:07 - *** *** 1245,1250 --- 1245,1251 ds-ds_mbuf = NULL; } } + sc-sc_txpending = 0; if (disable) sf_rxdrain(sc);
Re: removing sendmail
Liviu Daia wrote: On 30 November 2007, Amarendra Godbole [EMAIL PROTECTED] wrote: Please note that postfix does not undergo the rigorous code scrub that sendmail goes through. [...] Will you please cut the crap? Thank you. Unlike Sendmail, Postfix was written from scratch with security in mind. It had only one published security flaw since its first public release in 1998. The author, Wietse Venema, is also the author of SATAN and tcpwrappers. He knew one or two things about writing secure code long before OpenBSD came into existence. The objections people occasionally have against Postfix are related to its license, not the code quality. I have seen several installations of Postfix go catatonic due to spam overload, large messages, mailing list expansions, and other undiagnosed problems. These were run by Postfix lovers, so I have always assumed that the installation was correct. In the one case I saw tested replacing Postfix with Sendmail resulted in no further problems. Given this anecdotal history I would suggest not running Postfix in a large production environment. geoff steckel
bridge.4 suggested clarification
This addition to the bridge(4) man page may make it a little easier for novices to use the combination of a bridge and pf. diff -u bridge.4.a bridge.4 --- share/man/man4/bridge.4.a Mon Oct 1 15:31:04 2007 +++ share/man/man4/bridge.4Mon Oct 1 15:36:54 2007 @@ -96,6 +96,9 @@ .Xr ip6 4 datagram; if so, the datagram is run through the pf interface so that it can be filtered. +The datagram is sent to pf as input on the incoming interface and as output +on all interfaces on which it is forwarded. +The pf functionality never sees a packet attributed to the bridge interface. .Sh IOCTLS A .Nm Geoff Steckel
Re: Security of the keyboard
Karel Kulhavy wrote: This kind of security design is assuming favourable constellation of uncontrollable environmental noises to scramble the information we are knowingly leaking. It's basically a snake oil. We have no proof that under every conceivable circumstances the noises will be present in a way that completely masks the information leak. Or, on exit to user mode, flush the caches, TLB, and branch target cache. This would kill any such leaks. Or, compile to PIC, and use a random number generator to reassign load addresses every time the kernel is loaded, and break up the kernel into chunks so the addresses used by the keyboard code are unknown. Or, for any process which is CPU bound, sample its instruction stream and if it is using instructions which probe CPU resources, kill the process. Or, who knows... there are probably dozens of ways to prevent leakage of this kind. It's still amazingly improbable that any information could usefully get through this way. Banging on the walls deliberately trying to transfer information from process to process can only send a few bits a second in most cases, which is why getting information about computationally intensive things like cryptography might work.
Re: Kernel interrupt timer?
Federico Giannici wrote: Geoff Steckel wrote: I worked on a commercial product based on altq on which a 1KHz clock was very useful. This used slow (400MHz) Pentium-class CPUs, and the increase in system overhead over a 100Hz clock was approximately 2%. Without the fast clock, accurately and consistently managing bandwidth down to 1% slices was difficult. We too have problems to obtain an accurate bandwidth management with many queues (hfsc) with low bandwidth percentages (sometimes a queue gets almost all bandwidth and other queues starve...) Do you think that increasing clock could help? The CPU is an Athlon 1.2 GHz, but CPU usage is quite low (85% idle time on average). Do you think a faster CPU could help? Thanks. Maybe. It depends on your link speed, packet sizes, and queue parameters. Getting good bandwidth management at low rates has problems if the time it takes to transmit a MSS packet (say 1500 bytes) is large relative to the time quota. On a T1 (1.5 megabit) link, a 1500 byte packet takes approximately 80 milliseconds to transmit. Depending on how the queue parameters are set, a 1% quota HFSC queue could get a lot more or a lot less. In this case, increasing the clock resolution won't help. On a 100Mbit link or higher, a fast clock is essential to get good queue discipline because there's no other way to reliably restart transmission on an idle link. Another factor which can upset queue discipline is the output buffer queue length, which if too long can make problems under some circumstances. I haven't looked at altq or hfsc code in years, so YMMV.
Re: Kernel interrupt timer?
Chris Kuethe wrote: On 5/29/07, Leon [EMAIL PROTECTED] wrote: Hi, I'm new to OpenBSD and I'm trying to setup a traffic shaping router using pf and altq. The question I want to ask is: Can the kernel interrupt timer be increased from 100 hz? and if so how do I do that? I though there would have been a sysctl tunable variable like kern.hz that could do this. I read somewhere that altq operates best when the clock interrupts are at 1000hz Where did you read that altq works better with a 1kHz clock - I have zero deployments of altq where I've found myself saying gosh, I wish I had finer timers. 100Hz works plenty good enough. I've seen otherwise capable machines be crippled by people who thought that 1kHz or faster was a good idea... Also, this hackathon we've been making pf (and the network stack in general) go faster by having fewer interrupts. So, yes, the clock rate can be increased. It is left as an exercise to the reader to do so. It is further left as an exercise to prove that this is desirable. CK I worked on a commercial product based on altq on which a 1KHz clock was very useful. This used slow (400MHz) Pentium-class CPUs, and the increase in system overhead over a 100Hz clock was approximately 2%. Without the fast clock, accurately and consistently managing bandwidth down to 1% slices was difficult. I'm sure the systems you saw which were crippled by a fast clock had some hidden configuration problems which if fixed could have reduced the overhead significantly. I agree that reducing the number of interrupts is almost always a good thing. If that reduction increases latency significantly it almost always makes system throughput worse and increases demand for buffers, etc. Reducing the number of external (PCI, etc.) bus references in drivers can make an astonishing speedup, sometimes 10% of total interrupt processing time per reference.
Fintek F71805 driver for test
I've mangled the lm78 driver into a Fintek F71805 sensor driver.
If anyone else has a board using this chip I'd appreciate a test of it.
The files are at http://www.oat.com/ot/fintek/
Here's the output from sysctl:
store:gwes {32} sysctl hw.sensors
hw.sensors.0=f71805f1, +3.3V, 3.30 V DC
hw.sensors.1=f71805f1, Vtt, 1.00 V DC
hw.sensors.2=f71805f1, Vram, 1.50 V DC
hw.sensors.3=f71805f1, Vchips, 1.60 V DC
hw.sensors.4=f71805f1, +5V, 5.09 V DC
hw.sensors.5=f71805f1, +12V, 11.97 V DC
hw.sensors.6=f71805f1, Vcc 1.5V, 1.10 V DC
hw.sensors.7=f71805f1, VCore, 1.08 V DC
hw.sensors.8=f71805f1, Vsb, 4.96 V DC
hw.sensors.9=f71805f1, Temp1, 38.00 degC
hw.sensors.10=f71805f1, Temp2, 33.00 degC
hw.sensors.12=f71805f1, Fan1, 6276 RPM
hw.sensors.13=f71805f1, Fan2, 366 RPM
hw.sensors.14=f71805f1, Fan3, 5376 RPM
store:gwes {33}
It seems to correlate pretty well with the BIOS display at boot time.
geoff steckel
fintek #if 0 code
The #if 0 code should be deleted. It's a copy of fins_isa_match code. In the ideal case the match would be done in fins.c using routines in fins_isa.c but there were no visible instances of non-isa chips. I'll blame my quick-and-dirty rework of the LM78 code for this blob. The spec I worked from is http://www.fintek.com.tw/files/productfiles/F71872F_V026P.pdf As far as I can tell, the sensors part of the 71805 is the same as the 71872. If I had more examples to test I'd add checks for the other chips in this series. Is this what you were looking for? geoff
format for submissions
Is there a page somewhere giving the proper format for source code submissions. Yes, I know about the knf page. I thank Theo very much for his attention to anything which might impact the code base, but it would save him a lot of trouble if he wrote up a 30-line how to announce source for trial before submission and a 30-line what I expect in a finished code package and put those pages easily visible from the OpenBSD home page.
Re: Bug in dd?
Index: position.c
===
RCS file: /cvs/src/bin/dd/position.c,v
retrieving revision 1.7
diff -u -p -r1.7 position.c
--- position.c11 Jun 2003 23:42:12 - 1.7
+++ position.c6 Nov 2006 12:07:54 -
@@ -71,7 +71,7 @@ pos_in(void)
int warned;
/* If not a pipe or tape device, try to seek on it. */
- if (!(in.flags (ISPIPE|ISTAPE))) {
+ if (!(in.flags (ISPIPE|ISTAPE|ISCHR))) {
if (lseek(in.fd, in.offset * in.dbsz, SEEK_CUR) == -1)
err(1, %s, in.name);
return;
I may be reading this totally wrong, but won't this
change prevent using seek on a character disk device?
That would be a -major- loss of functionality as
dd has traditionally been used to read areas past
bad spots on disks. If I'm wrong, apologies to everyone.
geoff steckel
Re: Bug in dd?
From [EMAIL PROTECTED] Tue Nov 7 05:46:56 2006
Received: from vera.drijf.net (ip234-142-210-87.adsl2.versatel.nl
[87.210.142.234])
by ook.oat.com (8.13.8/8.13.8) with ESMTP id kA7AkuNC015972
for [EMAIL PROTECTED]; Tue, 7 Nov 2006 05:46:56 -0500 (EST)
Received: from lou.intra.drijf.net ([EMAIL PROTECTED] [10.0.1.14])
by vera.drijf.net (8.13.8/8.13.8) with ESMTP id kA7ANk2n029198
(version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO);
Tue, 7 Nov 2006 11:23:47 +0100 (CET)
Date: Tue, 7 Nov 2006 11:23:46 +0100 (CET)
To: Geoff Steckel [EMAIL PROTECTED]
cc: [EMAIL PROTECTED], misc@openbsd.org
Subject: Re: Bug in dd?
On Tue, 7 Nov 2006, Otto Moerbeek [EMAIL PROTECTED] wrote:
On Tue, 7 Nov 2006, Geoff Steckel wrote:
Index: position.c
===
RCS file: /cvs/src/bin/dd/position.c,v
retrieving revision 1.7
diff -u -p -r1.7 position.c
--- position.c 11 Jun 2003 23:42:12 - 1.7
+++ position.c 6 Nov 2006 12:07:54 -
@@ -71,7 +71,7 @@ pos_in(void)
int warned;
/* If not a pipe or tape device, try to seek on it. */
- if (!(in.flags (ISPIPE|ISTAPE))) {
+ if (!(in.flags (ISPIPE|ISTAPE|ISCHR))) {
if (lseek(in.fd, in.offset * in.dbsz, SEEK_CUR) == -1)
err(1, %s, in.name);
return;
I may be reading this totally wrong, but won't this
change prevent using seek on a character disk device?
That would be a -major- loss of functionality as
dd has traditionally been used to read areas past
bad spots on disks. If I'm wrong, apologies to everyone.
geoff steckel
Hmmm, I think you are right. Maybe better test for isatty()?
-Otto
Index: position.c
===
RCS file: /cvs/src/bin/dd/position.c,v
retrieving revision 1.8
diff -u -p -r1.8 position.c
--- position.c 7 Nov 2006 07:10:24 - 1.8
+++ position.c 7 Nov 2006 10:23:17 -
@@ -70,8 +70,8 @@ pos_in(void)
off_t cnt;
int warned;
- /* If not a pipe, tape or char device, try to seek on it. */
- if (!(in.flags (ISPIPE|ISTAPE|ISCHR))) {
+ /* If not a pipe, tape or tty device, try to seek on it. */
+ if (!(in.flags (ISPIPE|ISTAPE)) !isatty(in.fd)) {
if (lseek(in.fd, in.offset * in.dbsz, SEEK_CUR) == -1)
err(1, %s, in.name);
return;
Looks better to me! Are there any other character devices which
should be tested for while this section of code is under repair?
geoff
pf.conf grammar botch
The recent request for better comments in pf.conf files as well as #include functionality points out a basic flaw in the input language design: The newline delimited input without /* */ comments. And a basic flaw in the parser/lexer: Comment handling at parse level not lexer level. A better input language which is a true SUPERSET of the current input language is easy to make with the following changes: 1) move comment and #include functions into the lexer from the parser 2) remove newline as an end-of-statement operator 3) add /* */ comments to the lexer This would simplify the parser and lexer slightly. In addition, adding error productions to the grammar to reduce the occurrence of line 19: syntax error is pretty easy too. The input grammar is sufficiently complex that the program should give the user a good hint about what it saw and what it wants instead. I'd be glad to donate these changes if they have any hope of adoption. Note that any existing pf.conf files would work without any changes. geoff steckel
rl_diag strangeness??
In trying to upgrade to 3.8, the rl driver complains
from rl_diag that loopback failed. There is a bit of code
that doesn't make sense to me:
* Wait for it to propagate through the chip */
DELAY(10);
for (i = 0; i RL_TIMEOUT; i++) {
status = CSR_READ_2(sc, RL_ISR);
if ((status (RL_ISR_TIMEOUT_EXPIRED|RL_ISR_RX_OK)) ==
^^
(RL_ISR_TIMEOUT_EXPIRED|RL_ISR_RX_OK))
^
break;
DELAY(10);
}
if (i == RL_TIMEOUT) {
printf(%s: diagnostic failed, failed to receive packet
in loopback mode\n, sc-sc_dev.dv_xname);
error = EIO;
goto done;
}
It appears that the loop will always run RL_TIMEOUT cycles
because the test is for both RL_ISR_TIMEOUT_EXPIRED and RL_ISR_RX_OK.
On my systems, 3.7 runs the rl8169 fine, but fails the attach
with the above message.
I believe the marked code should be removed, leaving only the
test for either of the two status bits being true.
Anyone have an opinion?
thanks
Geoff Steckel [EMAIL PROTECTED]
