Re: pgbench performance is lagging compared to Linux and DragonflyBSD?
Your entire email is conjecture, the performance of DragonFly 3.2 is improved across the board vs 3.0. Not just batch performance, interactive performance (especially under X11) is also greatly improved. Sam On Tue, Nov 6, 2012 at 2:25 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: some serious system issue. It looks like the DragonflyBSD folks made a goal to do well on pgbench and got to the level of ~88% of linux with 80 clients. It's just bad that anyone judge and (even worse) modify/tune operating system to do well in SINGLE benchmark running basically single program doing few repetitive things. Linux is tuned to win in benchmark and it does, while having disastrous performance in normal unix style usage - multiple different programs doing multiple different things for multiple different users - in the same time. This is a case with at least 99% of users. The less than 1% that have so heavy load that needs separete machine dedicated to single program doing one thing - could use linux (if it REALLY will be better in production workload ) or even better - use some dedicated hardware just for this, if it exist. Does machine that is dedicated to run single program need OS at all? In such benchmark FreeBSD with UFS wins hands down and that's the reason i use it. Still it is interesting WHY FreeBSD is slower in that special case, and if improvements on general behaviour can be found then it's nice to do them. I tried dragonflybsd some time ago and it's performance on normal usage is disastrous. Seems like Matthew Dillion years after splitting from FreeBSD because the algorithms used in FreeBSD were plain wrong - cannot do this better but still waste time and still at all cost want to prove he can. Tuning operating system for single benchmark is an example of that childish behaviour. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: pgbench performance is lagging compared to Linux and DragonflyBSD?
On Tue, Nov 6, 2012 at 10:58 AM, Yuri y...@rawbw.com wrote: On 11/05/2012 12:52, Garrett Cooper wrote: FWIW, I think that the last time scheduler benchmarks from anyone at @FreeBSD.org (was kris@ the last one, or has flo@ run benchmarks since I myself ran the similar test on i7 920 (4 cores 8 threads) @ 2.67 24GB with 9.1-RC3 with all the same params except shmem size was 4GB, not 6GB: http://i.imgur.com/mfnqr.png In DragonflyBSD tests FreeBSD peaked at 96k tps. And my machine, with roughly 3X lesser power, peaked at 44.5k tps. So in my test BSD performed relatively better. And graph shape is more resembling linux/DragonflyBSD ones. It looks like in their test FreeBSD behaved in somewhat impaired way. Any ideas what can I try to tune? Single and multi-socket hardware are not really directly comparable in PostgreSQL tests. Sam Yuri ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: pgbench performance is lagging compared to Linux and DragonflyBSD?
On Tue, Nov 6, 2012 at 2:01 PM, Yuri y...@rawbw.com wrote: On 11/06/2012 11:10, Samuel J. Greear wrote: Single and multi-socket hardware are not really directly comparable in PostgreSQL tests. So if the CPUs are split between sockets, would such system generally perform better or worse with PostgeSQL vs. non-split situation? Yuri Unless the algorithms you are testing are able to operate entirely out of the processors caches (and PostgreSQL does not fall into that category) performance will be generally lower as you add sockets. FreeBSD's ULE scheduler is aware of this and takes it into account, but the performance ULE is able to maintain across sockets (this applies to other OS's schedulers too) is more damage control than anything else. Sam ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Replacing BIND with unbound
On Tue, Aug 21, 2012 at 12:56 PM, Doug Barton do...@freebsd.org wrote: On 8/21/2012 11:08 AM, Bjoern A. Zeeb wrote: On Tue, 21 Aug 2012, Doug Barton wrote: Neither importing ldns nor removing BIND is going to have any effect on the stub resolver library in libc. Yes it does as if we are not carefull, we'll neither have a _proper_ validating caching resolver but 4 different resolver libraries 3 of which needing crypto and only 2 with a well known support plan and only 2 with the same interface in base. snip Just a data point. DragonFly removed BIND from base in favor of ldns/drill in June of 2010, there has not been any quantifiable fallout. We did not bring in unbound and there are no plans to do so. Sam ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: PostgreSQL benchmarks (now with Linux numbers)
On Wed, Feb 22, 2012 at 7:59 PM, Doug Barton do...@freebsd.org wrote: On 02/22/2012 01:42, Ivan Voras wrote: The Dragonfly team has recently liberated their VM from the giant lock and there are some interesting benchmarks comparing it to FreeBSD 9 and a derivative of RedHat Enterprise Linux: http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/msg8.html Other developments are described in their release notes: http://www.dragonflybsd.org/release30/ The 4.5 times improvement by enabling kern.ipc.shm_use_phys is pretty notable, what prevents us from enabling that by default? Doug -- I just saw this, so I thought I would mention -- DragonFly was not doing a great job allocating pv entries rapidly, which is needed during PostgreSQL warm up -- This made shm_use_phys appear very effective. It only makes 1-2% difference now on DragonFly, and only 1-2% difference on FreeBSD. Sam ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: PostgreSQL benchmarks (now with Linux numbers)
On Tue, Mar 27, 2012 at 8:27 AM, Ivan Voras ivo...@freebsd.org wrote: On 02/22/2012 01:42, Ivan Voras wrote: The Dragonfly team has recently liberated their VM from the giant lock and there are some interesting benchmarks comparing it to FreeBSD 9 and a derivative of RedHat Enterprise Linux: I just saw this, so I thought I would mention -- DragonFly was not doing a great job allocating pv entries rapidly, which is needed during PostgreSQL warm up -- This made shm_use_phys appear very effective. It only makes 1-2% difference now on DragonFly, and only 1-2% difference on FreeBSD. Great! Do you have a link to the updated results? I looked back at this and I guess I mis-spoke a little. This is the best I have, http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/pdf0.pdf -- the second page is I think the same as what was posted here before, but the first page is intermediate results. You can see there that we had a significant fall-off without shm_use_phy's after a certain point (yellow line). What I mis-remembered was that FreeBSD had a fall-off after a certain point as well -- it just isn't nearly as bad as ours was. https://github.com/DragonFlyBSD/DragonFlyBSD/commit/5669360730c92e705c92afd02210e7859b0dc722 -- Here is the relevant commit that enabled bursting of pv_entry's to our per-cpu cache, iirc this commit alone made up all of that difference (between the yellow line and the light green line on the first page of the pdf). I think that probably we are a little generous with the bursting now and that real-world this won't have much of an impact, real-world FreeBSD is probably already fine in this regard, caching things like this mostly just makes stuff look good on benchmarks. Another thing of note, the performance of PostgreSQL benchmarks is heavily limited by the concurrent fault rate of the kernel. (again, this is iirc) Pg opens files that are tens to hundreds of megabytes and these get faulted in 4096 bytes at a time, during these benchmarks it will be doing this from potentially several dozen processes on the same file. If these faults can be taken concurrently performance improves rather dramatically. Again, this is probably one of those things that does not have a dramatic effect on most real-life workloads but makes things look really good in benchmarks. Sam ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Idea about 'skeleton jail
On Sunday 13 March 2005 14:24, Anish Mistry wrote: On Sunday 13 March 2005 01:23 pm, Chris Hodgins wrote: Samuel J. Greear wrote: Not a bad 'idea' at all, although I won't comment on semantics. I had something implemented using fs stacking (in a very hackish way, and I believe it's lost now, so don't ask to see it...) to implement per-jail quota's that seemed to work quite well. Sam Feel free to comment on the semantics. As I said before, I am not very knowledgable about filesystems and any insight or alternative implementation you can provide would be interesting I'm sure to everyone. Yeah, if there was jailfs that was setup automatically for the jails that supported quotas out of the box that would kill my major gripe about setting up jails. Chris, your concept looks reasonable to me. I think I would probably do something along those lines but borrow some idea's from my 'jail-build' script. It has the concept of both includes and excludes, but it also handles another directory for what I call overrides. My overrides directories are per-jail and typically include nothing more than config. files, but it works pretty handily. The overrides may best be implemented in a seperate layer... and I don't even know that I would call something like this a jailfs, more like a globfs or something... I can see potential uses beyond jails. The reasons that I never finished implementing my jailfs with quota support were primarily, that stackable filesystems seem to be somewhat of a black-art. Secondarily, I concluded that the time would be better spent implementing filesystem agnostic quota's in the vfs layer. A proper design should enable you to do a lot of fun things, I was thinking something along the lines of just a simple aggregator that a module could hand function pointers to and register interest in events, with options like.. just-notify-me and dont-continue-without-my-approval. Throw in some helpers for synchronizing module state to disk. The kernel side of this shouldn't really be very hard, but all of the userland quota utilities would need to be rewritten as they are tied to UFS at the block level. This all from about 3 years ago, and I haven't implemented any of it. I rock! Sam ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Idea about 'skeleton jail
Not a bad 'idea' at all, although I won't comment on semantics. I had something implemented using fs stacking (in a very hackish way, and I believe it's lost now, so don't ask to see it...) to implement per-jail quota's that seemed to work quite well. Sam This might be a very stupid idea but how about a jailfs. Now I don't know all that much about filesystem design so bear with me. How about something like this: snippay SO the jail filesystem is configured at jail-creation time and uses the hosts files or jail files depending on the configuration. Might have to pass the config file into the jail command. As I said I am not an expert. Mabye one of the experts could let me know what they think? Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: In-Kernel HTTP Server (name preference)
On Monday 18 February 2002 07:54 pm, Peter Wemm wrote: Mike Silbersack wrote: On Mon, 18 Feb 2002, Hiten Pandya wrote: hi all, As to conclude this thread (for me.), I have come to the decision of actually starting a project for making a BSD Licensed in-kernel HTTPd server. The project will be on SourceForge.net. As you all know, that when starting a project, a name is needed for project; I completely out of ideas, and I have literally no creative skills. :) If you want to be really useful, I have a better first step for you. :) Common wisdom seems to be that Apache is slow, other httpds are faster, custom ones are fastest. However, I don't think I've actually seen any comparisons since this one of thttpd vs others: http://www.acme.com/software/thttpd/benchmarks.html Before starting work on a kernel httpd, you might wish to run similar benchmarks (with perhaps only 5 different httpds) to see what the current performance of FreeBSD is; it may turn out that some limitation in the TCP stack is hit even by userland httpds, and your effort would be better spent on fixing that first. The problem is that our threads implementation sucks. The moment that thttpd has to do an actual disk read on freebsd, the whole thing comes to a screeching halt. Threaded http servers do not stand up to real-world loads on freebsd, unless there are very specially constructred scenarios in place.. ie: everything is in ram, no FS calls ever block, etc. Cheers, -Peter I don't suppose it matters much than thttpd isn't threaded. It's a non-blocking server that uses it's own abstraction layer over select/poll/kevent called fdevent. It also maintains an mmap'd cache of frequently accessed files. Sam To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: jail patch
On Monday 26 November 2001 03:45 am, Antony T Curtis wrote: Robert Watson wrote: On Sun, 25 Nov 2001, Gregory Neil Shapiro wrote: snip In the jailng code, I allow jails to be identified using a name (other than the hostname) when they are created, and that can later be used as a handle for signalling. Two of the concepts that are useful in jailng are (1) the ability to identify jails and manage them from the outside more easily, and (2) jailinit, which permits a jail to maintain a runlevel, meaning that you don't have to be 'in' a jail in order to start an orderly shutdown (as you can signal jailinit), not to mention introducing the notion of an orderly shutdown :-). snip I currently make use of a hacked version of init which allows me to have a whole system in a jail - this allows me to telnet in to a jail and do a shutdown. The only downside is that many things expect init to be pid=1 but in the jail, this isn't true - I keep the pid of the init in a temporary file (ugly hack, a better hack would probably involve hacking the kernel sources so that the jailed pid is 1 and that when that process dies, the whole jail gets a kill -9. http://www.jailbsd.net/tarballs/jailinit.rat.gz This is a little something that I whipped up some time back, but haven't put much effort into lately. Sam To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
