Re: Does anyone use nscd?
Doug Barton wrote: > On 10/10/2011 11:55, David Brodbeck wrote: > > Is there any reason to cache negative hits? > > It's very important for DNS since there are a fairly large number > of misbehaving applications that don't stop querying until they > get some kind of answer. Would this need be sufficiently covered if negative cache timeout were set to, say, 1/4 of a second? That should be short enough to cover virtually any instance in which a missing entry is added manually and the new entry then needs to be found. > And speaking of DNS, while I think that improving nscd is a good > goal I wonder how much use it will be in the world to come when > DNSSEC becomes more important ... Is there something about DNSSEC that makes it fundamentally incompatible with a local cache such as nscd, or is it simply a matter of nscd needing a bit of work to support DNSSEC? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: mmap performance and memory use
Notice that vm.pmap.pde.promotions increased by 31. This means that 31 superpage mappings were created by promotion from small page mappings. thank you. i looked at .mappings as it seemed logical for me that is shows total. In contrast, vm.pmap.pde.mappings counts superpage mappings that are created directly and not by promotion from small page mappings. For example, if a large executable, such as gcc, is resident in memory, the text segment will be pre-mapped using superpage mappings, avoiding soft fault and promotion overhead. Similarly, mmap(..., MAP_PREFAULT_READ) on a large, memory resident file may pre-map the file using superpage mappings. your options are not described in mmap manpage nor madvise (MAP_PREFAULT_READ). when can i find the up to date manpage or description? is it possible to force VM subsystem to operate on superpages when possible - i mean swapping in 2MB chunks? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
On Mon, Oct 10, 2011 at 1:09 PM, Doug Barton wrote: > On 10/10/2011 11:55, David Brodbeck wrote: > > Is there any reason to cache negative hits? > > It's very important for DNS since there are a fairly large number of > misbehaving applications that don't stop querying until they get some > kind of answer. > That's a good point. I hadn't thought about it in terms of DNS; we use it mainly for caching LDAP lookups. -- David Brodbeck System Administrator, Linguistics University of Washington ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Clustering server in freebsd
see hastd(8) On 10/10/11, elman wrote: > Dear all > > I have plan to cluster server with freebsd 8.2 for mailserver. But I'm > confusing with the software for clustering. Do you have a reference for me, > or do you have blog and I can see your blog for reference to create > clustering with freebsd. > > Thanks hacker > Best regards. > Mr. L > Powered by Telkomsel BlackBerry® ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Clustering server in freebsd
On Mon, 10 Oct 2011, elman wrote: Dear all I have plan to cluster server with freebsd 8.2 for mailserver. But I'm confusing with the software for clustering. Do you have a reference for me, or do you have blog and I can see your blog for reference to create clustering with freebsd. You might look at http://www.linux-ha.org/wiki/About to see if that helps. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
On 10/10/2011 11:55, David Brodbeck wrote: > Is there any reason to cache negative hits? It's very important for DNS since there are a fairly large number of misbehaving applications that don't stop querying until they get some kind of answer. And speaking of DNS, while I think that improving nscd is a good goal I wonder how much use it will be in the world to come when DNSSEC becomes more important ... -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Clustering server in freebsd
Dear all I have plan to cluster server with freebsd 8.2 for mailserver. But I'm confusing with the software for clustering. Do you have a reference for me, or do you have blog and I can see your blog for reference to create clustering with freebsd. Thanks hacker Best regards. Mr. L Powered by Telkomsel BlackBerry®___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
On Fri, Oct 7, 2011 at 3:30 AM, Michael Bushkov wrote: > While I agree that nscd negative caching bug should be fixed, it won't > help with the problem that users encounter during ports installation. > When, for example, user "x" is added during port install, the > following steps are involved: > 1. Script checks if "x" is present in the users list. Nscd is queried, > it returns negative and caches negative answer. > 2. Script adds user "x". > 3. Script checks that "x" have indeed been added. Nscd is queried, > cachned negative answer is returned. Script fails as a result. > > So unless negative caching time is less than the time between steps 1) > and 3) the issues during ports installation will persist. Is there any reason to cache negative hits? It seems like you shouldn't see large volumes of them except in some fairly specific circumstances, like extracting a tarball as the root user with invalid UIDs. Maybe I'm missing something, but it seems like turning off negative caching would avoid a lot of potential problems for not much cost. -- David Brodbeck System Administrator, Linguistics University of Washington ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: mmap performance and memory use
On 10/07/2011 12:23, Wojciech Puchar wrote:
You are correct about the page table page. However, a superpage
mapping consumes a single PV entry, in place of 512 or 1024 PV
entries. This winds up saving about three physical pages worth of
memory for every superpage mapping.
does it actually work?
Yes, the sysctl output shows that it is working. You can also verify
this with mincore(2).
simple test
before (only idle system with 2GB RAM and most free)
vm.pmap.pde.promotions: 921
vm.pmap.pde.p_failures: 21398
vm.pmap.pde.mappings: 299
vm.pmap.pde.demotions: 596
vm.pmap.shpgperproc: 200
vm.pmap.pv_entry_max: 696561
vm.pmap.pg_ps_enabled: 1
vm.pmap.pat_works: 1
and with that program running (==sleeping)
#include
int a[1<<24];
main() {
int b;
for(b=0;b<(1<<24);b++) a[b]=b;
sleep(1000);
}
vm.pmap.pdpe.demotions: 0
vm.pmap.pde.promotions: 952
vm.pmap.pde.p_failures: 21398
vm.pmap.pde.mappings: 299
vm.pmap.pde.demotions: 596
vm.pmap.shpgperproc: 200
vm.pmap.pv_entry_max: 696561
vm.pmap.pg_ps_enabled: 1
vm.pmap.pat_works: 1
seems like i don't understand what these sysctl things mean (i did
sysctl -d)
or it doesn't really work. with program allocating and using linear
64MB chunk it should be 31 or 32 more mappings in vm.pmap.pde.mappings
there are zero difference.
Notice that vm.pmap.pde.promotions increased by 31. This means that 31
superpage mappings were created by promotion from small page mappings.
In contrast, vm.pmap.pde.mappings counts superpage mappings that are
created directly and not by promotion from small page mappings. For
example, if a large executable, such as gcc, is resident in memory, the
text segment will be pre-mapped using superpage mappings, avoiding soft
fault and promotion overhead. Similarly, mmap(..., MAP_PREFAULT_READ)
on a large, memory resident file may pre-map the file using superpage
mappings.
Alan
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [RFC] stdbuf: Force stdio's streams initial buffering mode (patch)
Hi, On Thu, Aug 04, 2011 at 07:19:28PM +0200, Jeremie Le Hen wrote: > On Sat, Feb 19, 2011 at 07:50:43PM +0100, Jeremie Le Hen wrote: > > I've been annoyed multiple time when running a command such like > > iostat -x 1 | grep -v ad10 | cat -n > > > > The problem stems from two factors: > > - grep's stdio sees that its stdout is not a terminal, so stdout is > > full buffered and not line-buffered; > > - iostat produces output too slowly so the aforementioned buffer takes > > numerous seconds to be filled and flushed to the last command. > > > > This problems is not specific to FreeBSD, it is actually a consequence > > of POSIX specification. I've checked this on Solaris and Linux. > > > > I've attached a small patch for stdio, so if the environment variable > > STDIO_IOLBF is set, the output streams will be line-oriented by default. > > iostat -x 1 | env STDIO_IOLBF=1 grep -v ad10 | cat -n > > I improved the whole picture. Now there is a shared library > libstdbuf.so which can be loaded with LD_PRELOAD and configured through > a environment variables. It is able to initial control buffering for > stdin, stdout and stderr (no buffering, line buffering, block > buffering). There is also an utility named stdbuf(1) which can be used > to run a command with the appropriate environment variables. Those are > named after a similar tool in recent versions of GNU coreutils; of > course, I also borrowed the interface for POLA. > > Here is how to use it (example taken from the manpage): > > vmstat 1 | stdbuf -o L awk '$2 > 1 || $3 > 1' | cat -n > > I think that using a preloaded shared library is better performance-wise > because libc doesn't bother looking up configuration variables in the > environment upon each execve(2), especially for something which is to be > rarely used. > > Manpages for both stdbuf(1) and libstdbuf(3) are provided. There is > surely room for improvement, so feel free to propose corrections to > them. > > The patch can be applied as-is I think, although I've developped it > against an old -CURRENT source tree (about 6 months ago). > > I'm looking for some feedback and review and hopefully this feature will > be committed for FreeBSD 9.0-RELEASE. It's now too late for 9.0-RELEASE :). But I'm still willing to see this feature committed eventually! While talking about this implementation with a friend, he told me that having a separate shared library sounds a little bit hackish and two or three getenv(3) in the libc startup path shouldn't eat too much CPU cycles. Any opinion on this? Anyone willing to commit this? Thanks. Regards, -- Jeremie Le Hen Men are born free and equal. Later on, they're on their own. Jean Yanne ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
