Re: mksnap_ffs time
On 01/03/08 17:46, Daan Vreeken [PA4DAN] wrote: On Friday 04 January 2008 00:30:20 you wrote: Hi! 1) Is anyone else seeing this? As far as I know, this is somehow related to the SoftUpdates, but I am not 100% sure. Look at the mailinglist archives, there was a discussion a few months ago about the same topic as far as I remember. Please note that according to http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/snapshots.html a filesystem can only have 20 snapshots! :-) Hmm.. Then at least time spent per snapshot isn't going to get much worse than it is today ;-) Too bad this limitation isn't mentioned in 'man 8 mksnap_ffs'. If this limitation is still there, I think it deserves a place in the manual. The limitation is still there, and probably should be mentioned in the manual - good point. The time is spent making copies of all the cylinder groups, which there are a *lot* of on a 700GB file system. When UFS snapshots were developed, 4-20GB file systems were pretty big. Over time, this problem is going to just get worse. :( The only way to help it, is to use some custom newfs options to reduce the number of cylinder groups, but it won't buy you much. Snapshots also impede on your general file system performance. In FreeBSD 7, you could use ZFS, which would give you the power and control you need (plus FreeBSD 7 has many many performance improvements). Eric ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: mksnap_ffs time
On Friday 04 January 2008 00:30:20 you wrote: > Hi! > > > 1) Is anyone else seeing this? > > As far as I know, this is somehow related to the SoftUpdates, but I am not > 100% sure. Look at the mailinglist archives, there was a discussion a few > months ago about the same topic as far as I remember. > > Please note that according to > http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/snapshots.html > a filesystem can only have 20 snapshots! :-) Hmm.. Then at least time spent per snapshot isn't going to get much worse than it is today ;-) Too bad this limitation isn't mentioned in 'man 8 mksnap_ffs'. If this limitation is still there, I think it deserves a place in the manual. Thanks! -- Daan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
mksnap_ffs time
Hi Hackers, I'm running a 6.2-RELEASE server with a /home partition that's used to backup a customer's data. The backups are being made during the night. Since a couple of weeks I'm running mksnap_ffs daily to create filesystem snapshots of the data. I've been logging the time it takes mksnap_ffs to create it's snapshot. Here are the 'time' numbers for the last 11 days : 1696.25 real 0.00 user16.55 sys 1884.09 real 0.00 user18.63 sys 2038.50 real 0.00 user20.72 sys 2354.69 real 0.00 user25.07 sys 2202.43 real 0.00 user23.08 sys 2621.88 real 0.00 user27.46 sys 2791.46 real 0.00 user29.57 sys 2845.48 real 0.00 user31.69 sys 2894.24 real 0.00 user32.60 sys 3085.37 real 0.00 user35.62 sys 3253.11 real 0.00 user37.25 sys Right now the filesystem contains 'just' 16 snapshots. The filesystem itself is a 700GB partition : # df Filesystem1K-blocks Used Avail Capacity Mounted on ... /dev/twed0s3d 700905640 271350848 37348234242%/home The filesystem has been created with default newfs flags. Today the filesystem contains 186615 files. Average file size is about 140 to 150KB. The data consists of lots of very small files, lots of middle-sized files, and a bunch of huge files, so 'average' file size might not be a very good measure. Every day between 300 to 400MB of data is changed/added. The time it takes to take another snapshot seems to increase with about 140 seconds a day. If the snapshot time keeps increasing in this fashion, the server will spend half of it's time in a day creating snapshots by the end of the year ;-) 1) Is anyone else seeing this? 2) Is there anything that can be done to improve snapshot creation time? 3) Is this something that has already been improved in version X of FreeBSD? And just for my own curiosity: Where does mksnap_ffs spend most of it's time? Is this something that simply can't be changed by the nature of UFS, or is it a known deficiency in the code that could by improved somehow? (btw: I can provide more information on the hardware or the filesystem (file size histograms anyone?) if anyone is interested.) Thanks for any insights, -- Daan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Porting from Linux to FreeBSD (procfs question)
On Jan 3, 2008 9:53 PM, Ivan Voras <[EMAIL PROTECTED]> wrote: > Fernando Apesteguía wrote: > > > Maybe a mix of both could be good: use the linxprocfs when it is > > almost straightforward (in fact I could run the app, just changing few > > lines) and sysctl when the linprocfs doesn't provide the information > > that I need. > > Also, linprocfs is never mounted by default - this might or might not be > an obstacle, depending on how user-friendly the application is (asking > desktop users to alter their fstabs may be asking too much :) ). > I hadn't thought about it... but it's quite to the point. Maybe it makes no sense to ask for a particular filesystem from a graphical Gnome application... > > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Porting from Linux to FreeBSD (procfs question)
Fernando Apesteguía wrote: > Maybe a mix of both could be good: use the linxprocfs when it is > almost straightforward (in fact I could run the app, just changing few > lines) and sysctl when the linprocfs doesn't provide the information > that I need. Also, linprocfs is never mounted by default - this might or might not be an obstacle, depending on how user-friendly the application is (asking desktop users to alter their fstabs may be asking too much :) ). signature.asc Description: OpenPGP digital signature
Re[2]: select
> Metin KAYA <[EMAIL PROTECTED]> writes: >> Yes Rick, I'm asking this "indefinitely" issue. Is there anything that >> handle this NULL situation a signal, or etc.? How does Linux or >> FreeBSD behave? > Please don't top-post. > Like most other system calls that block "indefinitely", select(2) will > be interrupted by signals. This is *also* documented in the man page > you didn't read: > [EINTR]A signal was delivered before the time limit expired > and before any of the selected events occurred. > See sigaction(2) for details on how to modify the way system calls > behave when a signal is delivered. > DES Thanks. I've read select_tut(2) man page in FC8 but there is no such explanation about EINTR, so don't mind my this mistake. -- Metin KAYA EnderUNIX Software Developer Endersys Software Engineer http://www.EnderUNIX.org/metinhttp://www.Endersys.com/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: select
Garrett Cooper wrote:
Metin KAYA wrote:
Yes Rick, I'm asking this "indefinitely" issue. Is there anything
that handle this NULL situation a signal, or etc.? How does Linux or
FreeBSD behave?
On Thu, Jan 03, 2008 at 08:52:48PM +0200, Metin KAYA wrote:
How select(2) will behave if I give the "utimeout" parameter as
NULL?
According to the man page:
If timeout is not a null pointer, it specifies the maximum
interval to
wait for the selection to complete. System activity can
lengthen the
interval by an indeterminate amount.
If timeout is a null pointer, the select blocks indefinitely.
To effect a poll, the timeout argument should not be a null
pointer, but
it should point to a zero-valued timeval structure.
-- Rick C. Petty
-- Metin KAYA EnderUNIX
Software Developer Endersys Software Engineer
http://www.EnderUNIX.org/metinhttp://www.Endersys.com/
Nevermind -- yes, block indefinitely, which implies that the program
won't proceed until it receives an umasked signal and exits or a file
descriptor becomes available in the 'infinite' time frame.
That would essentially be the same as listen or send though with
blocking sockets, correct?
-Garrett
Give this beauty a shot [manually modified from the FC7 select(2) manpage]:
#include
#include
int main() {
fd_set rfds;
FD_ZERO(&rfds);
FD_SET(0, &rfds);
printf("Wait for it...\n");
select(1, &rfds, NULL, NULL, NULL);
printf("Done!\n");
return 0;
}
It will indefinitely block until you provide an EOF on the terminal (or
have some other EOF'ed data written to STDIN for the program, i.e. via a
pipe).
Sending signals (apart from SIGIO -- 23) instantly kills the program as
nothing's masked. SIGIO not killing the program is to be expected as per
signal(7) and fcntl(2)'s manpage descriptions.
Tested on an FC7 machine, so your mileage on a FreeBSD machine may differ.
Cheers,
-Garrett
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Porting from Linux to FreeBSD (procfs question)
On Jan 3, 2008 8:50 PM, Dag-Erling Smørgrav <[EMAIL PROTECTED]> wrote: > "Fernando Apesteguía" <[EMAIL PROTECTED]> writes: > > I have an application written in C + Gtk + Glib that gathers > > information from procfs under Linux. Now, I'm trying to port this > > application to FreeBSD (I'm working with 7.0 RC1). > > man linprocfs Yes, I checked it. But although the cpuinfo file is quite close to the "original" one in Linux, there are other important files missed. That's because I'm trying to find the best approach. Maybe a mix of both could be good: use the linxprocfs when it is almost straightforward (in fact I could run the app, just changing few lines) and sysctl when the linprocfs doesn't provide the information that I need. Cheers > > DES > -- > Dag-Erling Smørgrav - [EMAIL PROTECTED] > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: select
In the last episode (Jan 03), Metin KAYA said: > Hi all, > > How select(2) will behave if I give the "utimeout" parameter as > NULL? >From the man page: If timeout is a null pointer, the select blocks indefinitely. http://www.freebsd.org/cgi/man.cgi?query=select -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: select
Metin KAYA <[EMAIL PROTECTED]> writes: > Yes Rick, I'm asking this "indefinitely" issue. Is there anything that > handle this NULL situation a signal, or etc.? How does Linux or > FreeBSD behave? Please don't top-post. Like most other system calls that block "indefinitely", select(2) will be interrupted by signals. This is *also* documented in the man page you didn't read: [EINTR]A signal was delivered before the time limit expired and before any of the selected events occurred. See sigaction(2) for details on how to modify the way system calls behave when a signal is delivered. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Porting from Linux to FreeBSD (procfs question)
"Fernando Apesteguía" <[EMAIL PROTECTED]> writes: > I have an application written in C + Gtk + Glib that gathers > information from procfs under Linux. Now, I'm trying to port this > application to FreeBSD (I'm working with 7.0 RC1). man linprocfs DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: select
On Thu, Jan 03, 2008 at 08:52:48PM +0200, Metin KAYA wrote: > > How select(2) will behave if I give the "utimeout" parameter as > NULL? According to the man page: If timeout is not a null pointer, it specifies the maximum interval to wait for the selection to complete. System activity can lengthen the interval by an indeterminate amount. If timeout is a null pointer, the select blocks indefinitely. To effect a poll, the timeout argument should not be a null pointer, but it should point to a zero-valued timeval structure. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: select
Garrett Cooper wrote: Metin KAYA wrote: Yes Rick, I'm asking this "indefinitely" issue. Is there anything that handle this NULL situation a signal, or etc.? How does Linux or FreeBSD behave? On Thu, Jan 03, 2008 at 08:52:48PM +0200, Metin KAYA wrote: How select(2) will behave if I give the "utimeout" parameter as NULL? According to the man page: If timeout is not a null pointer, it specifies the maximum interval to wait for the selection to complete. System activity can lengthen the interval by an indeterminate amount. If timeout is a null pointer, the select blocks indefinitely. To effect a poll, the timeout argument should not be a null pointer, but it should point to a zero-valued timeval structure. -- Rick C. Petty -- Metin KAYA EnderUNIX Software Developer Endersys Software Engineer http://www.EnderUNIX.org/metinhttp://www.Endersys.com/ Nevermind -- yes, block indefinitely, which implies that the program won't proceed until it receives an umasked signal and exits or a file descriptor becomes available in the 'infinite' time frame. That would essentially be the same as listen or send though with blocking sockets, correct? -Garrett Well, the waiting part at least, not the sending and receiving part of send and listen, respectively.. -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: select
Metin KAYA wrote: Yes Rick, I'm asking this "indefinitely" issue. Is there anything that handle this NULL situation a signal, or etc.? How does Linux or FreeBSD behave? On Thu, Jan 03, 2008 at 08:52:48PM +0200, Metin KAYA wrote: How select(2) will behave if I give the "utimeout" parameter as NULL? According to the man page: If timeout is not a null pointer, it specifies the maximum interval to wait for the selection to complete. System activity can lengthen the interval by an indeterminate amount. If timeout is a null pointer, the select blocks indefinitely. To effect a poll, the timeout argument should not be a null pointer, but it should point to a zero-valued timeval structure. -- Rick C. Petty -- Metin KAYA EnderUNIX Software Developer Endersys Software Engineer http://www.EnderUNIX.org/metinhttp://www.Endersys.com/ Nevermind -- yes, block indefinitely, which implies that the program won't proceed until it receives an umasked signal and exits or a file descriptor becomes available in the 'infinite' time frame. That would essentially be the same as listen or send though with blocking sockets, correct? -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re[2]: select
Thank you a lot for your good answer. Please read my other mail. I'm asking a question in man page. > Metin KAYA wrote: >> Hi all, >> >> How select(2) will behave if I give the "utimeout" parameter as >> NULL? >> >> -- >> Metin KAYA >> EnderUNIX Software Developer Endersys Software Engineer >> http://www.EnderUNIX.org/metinhttp://www.Endersys.com/ >> >> ___ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "[EMAIL PROTECTED]" >> > In the future: > Why not try compiling a simple test program to try it out or google the > subject...? Your doing that would be more effective use of your time and > others'.. > My guess is that it's implementation independent and may involve > segfaults or other 'unknown' behavior, as it's not a defined input.. > -Garrett -- Metin KAYA EnderUNIX Software Developer Endersys Software Engineer http://www.EnderUNIX.org/metinhttp://www.Endersys.com/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: select
Metin KAYA wrote: Hi all, How select(2) will behave if I give the "utimeout" parameter as NULL? -- Metin KAYA EnderUNIX Software Developer Endersys Software Engineer http://www.EnderUNIX.org/metinhttp://www.Endersys.com/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]" In the future: Why not try compiling a simple test program to try it out or google the subject...? Your doing that would be more effective use of your time and others'.. My guess is that it's implementation independent and may involve segfaults or other 'unknown' behavior, as it's not a defined input.. -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re[2]: select
Yes Rick, I'm asking this "indefinitely" issue. Is there anything that handle this NULL situation a signal, or etc.? How does Linux or FreeBSD behave? > On Thu, Jan 03, 2008 at 08:52:48PM +0200, Metin KAYA wrote: >> >> How select(2) will behave if I give the "utimeout" parameter as >> NULL? > According to the man page: > If timeout is not a null pointer, it specifies the maximum interval to > wait for the selection to complete. System activity can lengthen the > interval by an indeterminate amount. > If timeout is a null pointer, the select blocks indefinitely. > To effect a poll, the timeout argument should not be a null pointer, but > it should point to a zero-valued timeval structure. > -- Rick C. Petty -- Metin KAYA EnderUNIX Software Developer Endersys Software Engineer http://www.EnderUNIX.org/metinhttp://www.Endersys.com/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
select
Hi all, How select(2) will behave if I give the "utimeout" parameter as NULL? -- Metin KAYA EnderUNIX Software Developer Endersys Software Engineer http://www.EnderUNIX.org/metinhttp://www.Endersys.com/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Semaphores
Neeku Shamekhi wrote: I need some article about semaphores in freeBSD and mainly about kern_sema.c file. man 9 sema ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Porting from Linux to FreeBSD (procfs question)
On Thu, Jan 03, 2008 at 06:26:21PM +0100, Fernando Apestegu?a wrote: > Hi all, > > I post my question in this list cause Ivan Voras ([EMAIL PROTECTED]) > suggested that here I could find more help. > > I have an application written in C + Gtk + Glib that gathers > information from procfs under Linux. Now, I'm trying to port this > application to FreeBSD (I'm working with 7.0 RC1). > > I've been pointed to get rid of the procfs stuff in the freebsd code > and use sysctl instead. However, I can't find some of the information > that is present under /proc in Linux. > > I can only get for the CPU for instance some small bits of info in > "hw.model" but I can't get the cache size, supported extensions, etc. > I have the same problem with the memory (how much memory is cached, in > buffers, marked as dirty, etc...). > > How can I get all that information? Could you point me to some source > I can look at? > > Is this information (that I suppose it is known by the kernel) somehow > exported to userland? > > Here is a small list of the information the Linux application is > available to show: > > - CPU info: (vendor, model, clock speed, stepping, cache size, > supported extensions) See misc/cpuid for a port that extracts this from the CPU. To some extent it would be better if the kernel did this because it can better know which CPU it's on, but this does most of it. > - Memory: total, free, shared, cached, used in buffers, marked as dirty, etc.. You can't get a perfect mapping do these because we have different virtual memory systems. The continued delusion that any of this is comparable between OSes is a pet peeve of mine. :) The sysutils/ganglia-monitor-core port does some of this (search for freebsd.c). > - Devices: list of block and char devices ls /dev :-) > - Filesystems loaded in the kernel lsvfs(1) reports this information. > - Network information (bytes sent and received, packets, errors, for each NIC) See ganglia. > - Loaded modules (sizes of the modules, if they are used by other modules...) see kldstat(8) > - Some kernel parameters (max files per process, semaphores, etc...) > I've seen this is fully covered with sysctl. -- Brooks pgpciLAjvh8bb.pgp Description: PGP signature
Porting from Linux to FreeBSD (procfs question)
Hi all, I post my question in this list cause Ivan Voras ([EMAIL PROTECTED]) suggested that here I could find more help. I have an application written in C + Gtk + Glib that gathers information from procfs under Linux. Now, I'm trying to port this application to FreeBSD (I'm working with 7.0 RC1). I've been pointed to get rid of the procfs stuff in the freebsd code and use sysctl instead. However, I can't find some of the information that is present under /proc in Linux. I can only get for the CPU for instance some small bits of info in "hw.model" but I can't get the cache size, supported extensions, etc. I have the same problem with the memory (how much memory is cached, in buffers, marked as dirty, etc...). How can I get all that information? Could you point me to some source I can look at? Is this information (that I suppose it is known by the kernel) somehow exported to userland? Here is a small list of the information the Linux application is available to show: - CPU info: (vendor, model, clock speed, stepping, cache size, supported extensions) - Memory: total, free, shared, cached, used in buffers, marked as dirty, etc.. - Devices: list of block and char devices - Filesystems loaded in the kernel - Network information (bytes sent and received, packets, errors, for each NIC) - Loaded modules (sizes of the modules, if they are used by other modules...) - Some kernel parameters (max files per process, semaphores, etc...) I've seen this is fully covered with sysctl. Thanks in advance and Happy New Year. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Porting from linux to FreeBSD (procfs question)
On Thu, Jan 03, 2008 at 05:16:42PM +0100, Ivan Voras wrote: > On 03/01/2008, Fernando Apesteguía <[EMAIL PROTECTED]> wrote: > > > Yes, that's my problem. In Linux I can get from /proc/cpuinfo for > > example: name, model, stepping, cache size, clock speed, supported > > extensions, etc... > > But using sysctl in FreeBSD (sysctl -a) I can only see name and vendor > > for the cpu and a few more things. Am I limited to the variables > > showed in sysctl -a? > > Probably. I don't know if there's anything that can give you the > details present in cpuinfo (except using CPUID data directly). Best bet is parsing or using the hw.model sysctl, or if you need lower-level information, there is a port that apparently gets cache size and other data. There are very few things I liked about Linux /proc when I used it, but getting h/w information happened to be one of them... -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Porting from linux to FreeBSD (procfs question)
On 03/01/2008, Fernando Apesteguía <[EMAIL PROTECTED]> wrote: > Yes, that's my problem. In Linux I can get from /proc/cpuinfo for > example: name, model, stepping, cache size, clock speed, supported > extensions, etc... > But using sysctl in FreeBSD (sysctl -a) I can only see name and vendor > for the cpu and a few more things. Am I limited to the variables > showed in sysctl -a? Probably. I don't know if there's anything that can give you the details present in cpuinfo (except using CPUID data directly). > Thanks in advance. > > BTW if this is isn't the proper list to continue with this thread, let me > know. Maybe hackers@ would help you more. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Disk sync at shutdown and fusefs filesystems
On 2007-12-17, Alejandro Pulver <[EMAIL PROTECTED]> wrote:
> When the implementation is ready, and if these problems are sorted out,
> do you think it could be enabled by default (at least for root)? Because
> that's the behavior most filesystems would prefer I think.
I made up a testable implementation, see it here:
http://mercurial.creo.hu/repos/fuse4bsd-hg-experimental/?rev/abc018d9f535
It seems to work fine. In order to have a fuse daemon which is
"malicious", ie. tries to stall shutdown, I hacked fusexmp_fh.c as
follows:
--- /dispatch/root/fuse-2.7.0/example/fusexmp_fh.c 2007-05-20
13:05:43.0 +0200
+++ fusexmp_fh.c2008-01-03 02:47:00.0 +0100
@@ -416,6 +416,17 @@ static int xmp_lock(const char *path, st
sizeof(fi->lock_owner));
}
+static void xmp_destroy(void *foo)
+{
+ unsigned i = 0;
+
+ for(;;) {
+ fprintf(stderr, "%d ", i++);
+ sleep(1);
+ }
+}
+
+
static struct fuse_operations xmp_oper = {
.getattr = xmp_getattr,
.fgetattr = xmp_fgetattr,
@@ -451,6 +462,7 @@ static struct fuse_operations xmp_oper =
.removexattr= xmp_removexattr,
#endif
.lock = xmp_lock,
+.destroy = xmp_destroy,
};
int main(int argc, char *argv[])
shutdown(8) was able to complete. The above hack could only cause a ten second
delay -- see the output of the daemon:
unique: 0, opcode: INIT (26), nodeid: 0, insize: 56
INIT: 7.8
flags=0x
max_readahead=0x
INIT: 7.8
flags=0x0002
max_readahead=0x
max_write=0x0002
unique: 0, error: 0 (Unknown error: 0), outsize: 40
0 1 2 3 4 5 6 7 8 9
So FreeBSD's shutdown handles this fine, the system can't really be
DoS'd this way.
Therefore we don't need a too strict access policy. I think that from
our POV, it would be sufficient to add a "-osync_unmount" mount option
and a sysctl via which it's availability for unpriviliged users can be
set. But life is not that simple: if we added such a mount opt that
would remain FreeBSD specific (on Linux it won't happen, for reasons I
don't want to digress on here), and therefore filesystem authors -- who
usually use fuse options internally -- won't use it; although they are
the authorative persons whether their filesystems needs to have a
sychronized unmount or not. I'll try to find the fine middle ground
with Miklos with respect to this.
So while the interface to this feature is under construction, you can
already play with it and I'd like to know about your experiences.
This can be done as follows:
- Get the experimental version of fuse4bsd from the above mentioned
URL. (More exactly, the above URL shows the cset which brings in
the current implementation; if you want to get the latest of
this branch, use
hg {clone,pull} -r sync_unmount0
http://mercurial.creo.hu/repos/fuse4bsd-hg-experimental
or
fetch
http://mercurial.creo.hu/repos/fuse4bsd-hg-experimental/?archive/sync_unmount0.tar.gz
As of the time of writing this, abc018d9f535 and sync_unmount0 refer to
the same revision.)
- Compile it with the CFLAG -DFUSE_HAS_DESTROY=1 (the proto version
hasn't been bumped when DESTROY was added, so I can't figure it
out if DESTROY is available; you have to pass this setting
manually). (Apart from loading the kld, don't forget to use the
mount_fusefs(8) binary compiled from this code!)
- Recompile fusefs-libs using the following revision of
lib/mount_bsd.c:
http://fuse.cvs.sourceforge.net/*checkout*/fuse/fuse/lib/mount_bsd.c?revision=1.14
(this includes the patch
http://fuse.cvs.sourceforge.net/fuse/fuse/lib/mount_bsd.c?r1=1.13&r2=1.14
which fixes a bug referred to in this thread as "issue 2").
- Go wild with your experiments. ATM the easiest way to enable sync unmount
is adding MOUNT_FUSEFS_SYNC_UNMOUNT=1 to the environment. (Making it
settable via the environment lets us leave the lib/fs code intact.)
ATM sync unmount is available without restrictions.
Have fun,
Csaba
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nfs v2/v3 and diskless boot problem
Danny Braniss wrote: Danny Braniss wrote: Danny Braniss wrote: Danny Braniss wrote: there is an undocumented option: boot-nfsroot-options that the diskeless boot can use. I tried boot-nfsroot-options = "nfsv3" since the pxeboot does the initial mount via nfsv2, and this has at least one problem: removing a file from the readonly / will hang the system. so, the remount to v3 works in the case that the root is served by a Freebsd nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle is 32 bytes, and when switching to V3 it becomes 28bytes - sizeof(fhandle_t). This is not liked by the NetApp, which correctly gives error 1001: BADHANDLE :-) While I'm trying to come up with a solution, I am wondering if someone can shed some light: - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to 32 bytes will start WW3? NFSv3 file handles (by spec) can be up to 64bytes. true, but in freebsd, look at sys/nfs/nfsproto.h #define NFSX_V2FH 32 #define NFSX_V3FH (sizeof (fhandle_t)) #define NFSX_V4FH 128 so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) I'm not 100% sure what is happening, but it sounds like the file handle for the mount point or maybe one of the directories is not getting reset on remount. When do you get the BADHANDLE error? Can you capture a tshark/wireshark/tcpdump of the remount and error? I did, and if you look in sys/nfsclient/nfs_vfsops.c, nfs_convert_diskless is responsible for chopping off the 4 extra bytes. BTW, I tried to change the bcopy count to NFSX_V2FH/32, and it panics the kernel :-( danny oh - looks like this says it all: http://fxr.googlebit.com/source/sys/nfsclient/nfsdiskless.h?v=8-CURRENT#L51 that's where the boot-nfsroot-options comes from:-) if you notice, the filehandle for v3 is 64 bytes, but only 28 are used. but as I mentioned initially, this ONLY works when the server is FreeBSD, and breaks for other servers, ie NetAPP. AND the initial question stands: what's in a filehandle, or can it be > 28bytes. Yea, FreeBSD is making the assumption that all NFS servers will use the same size FH for NFSv3. That is just wrong. carful, I think this is the case only if fsb is the server, it will 'probably' accept filehandles of other sizes from other servers. I'm talking about the diskless root mounting code only at this point.. The FH is a server created opaque handle that it can create however it wishes. Most servers use information like inode, generation, fsid, etc to create it, but it's something that you can't necessarily decode. yes, but the FH has information that the server can/must use to figure out which local filesystem it refers to - remember that v2/v3 are stateless. Right, see my list right above your comment: inode, generation, fsid. Those three can uniquely identify a file on a filesystem on a server. There can be anything the server wants to stuff in the FH, or the FH can be a random number assigned to that file, etc. I've created a patch that might fix this, but I'm still testing and QEMU (which I use for my testing) keeps making my system either panic or lock up, so hopefully I should have something for you to try tonight. Also - can you tell me the exact 'mount' command you tried to do the remount/update? it's only in the diskless boot, where setting boot-nfsroot-options = "nfsv3" in /boot/loader.conf will do the remount. Ok - I'll do a little more testing on my patch tonight and let you know. Eric ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nfs v2/v3 and diskless boot problem
> Danny Braniss wrote: > >> Danny Braniss wrote: > Danny Braniss wrote: > > there is an undocumented option: > > boot-nfsroot-options > > that the diskeless boot can use. I tried > > boot-nfsroot-options = "nfsv3" > > since the pxeboot does the initial mount via nfsv2, and this has at > > least > > one problem: removing a file from the readonly / will hang the system. > > > > so, the remount to v3 works in the case that the root is served by a > > Freebsd > > nfs server, but fails if it's NetAPP. The reason is that the v2 > > filehandle > > is 32 bytes, and when switching to V3 it becomes 28bytes - > > sizeof(fhandle_t). > > This is not liked by the NetApp, which correctly gives error 1001: > > BADHANDLE > > :-) > > > > While I'm trying to come up with a solution, I am wondering if someone > > can shed some light: > > - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to > >32 bytes will start WW3? > NFSv3 file handles (by spec) can be up to 64bytes. > >>> true, but in freebsd, look at sys/nfs/nfsproto.h > >>> #define NFSX_V2FH 32 > >>> #define NFSX_V3FH (sizeof (fhandle_t)) > >>> #define NFSX_V4FH 128 > >>> > >>> so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) > >>> > >>> > I'm not 100% sure what is happening, but it sounds like the file handle > for the mount point or maybe one of the directories is not getting reset > on remount. > > When do you get the BADHANDLE error? Can you capture a > tshark/wireshark/tcpdump of the remount and error? > >>> I did, and if you look in sys/nfsclient/nfs_vfsops.c, > >>> nfs_convert_diskless is responsible > >>> for chopping off the 4 extra bytes. BTW, I tried to change the bcopy > >>> count to NFSX_V2FH/32, and > >>> it panics the kernel :-( > >>> > >>> danny > >> > >> oh - looks like this says it all: > >> http://fxr.googlebit.com/source/sys/nfsclient/nfsdiskless.h?v=8-CURRENT#L51 > >> > > that's where the boot-nfsroot-options comes from:-) > > if you notice, the filehandle for v3 is 64 bytes, but > > only 28 are used. > > > > but as I mentioned initially, this ONLY works when the server is FreeBSD, > > and > > breaks for other servers, ie NetAPP. AND the initial question stands: > > what's in a filehandle, or can it be > 28bytes. > > > Yea, FreeBSD is making the assumption that all NFS servers will use the > same size FH for NFSv3. That is just wrong. > carful, I think this is the case only if fsb is the server, it will 'probably' accept filehandles of other sizes from other servers. > The FH is a server created opaque handle that it can create however it > wishes. Most servers use information like inode, generation, fsid, etc > to create it, but it's something that you can't necessarily decode. > yes, but the FH has information that the server can/must use to figure out which local filesystem it refers to - remember that v2/v3 are stateless. > I've created a patch that might fix this, but I'm still testing and QEMU > (which I use for my testing) keeps making my system either panic or lock > up, so hopefully I should have something for you to try tonight. > > Also - can you tell me the exact 'mount' command you tried to do the > remount/update? > it's only in the diskless boot, where setting boot-nfsroot-options = "nfsv3" in /boot/loader.conf will do the remount. cheers, danny ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: nfs v2/v3 and diskless boot problem
Danny Braniss wrote: Danny Braniss wrote: Danny Braniss wrote: there is an undocumented option: boot-nfsroot-options that the diskeless boot can use. I tried boot-nfsroot-options = "nfsv3" since the pxeboot does the initial mount via nfsv2, and this has at least one problem: removing a file from the readonly / will hang the system. so, the remount to v3 works in the case that the root is served by a Freebsd nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle is 32 bytes, and when switching to V3 it becomes 28bytes - sizeof(fhandle_t). This is not liked by the NetApp, which correctly gives error 1001: BADHANDLE :-) While I'm trying to come up with a solution, I am wondering if someone can shed some light: - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to 32 bytes will start WW3? NFSv3 file handles (by spec) can be up to 64bytes. true, but in freebsd, look at sys/nfs/nfsproto.h #define NFSX_V2FH 32 #define NFSX_V3FH (sizeof (fhandle_t)) #define NFSX_V4FH 128 so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) I'm not 100% sure what is happening, but it sounds like the file handle for the mount point or maybe one of the directories is not getting reset on remount. When do you get the BADHANDLE error? Can you capture a tshark/wireshark/tcpdump of the remount and error? I did, and if you look in sys/nfsclient/nfs_vfsops.c, nfs_convert_diskless is responsible for chopping off the 4 extra bytes. BTW, I tried to change the bcopy count to NFSX_V2FH/32, and it panics the kernel :-( danny oh - looks like this says it all: http://fxr.googlebit.com/source/sys/nfsclient/nfsdiskless.h?v=8-CURRENT#L51 that's where the boot-nfsroot-options comes from:-) if you notice, the filehandle for v3 is 64 bytes, but only 28 are used. but as I mentioned initially, this ONLY works when the server is FreeBSD, and breaks for other servers, ie NetAPP. AND the initial question stands: what's in a filehandle, or can it be > 28bytes. Yea, FreeBSD is making the assumption that all NFS servers will use the same size FH for NFSv3. That is just wrong. The FH is a server created opaque handle that it can create however it wishes. Most servers use information like inode, generation, fsid, etc to create it, but it's something that you can't necessarily decode. I've created a patch that might fix this, but I'm still testing and QEMU (which I use for my testing) keeps making my system either panic or lock up, so hopefully I should have something for you to try tonight. Also - can you tell me the exact 'mount' command you tried to do the remount/update? Eric ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Semaphores
Neeku Shamekhi wrote: > I need some article about semaphores in freeBSD and mainly about kern_sema.c > file. About the best documentation you will find is this: http://fxr.watson.org/fxr/source/kern/kern_sema.c For general information on the FreeBSD kernel see the book "Design and implementation of the FreeBSD operating system", http://www.amazon.com/Design-Implementation-FreeBSD-Operating-System/dp/0201702452 and various articles at http://www.freebsd.org/docs/books.html . You won't find anything better. signature.asc Description: OpenPGP digital signature
Semaphores
I need some article about semaphores in freeBSD and mainly about kern_sema.c file. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
