Re: File tree replication in FreeBSD
"Kentucky Mandeloid Mo." <[EMAIL PROTECTED]> wrote: > On Sunday 27 June 2004 15:20, Luke Kearney wrote: > > On Sun, 27 Jun 2004 13:48:07 +0400 > > > > "Kentucky Mandeloid Mo." <[EMAIL PROTECTED]> spake thus: > > > I'm looking for file replication solution for FreeBSD. > > > > > > I have a task to replicate about 11G of files with 500k total files > > > count. I say replication not mirroring i.e. I need to copy modified files > > > to slaves almost imidiately after the change. > > > Tried to find some ready solution but fails. > > > There is FAM from SGI but it fails to work properly on FreeBSD (it makes > > > silly stat(2) calls). > > > kevent(2) also don't have anough API to watch >= 500k files. > > > So I see the only way is to have kernel module that watch kernel calls > > > for file operations. > > > > > > So is there some other points to build a file replication? > > > > Can I suggest you investigate rsync ? It will allow you to replicate > > file trees pretty much on demand across a LAN or indeed the internet > > should you choose to. Works over SSH so it's nice and secure and if you > > need virtually instant replication you could conceivably run this from a > > cron job every N minutes. > > > > /usr/ports/net/rsync > > Yes. I use rsync already and its a great tool. > But actually its too slow with numbers of fles > 100k. > Its takes more then 5mins to found just few files to update in list of 500k > files. > > So I decide rsync to be a transport manager for my own replication tool. > Right now I'm writing a kernel module that catches all file syscalls that can > modify a file. Its almost done exept rmdir & unlink calls - I don't know how > to get a cwd from a thread. Function kern___getcwd sometimes fails with "not > a directory" error. > BTW, whats a good place where can I ask about kernel programming? [EMAIL PROTECTED] is the best place I know of. -- Bill Moran Potential Technologies http://www.potentialtech.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: File tree replication in FreeBSD
On Sunday 27 June 2004 15:20, Luke Kearney wrote: > On Sun, 27 Jun 2004 13:48:07 +0400 > > "Kentucky Mandeloid Mo." <[EMAIL PROTECTED]> spake thus: > > I'm looking for file replication solution for FreeBSD. > > > > I have a task to replicate about 11G of files with 500k total files > > count. I say replication not mirroring i.e. I need to copy modified files > > to slaves almost imidiately after the change. > > Tried to find some ready solution but fails. > > There is FAM from SGI but it fails to work properly on FreeBSD (it makes > > silly stat(2) calls). > > kevent(2) also don't have anough API to watch >= 500k files. > > So I see the only way is to have kernel module that watch kernel calls > > for file operations. > > > > So is there some other points to build a file replication? > > Can I suggest you investigate rsync ? It will allow you to replicate > file trees pretty much on demand across a LAN or indeed the internet > should you choose to. Works over SSH so it's nice and secure and if you > need virtually instant replication you could conceivably run this from a > cron job every N minutes. > > /usr/ports/net/rsync Yes. I use rsync already and its a great tool. But actually its too slow with numbers of fles > 100k. Its takes more then 5mins to found just few files to update in list of 500k files. So I decide rsync to be a transport manager for my own replication tool. Right now I'm writing a kernel module that catches all file syscalls that can modify a file. Its almost done exept rmdir & unlink calls - I don't know how to get a cwd from a thread. Function kern___getcwd sometimes fails with "not a directory" error. BTW, whats a good place where can I ask about kernel programming? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: File tree replication in FreeBSD
On Sun, 27 Jun 2004 13:48:07 +0400 "Kentucky Mandeloid Mo." <[EMAIL PROTECTED]> spake thus: > I'm looking for file replication solution for FreeBSD. > > I have a task to replicate about 11G of files with 500k total files count. > I say replication not mirroring i.e. I need to copy modified files to slaves > almost imidiately after the change. > Tried to find some ready solution but fails. > There is FAM from SGI but it fails to work properly on FreeBSD (it makes silly > stat(2) calls). > kevent(2) also don't have anough API to watch >= 500k files. > So I see the only way is to have kernel module that watch kernel calls for > file operations. > > So is there some other points to build a file replication? Can I suggest you investigate rsync ? It will allow you to replicate file trees pretty much on demand across a LAN or indeed the internet should you choose to. Works over SSH so it's nice and secure and if you need virtually instant replication you could conceivably run this from a cron job every N minutes. /usr/ports/net/rsync HTH LukeK -- Luke Kearney <[EMAIL PROTECTED]> ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
File tree replication in FreeBSD
I'm looking for file replication solution for FreeBSD. I have a task to replicate about 11G of files with 500k total files count. I say replication not mirroring i.e. I need to copy modified files to slaves almost imidiately after the change. Tried to find some ready solution but fails. There is FAM from SGI but it fails to work properly on FreeBSD (it makes silly stat(2) calls). kevent(2) also don't have anough API to watch >= 500k files. So I see the only way is to have kernel module that watch kernel calls for file operations. So is there some other points to build a file replication? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
