Re: HD Mirroring
On Mon, 29 Nov 2004, Andre Oppermann wrote: If you have many TCP connections to one target it may happen that you get the same source port on the originator again within the TIME_WAIT timeout. And if the ISN wrapped in the meantime the new connection will 'hang'. Just to clear this up, the problem with randomized source ports and TIME_WAIT is not that the ISN is wrapping. The problem is that if a port is reused too quickly, the ISN has not incremented enough and is less than the final sequence number of the previous connection. There's code in 5.3 which eliminates this problem by incrementing a global offset for each connection established, I will probably MFC it before 4.11 so that this problem is over with once and for all. Mike "Silby" Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: HD Mirroring
On Mon, 2004-11-29 at 12:24, Andre Oppermann wrote: ...(snip)... > > Perhaps somebody else can do a more thorough test of MySQL clustering on > > FreeBSD to make sure that it is in fact fully stable. It seems like a > > remarkable system, assuming you have the gigs of RAM it takes to run it > > with a table of any substantial size... > > But that is an application problem, not FreeBSD's fault. I certainly did not mean to imply that this was a problem with FreeBSD, nor even a problem in MySQL Cluster, I simply meant that the RAM requirements for MySQL Cluster are higher than one might guess. -- Justin Hopper <[EMAIL PROTECTED]> UNIX Systems Engineer BSDHosting.net Hosting Division of Digital Oasys Inc. http://www.bsdhosting.net ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: HD Mirroring
Justin Hopper wrote: On Wed, 2004-11-24 at 23:08, Justin Hopper wrote: On Wed, 2004-11-24 at 13:31, Charles Sprickman wrote: On Wed, 24 Nov 2004, Brian Reichert wrote: And, although I've not tested it, recent versions of MySQL can outright support a cluster: http://dev.mysql.com/doc/mysql/en/NDBCluster.html I'm just curious if there's any other solution that will work on FreeBSD. I have about 5 mysql servers (4 slaves, 1 master) and one application in particular is not smart enough to try other servers if the configured server does not answer. Is there any type of local proxy that can intelligently route requests to the "best" server? I too was curious about the MySQL Clustering support and its status on FreeBSD, since it wasn't as a supported OS. Over the last couple of hours I was able to set up a cluster consisting of a management process and data node running in one jail, and a MySQL server and another data node running in a different jail. Once everything was up and running, the cluster seemed to be working excellent, data was synchronizing flawlessly throughout the cluster. Nuking either of the data node processes did not affect access to the data in the cluster, so failover seemed to be working as well. The only problem that I ran into, and it may be user error on my part, is that when the cluster is shut down (or all data node processes are killed), the data contained in the node is lost when the cluster is brought back online. Perhaps there is some recovery step that is required before the cluster can be used again. If someone else has already tested MySQL's clustering ability with FreeBSD, then please let us know the results so that I don't recreate the wheel here. If not, I'll continue seeing how far I can get with it, as I would definitely like to implement this functionality on several of the more critical databases that I manage. I'm sure it's taboo to respond to one's own message, but thought I would follow up with some information on the problems I was running into with MySQL Cluster. The first problem, where it appeared that the data in the cluster was lost when the cluster was shut down, turned out to be there are some problems with the MySQL servers, which act as API clients to the cluster, reliably connecting into the cluster. Several times I could not get a MySQL server to connect to the cluster, but found no rhyme or reason for it so far. The cluster seems to be retaining data just fine upon shutdown, when the MySQL servers can actually connect to it to query data that is... If you have many TCP connections to one target it may happen that you get the same source port on the originator again within the TIME_WAIT timeout. And if the ISN wrapped in the meantime the new connection will 'hang'. The second problem I encountered was while trying to load a table that was 163MB in size that contained around 1 million rows. The NDB cluster would continually report that the table was "full" when trying to import the data. After checking around on mailing lists, I found out that the NDB clustering engine will require around table_size*2*10% RAM to load a table. NDB keeps all of the data in main memory, and has a fair amount of overhead per row. Perhaps somebody else can do a more thorough test of MySQL clustering on FreeBSD to make sure that it is in fact fully stable. It seems like a remarkable system, assuming you have the gigs of RAM it takes to run it with a table of any substantial size... But that is an application problem, not FreeBSD's fault. -- Andre ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: HD Mirroring
On Wed, 2004-11-24 at 23:08, Justin Hopper wrote: > On Wed, 2004-11-24 at 13:31, Charles Sprickman wrote: > > On Wed, 24 Nov 2004, Brian Reichert wrote: > > > > > And, although I've not tested it, recent versions of MySQL can > > > outright support a cluster: > > > > > > http://dev.mysql.com/doc/mysql/en/NDBCluster.html > > > > I'm just curious if there's any other solution that will work on FreeBSD. > > I have about 5 mysql servers (4 slaves, 1 master) and one application in > > particular is not smart enough to try other servers if the configured > > server does not answer. Is there any type of local proxy that can > > intelligently route requests to the "best" server? > > > I too was curious about the MySQL Clustering support and its status on > FreeBSD, since it wasn't as a supported OS. Over the last couple of > hours I was able to set up a cluster consisting of a management process > and data node running in one jail, and a MySQL server and another data > node running in a different jail. Once everything was up and running, > the cluster seemed to be working excellent, data was synchronizing > flawlessly throughout the cluster. Nuking either of the data node > processes did not affect access to the data in the cluster, so failover > seemed to be working as well. > > The only problem that I ran into, and it may be user error on my part, > is that when the cluster is shut down (or all data node processes are > killed), the data contained in the node is lost when the cluster is > brought back online. Perhaps there is some recovery step that is > required before the cluster can be used again. > > If someone else has already tested MySQL's clustering ability with > FreeBSD, then please let us know the results so that I don't recreate > the wheel here. If not, I'll continue seeing how far I can get with it, > as I would definitely like to implement this functionality on several of > the more critical databases that I manage. I'm sure it's taboo to respond to one's own message, but thought I would follow up with some information on the problems I was running into with MySQL Cluster. The first problem, where it appeared that the data in the cluster was lost when the cluster was shut down, turned out to be there are some problems with the MySQL servers, which act as API clients to the cluster, reliably connecting into the cluster. Several times I could not get a MySQL server to connect to the cluster, but found no rhyme or reason for it so far. The cluster seems to be retaining data just fine upon shutdown, when the MySQL servers can actually connect to it to query data that is... The second problem I encountered was while trying to load a table that was 163MB in size that contained around 1 million rows. The NDB cluster would continually report that the table was "full" when trying to import the data. After checking around on mailing lists, I found out that the NDB clustering engine will require around table_size*2*10% RAM to load a table. NDB keeps all of the data in main memory, and has a fair amount of overhead per row. Perhaps somebody else can do a more thorough test of MySQL clustering on FreeBSD to make sure that it is in fact fully stable. It seems like a remarkable system, assuming you have the gigs of RAM it takes to run it with a table of any substantial size... -- Justin Hopper <[EMAIL PROTECTED]> UNIX Systems Engineer BSDHosting.net Hosting Division of Digital Oasys Inc. http://www.bsdhosting.net ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: HD Mirroring
Have you looked into http://balance.sourceforge.net/ ? > I'm just curious if there's any other solution that will work on FreeBSD. > I have about 5 mysql servers (4 slaves, 1 master) and one application in > particular is not smart enough to try other servers if the configured > server does not answer. Is there any type of local proxy that can > intelligently route requests to the "best" server? > > Thanks, > > Charles -- //| //|| // | // || -//--//---|| ARIO LOBO // //|| - [EMAIL PROTECTED] http://www.ipad.com.br ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: HD Mirroring
On Wed, 2004-11-24 at 13:31, Charles Sprickman wrote: > On Wed, 24 Nov 2004, Brian Reichert wrote: > > > And, although I've not tested it, recent versions of MySQL can > > outright support a cluster: > > > > http://dev.mysql.com/doc/mysql/en/NDBCluster.html > > I'm just curious if there's any other solution that will work on FreeBSD. > I have about 5 mysql servers (4 slaves, 1 master) and one application in > particular is not smart enough to try other servers if the configured > server does not answer. Is there any type of local proxy that can > intelligently route requests to the "best" server? > I too was curious about the MySQL Clustering support and its status on FreeBSD, since it wasn't as a supported OS. Over the last couple of hours I was able to set up a cluster consisting of a management process and data node running in one jail, and a MySQL server and another data node running in a different jail. Once everything was up and running, the cluster seemed to be working excellent, data was synchronizing flawlessly throughout the cluster. Nuking either of the data node processes did not affect access to the data in the cluster, so failover seemed to be working as well. The only problem that I ran into, and it may be user error on my part, is that when the cluster is shut down (or all data node processes are killed), the data contained in the node is lost when the cluster is brought back online. Perhaps there is some recovery step that is required before the cluster can be used again. If someone else has already tested MySQL's clustering ability with FreeBSD, then please let us know the results so that I don't recreate the wheel here. If not, I'll continue seeing how far I can get with it, as I would definitely like to implement this functionality on several of the more critical databases that I manage. -- Justin Hopper <[EMAIL PROTECTED]> UNIX Systems Engineer BSDHosting.net Hosting Division of Digital Oasys Inc. http://www.bsdhosting.net ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: HD Mirroring
On Wednesday, 24. November 2004 22:31, Charles Sprickman wrote: > On Wed, 24 Nov 2004, Brian Reichert wrote: > > And, although I've not tested it, recent versions of MySQL can > > outright support a cluster: > > > > http://dev.mysql.com/doc/mysql/en/NDBCluster.html > I'm just curious if there's any other solution that will work on FreeBSD. > I have about 5 mysql servers (4 slaves, 1 master) and one application in > particular is not smart enough to try other servers if the configured > server does not answer. Is there any type of local proxy that can > intelligently route requests to the "best" server? maybe use CARP for that? cheers simon -- /"\ \ / \ ASCII Ribbon Campaign / \ Against HTML Mail and News pgp3qrXgKV7fX.pgp Description: PGP signature
Re: HD Mirroring
On Wed, 24 Nov 2004, Brian Reichert wrote: And, although I've not tested it, recent versions of MySQL can outright support a cluster: http://dev.mysql.com/doc/mysql/en/NDBCluster.html I'm just curious if there's any other solution that will work on FreeBSD. I have about 5 mysql servers (4 slaves, 1 master) and one application in particular is not smart enough to try other servers if the configured server does not answer. Is there any type of local proxy that can intelligently route requests to the "best" server? Thanks, Charles Thanks before-hand for any info, -- //| //|| // | // || -//--//---|| ARIO LOBO // //|| - [EMAIL PROTECTED] http://www.ipad.com.br -- Brian Reichert <[EMAIL PROTECTED]> 37 Crystal Ave. #303Daytime number: (603) 434-6842 Derry NH 03038-1713 USA BSD admin/developer at large ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: HD Mirroring
Ok Brian; I read about NDBClusters but the info I found said it would only run on linux and solaris. No FreeBSD. I didn´t know of mysql replication ! I´ll check into it too, but what i´ve been reading about vinum so far has impressed me a lot, specially in terms of not only mirroring the database but most anything you want. Maybe I´ll try to replicate the database on vinum volumes to get the best service performance from mysql, with the "failsafetyness" of vinum mirrors. Thanks for your help, On 24 Nov 2004 at 12:13, Brian Reichert wrote: > On Wed, Nov 24, 2004 at 10:52:11AM -0300, [EMAIL PROTECTED] wrote: > > I have a mysql server that I need to mirror its data on a 2nd HD, either on > > the same machine or on > > a remote one, but it has to be in real time. Has it been or can it be done > > at all? > > Is MySQL's replication not fast enough? I can see why it wouldn't > be in all cases, but I wanted to make sure you were aware of it: > > http://dev.mysql.com/doc/mysql/en/Replication.html > > And, although I've not tested it, recent versions of MySQL can > outright support a cluster: > > http://dev.mysql.com/doc/mysql/en/NDBCluster.html > > > Thanks before-hand for any info, > -- > Brian Reichert<[EMAIL PROTECTED]> > 37 Crystal Ave. #303 Daytime number: (603) 434-6842 > Derry NH 03038-1713 USA BSD admin/developer at large -- //| //|| // | // || -//--//---|| ARIO LOBO // //|| - [EMAIL PROTECTED] http://www.ipad.com.br ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: HD Mirroring
On Wed, Nov 24, 2004 at 10:52:11AM -0300, [EMAIL PROTECTED] wrote: > I have a mysql server that I need to mirror its data on a 2nd HD, either on > the same machine or on > a remote one, but it has to be in real time. Has it been or can it be done at > all? Is MySQL's replication not fast enough? I can see why it wouldn't be in all cases, but I wanted to make sure you were aware of it: http://dev.mysql.com/doc/mysql/en/Replication.html And, although I've not tested it, recent versions of MySQL can outright support a cluster: http://dev.mysql.com/doc/mysql/en/NDBCluster.html > Thanks before-hand for any info, > > -- >//| //|| > // | // || > -//--//---|| ARIO LOBO > // //|| > - > [EMAIL PROTECTED] > http://www.ipad.com.br -- Brian Reichert <[EMAIL PROTECTED]> 37 Crystal Ave. #303Daytime number: (603) 434-6842 Derry NH 03038-1713 USA BSD admin/developer at large ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: HD Mirroring
Thanks Peter and Ciprian for the directions !! following through right now !! Best regards, -- //| //|| // | // || -//--//---|| ARIO LOBO // //|| - [EMAIL PROTECTED] http://www.ipad.com.br ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: HD Mirroring
On Wed, Nov 24, 2004 at 10:52:11AM -0300, [EMAIL PROTECTED] wrote: > Hi; > > Don?t know if this is the proper list to ask. If not, please forgive me. > > Does anybody has any hints on where to go for info about HD real time > mirroring in FreeBSD? > > I have a mysql server that I need to mirror its data on a 2nd HD, > either on the same machine or on a remote one, but it has to be in > real time. Has it been or can it be done at all? Actually, it has been done, and it has been working for quite some time, in several different versions even :) Take a look at the FreeBSD Handbook: http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/ Look at chapter 16, "Storage", and especially 16.5, "RAID". After that, look at chapter 17, "The Vinum Volume Manager". Those will probably contain most of the information that you need :) There is also an excellent separate article about Vinum at http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/vinum/ Hope that helps! G'luck, Peter -- Peter Pentchev [EMAIL PROTECTED][EMAIL PROTECTED][EMAIL PROTECTED] PGP key:http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553 Hey, out there - is it *you* reading me, or is it someone else? pgpPb4IyFgkeY.pgp Description: PGP signature
HD Mirroring
Hi; Don´t know if this is the proper list to ask. If not, please forgive me. Does anybody has any hints on where to go for info about HD real time mirroring in FreeBSD? I have a mysql server that I need to mirror its data on a 2nd HD, either on the same machine or on a remote one, but it has to be in real time. Has it been or can it be done at all? Thanks before-hand for any info, -- //| //|| // | // || -//--//---|| ARIO LOBO // //|| - [EMAIL PROTECTED] http://www.ipad.com.br ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
