Re: Sharding, and all that

2008-12-19 Thread Paul Makepeace
On Fri, Dec 19, 2008 at 1:43 PM, Richard Huxton wrote: > > Andy Wardley wrote: > > Richard Huxton wrote: > >> Yep - that's what "sharding" is all about - separate disconnected silos > >> of data. > > > > I thought sharding specifically related to horizontal partitioning. i.e. > > splitting one ta

Re: Sharding, and all that

2008-12-19 Thread Richard Huxton
Andy Wardley wrote: > Richard Huxton wrote: >> Yep - that's what "sharding" is all about - separate disconnected silos >> of data. > > I thought sharding specifically related to horizontal partitioning. i.e. > splitting one table across several databases, e.g. records with even row > ids in one

Re: Sharding, and all that

2008-12-19 Thread Andy Wardley
Richard Huxton wrote: Yep - that's what "sharding" is all about - separate disconnected silos of data. I thought sharding specifically related to horizontal partitioning. i.e. splitting one table across several databases, e.g. records with even row ids in one DB, odd in another. Apologies i

Re: Sharding, and all that

2008-12-19 Thread James Laver
On Fri, Dec 19, 2008 at 9:52 AM, Richard Huxton wrote: > > Yep - that's what "sharding" is all about - separate disconnected silos > of data. You know, like the ones that were all the rage in the 60s that > drove people to invent RDBMS. The good thing is, if your application is > successful and is

Re: Sharding, and all that

2008-12-19 Thread Richard Huxton
Andy Wardley wrote: > My take on this is that you would have to (re-)build your application and > underlying DB to be distributed from the start. Your user DB/app would > provide an authentication service, perhaps something similar to OpenID, > that > your message board would use to authenticate u

Re: Sharding, and all that

2008-12-19 Thread Andy Wardley
Richard Huxton wrote: Hmm - skimming these articles, I'm not hugely impressed. The chap(ess?) behind them is clearly a developer rather than a DBA. You're right. I perhaps should have quantified that better as a good *introduction* to the subject. It was a bit hand-wavy on the detail, but it

Re: Sharding, and all that

2008-12-18 Thread Nigel Hamilton
HI Mark, So, > > It's nearly the new year and the time for new projects. On my project > radar for next year is "consider (better) sharding our MySQL database." > > I think it depends on your write/read ratio. If it is read heavy then replication and/or memcache will probably do the trick - on th

Re: Sharding, and all that

2008-12-18 Thread Simon Wistow
On Thu, Dec 18, 2008 at 10:19:06AM +, Mark Fowler said: > What's the collective group think on these? Obviously (for this list, at > least) the app that will be accessing the database is Perl based, so I'm > only considering compatible technologies at the moment. We use Data::ObjectDriver w

Re: Sharding, and all that

2008-12-18 Thread Jacqui Caren
Mark Fowler wrote: So, It's nearly the new year and the time for new projects. On my project radar for next year is "consider (better) sharding our MySQL database." Standard consultant question: Why? "go faster/scale up" is not a good reason. If you just wanted to go faster you would have

Re: Sharding, and all that

2008-12-18 Thread James Laver
On Thu, Dec 18, 2008 at 12:49 PM, Jonathan Stowe wrote: > 2008/12/18 Martin A. Brooks : >> Mark Fowler wrote: >>> >>> "consider (better) sharding our MySQL database." >> >> Is that when you throw it against the wall really, really, hard? > > Or make the DBAs roll naked in a bathtub of broken glass

Re: Sharding, and all that

2008-12-18 Thread Paul Makepeace
On Thu, Dec 18, 2008 at 10:19 AM, Mark Fowler wrote: > > So, > > It's nearly the new year and the time for new projects. On my project radar > for next year is "consider (better) sharding our MySQL database." There's a bunch of articles on Flickr out there, this one seems pretty decent, http:/

Re: Sharding, and all that

2008-12-18 Thread Richard Huxton
Jonathan Stowe wrote: > 2008/12/18 Martin A. Brooks : >> Mark Fowler wrote: >>> "consider (better) sharding our MySQL database." >> Is that when you throw it against the wall really, really, hard? > > Or make the DBAs roll naked in a bathtub of broken glass until they > sort the performance proble

Re: Sharding, and all that

2008-12-18 Thread Jonathan Stowe
2008/12/18 Martin A. Brooks : > Mark Fowler wrote: >> >> "consider (better) sharding our MySQL database." > > Is that when you throw it against the wall really, really, hard? Or make the DBAs roll naked in a bathtub of broken glass until they sort the performance problems out.

Re: Sharding, and all that

2008-12-18 Thread Richard Huxton
Andy Wardley wrote: > Mark Fowler wrote: >> What's the collective group think on these? > > There's a good series of articles on sharding starting here: > > http://lifescaler.com/2008/04/database-sharding-unraveled-part-i/ > > The conclusion I drew from it was that functional partitioning > (whe

Re: Sharding, and all that

2008-12-18 Thread Martin A. Brooks
Mark Fowler wrote: "consider (better) sharding our MySQL database." Is that when you throw it against the wall really, really, hard?

Re: Sharding, and all that

2008-12-18 Thread Jonathan Stowe
2008/12/18 Dave Hodgkinson : > > "sharding" is a horrible term. > But the computing 2.0 neophytes have to have obscure neologisms - it would be an affront to their freedom of thought to use a term that everyone else understands.

Re: Sharding, and all that

2008-12-18 Thread Dave Hodgkinson
On 18 Dec 2008, at 10:19, Mark Fowler wrote: So, It's nearly the new year and the time for new projects. On my project radar for next year is "consider (better) sharding our MySQL database." There's really good stuff in the Sybase Unleashed book on horizontal and vertical partitioning

Re: Sharding, and all that

2008-12-18 Thread Andy Wardley
Mark Fowler wrote: What's the collective group think on these? There's a good series of articles on sharding starting here: http://lifescaler.com/2008/04/database-sharding-unraveled-part-i/ The conclusion I drew from it was that functional partitioning (where possible) was much easier to impl

Re: Sharding, and all that

2008-12-18 Thread Leo Lapworth
2008/12/18 Mark Fowler > It's nearly the new year and the time for new projects. On my project > radar for next year is "consider (better) sharding our MySQL database." I came across this and noted it for future reference: http://spockproxy.sourceforge.net/ - doh, just read the rest of your e

Sharding, and all that

2008-12-18 Thread Mark Fowler
So, It's nearly the new year and the time for new projects. On my project radar for next year is "consider (better) sharding our MySQL database." Now I understand the basic principles behind this and can, if needs be, implement my own wheel, but before I do I'd love to find out what ever