Re: [SQL] Question about POSIX Regular Expressions performance on large dataset.

2010-08-17 Thread Sergey Konoplev
On 18 August 2010 06:30, Jose Ildefonso Camargo Tolosa wrote: > Hi, again, > > I just had this wacky idea, and wanted to share it: > > what do you think of having the dataset divided among several servers, > and sending the query to all of them, and then just have the > application "unify" the res

Re: [SQL] Question about POSIX Regular Expressions performance on large dataset.

2010-08-17 Thread Scott Marlowe
You can do something similar on the same machine if you can come up with a common way to partition your data. Then you split your 1B rows up into chunks of 10M or so and put each on a table and hit the right table. You can use partitioning / table inheritance if you want to, or just know the tabl

Re: [SQL] Question about POSIX Regular Expressions performance on large dataset.

2010-08-17 Thread Jose Ildefonso Camargo Tolosa
Hi, again, I just had this wacky idea, and wanted to share it: what do you think of having the dataset divided among several servers, and sending the query to all of them, and then just have the application "unify" the results from all the servers? Would that work for this kind of *one table* se

Re: [SQL] Question about POSIX Regular Expressions performance on large dataset.

2010-08-17 Thread Scott Marlowe
On Tue, Aug 17, 2010 at 8:21 PM, Jose Ildefonso Camargo Tolosa wrote: > Hi! > > I'm analyzing the possibility of using PostgreSQL to store a huge > amount of data (around 1000M records, or so), and these, even > though are short (each record just have a timestamp, and a string that > is less t

[SQL] Question about POSIX Regular Expressions performance on large dataset.

2010-08-17 Thread Jose Ildefonso Camargo Tolosa
Hi! I'm analyzing the possibility of using PostgreSQL to store a huge amount of data (around 1000M records, or so), and these, even though are short (each record just have a timestamp, and a string that is less than 128 characters in length), the strings will be matched against POSIX Regular E