Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-12 Thread Alexander Staubo
On Oct 13, 2006, at 01:36 , Robert L Mathews wrote: Michelle Konzack <[EMAIL PROTECTED]> wrote: I do this already but have problems since I have stored arround 130 million files on a server... > ... > MD5 hashes are 32 Bytes long, maybe they change it to 64 Bytes? I have already over 2000 co

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-12 Thread Robert L Mathews
Michelle Konzack <[EMAIL PROTECTED]> wrote: I do this already but have problems since I have stored arround 130 million files on a server... > ... > MD5 hashes are 32 Bytes long, maybe they change it to 64 Bytes? I have already over 2000 collisions and checked it, that the files are NOT the

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-12 Thread Bill Moran
In response to Michelle Konzack <[EMAIL PROTECTED]>: > Am 2006-10-05 21:22:04, schrieb Gregory S. Williamson: > > > a sufficiently large hash (MD5 for instance). Of course, there's > > I do this already but have problems since I have > stored arround 130 million files on a server... > > > time

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-12 Thread Michelle Konzack
Am 2006-10-05 21:22:04, schrieb Gregory S. Williamson: > a sufficiently large hash (MD5 for instance). Of course, there's I do this already but have problems since I have stored arround 130 million files on a server... > time to create the hash which might be an issue in a high volume > system.

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-08 Thread Karsten Hilbert
On Thu, Oct 05, 2006 at 05:08:27PM -0600, Leonel Nunez wrote: > Subject: Re: [GENERAL] Storing images in PostgreSQL databases (again) > User-Agent: SquirrelMail/1.4.8 > > > Leonel Nunez wrote: > >>> I think the arguments for keeping stuff inside the database are >

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-06 Thread Gregory S. Williamson
values. G -Original Message- From: [EMAIL PROTECTED] on behalf of Jean-Christophe Roux Sent: Thu 10/5/2006 4:54 PM To: Alexander Staubo Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] Storing images in PostgreSQL databases (again) Why would I set a bytea column (

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-06 Thread Jacob Coby
006 7:55 PM To: Alexander Staubo Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] Storing images in PostgreSQL databases (again)   Why would I set a bytea column (containing picures) as a primary key? Because I want to be sure that the same image is inserted only once (that requirement

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-06 Thread Merlin Moncure
On 10/5/06, Jean-Christophe Roux <[EMAIL PROTECTED]> wrote: Why would I set a bytea column (containing picures) as a primary key? Because I want to be sure that the same image is inserted only once (that requirement comes from a real project) and using a primary key for that purpose makes sense

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-06 Thread Ron Johnson
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/05/06 18:54, Jean-Christophe Roux wrote: > Why would I set a bytea column (containing picures) as a primary > key? Because I want to be sure that the same image is inserted > only once (that requirement comes from a real project) and using > a pr

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-06 Thread Jean-Christophe Roux
Jean-Christophe Roux <[EMAIL PROTECTED]>Cc: pgsql-general@postgresql.orgSent: Thursday, October 5, 2006 7:35:04 PMSubject: Re: [GENERAL] Storing images in PostgreSQL databases (again)On Oct 6, 2006, at 01:29 , Jean-Christophe Roux wrote:> By the way, is it practical to set a bytea column (c

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Gregory S. Williamson
[EMAIL PROTECTED] on behalf of Jean-Christophe Roux Sent: Thu 10/5/2006 4:29 PM To: pgsql-general@postgresql.org Cc: Subject: Re: [GENERAL] Storing images in PostgreSQL databases (again) Hi, If the database had built-in functions to manipulate images (make a thumbnail, add t

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Ron Johnson
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/05/06 19:41, Bill Moran wrote: > "Leonel Nunez" <[EMAIL PROTECTED]> wrote: >>> If the database had built-in functions to manipulate images (make a >>> thumbnail, add text ont it.., make a montage of two pictures) and I could >>> write something l

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Leonel Nunez
> "Leonel Nunez" <[EMAIL PROTECTED]> wrote: >> >> > If the database had built-in functions to manipulate images (make a >> > thumbnail, add text ont it.., make a montage of two pictures) and I >> could >> > write something like >> > select thumbnail(image_field, 100, 100) from images_table >> > tha

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Bill Moran
"Leonel Nunez" <[EMAIL PROTECTED]> wrote: > > > If the database had built-in functions to manipulate images (make a > > thumbnail, add text ont it.., make a montage of two pictures) and I could > > write something like > > select thumbnail(image_field, 100, 100) from images_table > > that would be

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Leonel Nunez
> Hi, > If the database had built-in functions to manipulate images (make a > thumbnail, add text ont it.., make a montage of two pictures) and I could > write something like > select thumbnail(image_field, 100, 100) from images_table > that would be a good reason to go the db route versus the file

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Alexander Staubo
On Oct 6, 2006, at 01:29 , Jean-Christophe Roux wrote: By the way, is it practical to set a bytea column (containing pictures) as primary key? That would severely slow down many operations I guess. Why would you? It's possible, but completely impractical, since image data typically exceed

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Jean-Christophe Roux
2006 6:30:07 PMSubject: Re: [GENERAL] Storing images in PostgreSQL databases (again)On Oct 5, 2006, at 19:47 , DEV wrote:> I have seen several posts pertaining to the "overhead" difference  > in storing> in a db table versus the file system.  What is this difference?Well, the

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Leonel Nunez
> Leonel Nunez wrote: >>> I think the arguments for keeping stuff inside the database are >>> (a) far easier to maintain transactional semantics for insert/delete, >>> and (b) easier to serve the data out to clients that aren't on the >>> same machine. You aren't going to find a performance win th

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Alexander Staubo
On Oct 5, 2006, at 19:47 , DEV wrote: I have seen several posts pertaining to the "overhead" difference in storing in a db table versus the file system. What is this difference? Well, there's not much space overhead to speak of. I tested with a bunch of JPEG files: $ find files | wc -l

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Guy Rouillier
Leonel Nunez wrote: >> I think the arguments for keeping stuff inside the database are >> (a) far easier to maintain transactional semantics for insert/delete, >> and (b) easier to serve the data out to clients that aren't on the >> same machine. You aren't going to find a performance win though.

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread DEV
3:57 PM To: pgsql-general@postgresql.org Subject: Re: [GENERAL] Storing images in PostgreSQL databases (again) TIJod wrote: > I need to store a large number of images in a > PostgreSQL database. In my application, this > represents a few hundreds of thousands of images. The > size of e

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread John Sidney-Woollett
Merlin Moncure wrote: i'm wondering if anybody has ever attempted to manage large collections of binary objects inside the database and has advice here. We have designed and built an image library using Postgres and NFS servers which currently holds 1.4 million images totalling more than 250Gb

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Leonel Nunez
> "Merlin Moncure" <[EMAIL PROTECTED]> writes: >> ... postgresql will toast all images over a cerain size which >> is actually pretty efficient although can be a problem if your images >> are really big. > > But any reasonable image format is compressed already (or at least, if > you are using an u

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Chris Browne
[EMAIL PROTECTED] ("Marco Bizzarri") writes: > Hi. > > I can provide some "insight" on the difference between the two interfaces. > > AFAIK, the difference is in size of the file you can store, and in the > interface you have when you want to access. > > The size is not important (I think), since y

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Merlin Moncure
On 10/5/06, Alexander Staubo <[EMAIL PROTECTED]> wrote: On Oct 5, 2006, at 16:18 , Merlin Moncure wrote: > i'm wondering if anybody has ever attempted to manage large > collections of binary objects inside the database and has advice here. We have a production system containing 10,000 images (JP

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Alexander Staubo
On Oct 5, 2006, at 16:18 , Merlin Moncure wrote: I see little value to storing the images in the database. For me that's a general statement (I'm sure others will disagree); but especially in your case, where you have a high volume and only want to store them for a couple days. Why incur

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: > ... postgresql will toast all images over a cerain size which > is actually pretty efficient although can be a problem if your images > are really big. But any reasonable image format is compressed already (or at least, if you are using an uncompresse

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Merlin Moncure
On 10/4/06, Guy Rouillier <[EMAIL PROTECTED]> wrote: TIJod wrote: > I need to store a large number of images in a > PostgreSQL database. In my application, this > represents a few hundreds of thousands of images. The > size of each image is about 100-200 Ko. There is a > large turnover in my data

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-05 Thread Marco Bizzarri
Hi. I can provide some "insight" on the difference between the two interfaces. AFAIK, the difference is in size of the file you can store, and in the interface you have when you want to access. The size is not important (I think), since you are far below the limit. For the interface, the bytea

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-04 Thread Jack Orenstein
On 10/4/06, TIJod <[EMAIL PROTECTED]> wrote: I think I'm about to ask a traditional (almost religious) question, but I haven't been able to find a crystal clear answer in the mailing lists so far. I think the key in deciding this, in your case, is your requirement for space reclamation: There

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-04 Thread Steve Atkins
On Oct 4, 2006, at 12:56 PM, Guy Rouillier wrote: TIJod wrote: I need to store a large number of images in a PostgreSQL database. In my application, this represents a few hundreds of thousands of images. The size of each image is about 100-200 Ko. There is a large turnover in my database, i.e.

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-04 Thread Guy Rouillier
TIJod wrote: > I need to store a large number of images in a > PostgreSQL database. In my application, this > represents a few hundreds of thousands of images. The > size of each image is about 100-200 Ko. There is a > large turnover in my database, i.e. each image stays > about 1 week in the datab

Re: [GENERAL] Storing images in PostgreSQL databases (again)

2006-10-04 Thread Leonel Nunez
> Hello, > > I think I'm about to ask a traditional (almost > religious) question, but I haven't been able to find a > crystal clear answer in the mailing lists so far. > Thus, here is my question: > > I need to store a large number of images in a > PostgreSQL database. In my application, this > re

[GENERAL] Storing images in PostgreSQL databases (again)

2006-10-04 Thread TIJod
Hello, I think I'm about to ask a traditional (almost religious) question, but I haven't been able to find a crystal clear answer in the mailing lists so far. Thus, here is my question: I need to store a large number of images in a PostgreSQL database. In my application, this represents a few hun