> My point is: if I need to be 100% sure that what is referenced on the database
> is accessible all the time when the reference is, then I need to have this on
> the database
Not necessarily. It does take carefully controlling access, with a good deal
of thought and error-checking on the part of
Clodoaldo wrote:
But the main factor to push me in the file system direction is the
HTTP cache management. I want the internet web clients and proxies to
cache the images. The Apache web server has it ready and easy. If
the images where to be stored in the DB I would have to handle the
HTTP cac
Clodoaldo wrote:
But the main factor to push me in the file system direction is the
HTTP cache management. I want the internet web clients and proxies to
cache the images. The Apache web server has it ready and easy. If
the images where to be stored in the DB I would have to handle the
HTTP cac
>>>apache has very good page and image caching. You could take advantage
>>>of that using this technique.
> I wonder why this HTTP cache headers argument didn't surface in this
> heated debate.
I did other up this argument by the way.
Andrew
Clodoaldo wrote:
2007/1/5, Jorge Godoy <[EMAIL PR
2007/1/6, Maurice Aubrey <[EMAIL PROTECTED]>:
Clodoaldo wrote:
> But the main factor to push me in the file system direction is the
> HTTP cache management. I want the internet web clients and proxies to
> cache the images. The Apache web server has it ready and easy. If
> the images where to be
On 1/5/07, Jorge Godoy <[EMAIL PROTECTED]> wrote:
Andrew Chernow <[EMAIL PROTECTED]> writes:
>> And how do you guarantee that after a failure? You're restoring two
>> different sets of data here:
>
>> How do you link them together on that specific operation? Or even on a daily
>> basis, if you
Is there any overwhelming reason you can't just stick an apache server
on your DB server? Unless you expect this thing to get hit *hard*, the
performance of having them both on the same machine is pretty acceptable
(I know, everyone's opinion about what constitutes heavy usage
differs). If th
> I wonder why this HTTP cache headers argument didn't surface in this
> heated debate.
I mentioned this earlier as well. Although you could do it in the app
layer - it would be easier to just let the web server handle it.
---(end of broadcast)---
On 1/6/07, Jorge Godoy <[EMAIL PROTECTED]> wrote:
Andrew Chernow <[EMAIL PROTECTED]> writes:
>>I mean, how do you handle integrity with data
>> outside the database?
> You don't, the file system handles integrity of the stored data. Although,
> one must careful to avoid db and fs orphans. Mean
2007/1/6, Andrew Chernow <[EMAIL PROTECTED]>:
>>>apache has very good page and image caching. You could take advantage
>>>of that using this technique.
> I wonder why this HTTP cache headers argument didn't surface in this
> heated debate.
I did other up this argument by the way.
Sorry,
2007/1/5, Jorge Godoy <[EMAIL PROTECTED]>:
Andrew Chernow <[EMAIL PROTECTED]> writes:
> meet those requirements. It is far more effecient to have apache access
> them
Where weren't we meeting his/her requirements? All the discussion is around
available means to do that. One option is having t
Clodoaldo wrote:
> 5 Jan 2007 06:59:18 -0800, imageguy <[EMAIL PROTECTED]>:
> >
> > I think I know the answer,
>
> If you know the answer please tell it as I have read some discussions
> on the web and although I have decided on a solution I'm still not
> sure about the best answer, if there is a
Andrew Chernow <[EMAIL PROTECTED]> writes:
> depends on how you do it. not everything has to go in a database to be
> reliable. Part of good engineering is realizing when to use a tool and when
> not to.
>
> I think a 10K view of the issue is in order. The bigger picture is being
> missed, or I
Andrew Chernow <[EMAIL PROTECTED]> writes:
> I wasn't saying to do this each time you run a backup, geez that would be
> horrible. Pickup from where you left off the last time you backed up
> data/records. How many images and how much data is being generated in a 60
> second period? I dought 3
> And introducint more failing points.
depends on how you do it. not everything has to go in a database to be
reliable. Part of good engineering is realizing when to use a tool and when not
to.
I think a 10K view of the issue is in order. The bigger picture is being
missed, or I am not com
Andrew Chernow <[EMAIL PROTECTED]> writes:
>>> Or am
>>>I the only one that is thinking about referential integrity with those files?
> Not at all. I'm not sure how 3rd party tools like apache, `ls`, `gzip`,
> `find`, nfs, etc... are breaking integrity. Any php, jsp, C or shell script
For gzip,
Ragnar <[EMAIL PROTECTED]> writes:
> for that matter, what happens if one transaction is using or
> even reading an image while another is updating it?
I believe that this also depends on how the file is updated. Some
applications create a temporary file with the new (or changed) content and
the
> copying 3 billion files and a few hundred terabytes while still maintaining an
> adequate service rate with part of its infra-structure down, just to use your
I wasn't saying to do this each time you run a backup, geez that would be
horrible. Pickup from where you left off the last time you b
> what happens if you rollback a transaction that just updated
> an image file?
>
> for that matter, what happens if one transaction is using or
> even reading an image while another is updating it?
One thing I mentioned was about a point in time backup, not updating the image.
This would rollb
On fös, 2007-01-05 at 15:49 -0500, Andrew Chernow wrote:
> I 100% agree. Use the database as a lookup into the filesystem. Don't load
> the
> database up with terabytes of non-searchable binary data? not sure how that
> would help you?
>
> >I mean, how do you handle integrity with data
>
Referential integrity would be one positive for storing the files in the
database. I wasn't responding to that. I'm simply saying that browsing
them with third-party tools, updating, compressing/archiving, etc. is
easier if they are not in the database. Those are all actions that I've
found usef
Andrew Chernow <[EMAIL PROTECTED]> writes:
>> And how do you guarantee that after a failure? You're restoring two
>> different sets of data here:
>
>> How do you link them together on that specific operation? Or even on a daily
>> basis, if you get corrupted data...
>
> I answered that already.
>> Or am
>>I the only one that is thinking about referential integrity with those files?
Not at all. I'm not sure how 3rd party tools like apache, `ls`, `gzip`, `find`,
nfs, etc... are breaking integrity. Any php, jsp, C or shell script you write
would be doing the same thing, accessing the da
If you end up storing the pictures on a separate database server for
performance reasons, Jorge's argument regarding integrity becomes
moot...And his argument so far is the strongest case I've seen for
storing the files in the database.
Scott Marlowe wrote:
On Fri, 2007-01-05 at 15:54, Jerem
"Jeremy Haile" <[EMAIL PROTECTED]> writes:
> Another thing to consider is that storing them in the file system makes
> it much easier to browse the images using third-party tools, update
> them, archive them (by gzipping or whatever). This is much more
> difficult if they are stored in the databa
On Fri, 2007-01-05 at 15:54, Jeremy Haile wrote:
> Yeah - it can make it easier to implement transactional semantics by
> storing them in the database, although for simple operations it wouldn't
> be hard to replicate this manually. And you are going to incur a
> performance penalty by storing the
Yeah - it can make it easier to implement transactional semantics by
storing them in the database, although for simple operations it wouldn't
be hard to replicate this manually. And you are going to incur a
performance penalty by storing them in the database.
Another thing to consider is that s
On Fri, Jan 05, 2007 at 15:26:45 -0500,
James Neff <[EMAIL PROTECTED]> wrote:
> "... and Moses said unto them, 'The eleventh commandment : thou shalt
> store images in a database!'..."
>
> What if you had another database where you stored just the images and
> not back it up if you don't want
> And how do you guarantee that after a failure? You're restoring two
> different sets of data here:
> How do you link them together on that specific operation? Or even on a daily
> basis, if you get corrupted data...
I answered that already.
> Not counting that depending on your choice of fi
Andrew Chernow <[EMAIL PROTECTED]> writes:
>>I mean, how do you handle integrity with data
>> outside the database?
> You don't, the file system handles integrity of the stored data. Although,
> one must careful to avoid db and fs orphans. Meaning, a record with no
> corresponding file or a file
John McCawley <[EMAIL PROTECTED]> writes:
> This is a web app, so in my example all of the images live on a web server,
> and our data lives on a separate database server. We have a completely
> duplicated setup offsite, and mirror images of every server at the backup
> site. Every night we use
How does it make it easier to control access and security? If your web
app makes a decision about allowing access to the database, it can just
as easily make a decision about allowing access to the filesystem.
Storing the images on the file system doesn't mean that there isn't a
piece of code tha
My comment about stark raving madmen was somewhat tongue-in-cheek.
There is certainly a case to be made for images in a database under some
circumstances. However, for the average Joe web developer, and
certainly someone new to the whole problem, I think storing them on the
filesystem is goin
cents,
-Jeanna
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of James Neff
Sent: Friday, January 05, 2007 2:27 PM
To: John McCawley
Cc: Clodoaldo; imageguy; pgsql-general@postgresql.org
Subject: Re: [GENERAL] Database versus filesystem for storing images
"...
This is a web app, so in my example all of the images live on a web
server, and our data lives on a separate database server. We have a
completely duplicated setup offsite, and mirror images of every server
at the backup site. Every night we use rsync to duplicate everything
offsite. Also, a
>> Don't store your images in the database. Store them on the filesystem and
>> store their path in the database
I 100% agree. Use the database as a lookup into the filesystem. Don't load the
database up with terabytes of non-searchable binary data? not sure how that
would help you?
Here i
"... and Moses said unto them, 'The eleventh commandment : thou shalt
store images in a database!'..."
What if you had another database where you stored just the images and
not back it up if you don't want to?
As an application developer, I like the idea of storing files and images
in the d
John McCawley <[EMAIL PROTECTED]> writes:
> Don't store your images in the database. Store them on the filesystem and
> store their path in the database. Anyone that tells you otherwise is a stark
> raving madman :)
>
> My system is very heavily used, and our pg_dump is only a few gigs. Meanwhi
Don't store your images in the database. Store them on the filesystem
and store their path in the database. Anyone that tells you otherwise
is a stark raving madman :)
My system is very heavily used, and our pg_dump is only a few gigs.
Meanwhile our images/documents storage is well over a h
It's almost always better to store the images on the file system and
just store the filename or relative path in the database.
This is more efficient, doesn't bloat the database by storing files in
it, and is easier to get proper browser caching behavior (depending on
how your app is setup). I
5 Jan 2007 06:59:18 -0800, imageguy <[EMAIL PROTECTED]>:
I think I know the answer,
If you know the answer please tell it as I have read some discussions
on the web and although I have decided on a solution I'm still not
sure about the best answer, if there is a best answer after all.
but if
Scott Ribe wrote:
> Personally, I'd put them on the file system, because then backup software
> can perform incremental backups. In the database, that becomes more of a
> difficulty. One suggestion, don't use a file name from a hash to store the
> image, just use the serial id, and break them up b
Personally, I'd put them on the file system, because then backup software
can perform incremental backups. In the database, that becomes more of a
difficulty. One suggestion, don't use a file name from a hash to store the
image, just use the serial id, and break them up by hundreds or thousands,
io
Clodoaldo wrote:
A web application will store hundreds of thousands images.
What is the best practice in this situation? I mean not only the two
options as above but any.
This discussion comes up regularly. See the archives for a thread
titled "Storing images in PostgreSQL databases (aga
A web application will store hundreds of thousands images.
The images meta data will be stored in postgresql but I'm in doubt
about where to store the images themselves, if in the database as byte
strings or in the file system.
Would one of them have a faster image search than the other?
In the
45 matches
Mail list logo
