Re: RE: image storage confusion ?? -- UUDECODE
I think that if you use Bfile oracle will not copy them to the standby database. It will only copy the pointers in the database. You will have to copy the files yourself. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 10, 2002 3:23 AM Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from
Re: Re: Re: RE: image storage confusion ?? -- UUDECODE
On NTFS there is an allocation unit and the Os gives space in multiple of allocations units which is 4k or 8k. So if you use bfile you will waste 3k for each file, increasing your disk space by 60%. I am not sure about the size of the allocation unit. Please verify this with your sysadmin. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 10, 2002 8:03 AM Thanx Chao. Still under confusion , whether to store 20,000,000 images of 5k each either in -oracle 8.1.6 on win2k ( BLOB or BFILE ) OR - just on plain linux file system - OR - any file server like NetApp on linux --- which will be better ? how do people usually handle data of such volume ? plz let me know how it is done normally. TIA. Jp. On Tue, 10 Dec 2002 chao_ping wrote : oraora oraora, For blob, it is ok, since it is all in the database, while for bfile, you actually store them in filesystem, so you have to backup those files indivudually. Regards zhu chao [EMAIL PROTECTED] www.happyit.net www.cnoug.org(Chinese Oracle User Group) === 2002-12-09 17:23:00 ,you wrote£º=== Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files
Re: RE: image storage confusion ?? -- UUDECODE
Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED]
Re: Re: RE: image storage confusion ?? -- UUDECODE
oraora oraora, For blob, it is ok, since it is all in the database, while for bfile, you actually store them in filesystem, so you have to backup those files indivudually. Regards zhu chao [EMAIL PROTECTED] www.happyit.net www.cnoug.org(Chinese Oracle User Group) === 2002-12-09 17:23:00 ,you wrote£º=== Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be
Re: Re: Re: RE: image storage confusion ?? -- UUDECODE
Thanx Chao. Still under confusion , whether to store 20,000,000 images of 5k each either in -oracle 8.1.6 on win2k ( BLOB or BFILE ) OR - just on plain linux file system - OR - any file server like NetApp on linux --- which will be better ? how do people usually handle data of such volume ? plz let me know how it is done normally. TIA. Jp. On Tue, 10 Dec 2002 chao_ping wrote : oraora oraora, For blob, it is ok, since it is all in the database, while for bfile, you actually store them in filesystem, so you have to backup those files indivudually. Regards zhu chao [EMAIL PROTECTED] www.happyit.net www.cnoug.org(Chinese Oracle User Group) === 2002-12-09 17:23:00 ,you wrote£º=== Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already
RE: image storage confusion ?? -- UUDECODE
As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line
RE: image storage confusion ?? -- UUDECODE
Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = Connor McDonald http://www.oracledba.co.uk http://www.oaktable.net GIVE a man a fish and he will eat for a day. But TEACH him how to fish, and...he will sit in a boat and drink beer all day __ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com -- Please see the official ORACLE-L FAQ:
RE: image storage confusion ?? -- UUDECODE
Ian, There are UUENCODE and UUDECODE functions in 9.0.1. http://download-west.oracle.com/docs/cd/A91202_01/901_doc/appdev.901/a89 852/u_encod5.htm#1004065 It's also a one-liner in Perl, which would presumably be a lot faster than doing it in PL/SQL if you're not constrained to doing the operation inside the database. http://search.cpan.org/author/ANDK/Convert-UU-0.40/lib/Convert/UU.pm Cary Millsap Hotsos Enterprises, Ltd. http://www.hotsos.com Upcoming events: - Hotsos Clinic, Dec 9-11 Honolulu - Hotsos Clinic 101, Jan 7-9 Knoxville - Steve Adams's Miracle Master Class, Jan 13-15 Copenhagen - 2003 Hotsos Symposium, Feb 9-12 Dallas -Original Message- Ian A. Sent: Tuesday, December 03, 2002 5:02 PM To: Multiple recipients of list ORACLE-L Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an
RE: image storage confusion ?? -- UUDECODE
Thank you very much for the pointers. Ian -Original Message- Sent: Tuesday, December 03, 2002 7:39 PM To: Multiple recipients of list ORACLE-L Ian, There are UUENCODE and UUDECODE functions in 9.0.1. http://download-west.oracle.com/docs/cd/A91202_01/901_doc/appdev.901/a89 852/u_encod5.htm#1004065 It's also a one-liner in Perl, which would presumably be a lot faster than doing it in PL/SQL if you're not constrained to doing the operation inside the database. http://search.cpan.org/author/ANDK/Convert-UU-0.40/lib/Convert/UU.pm Cary Millsap Hotsos Enterprises, Ltd. http://www.hotsos.com Upcoming events: - Hotsos Clinic, Dec 9-11 Honolulu - Hotsos Clinic 101, Jan 7-9 Knoxville - Steve Adams's Miracle Master Class, Jan 13-15 Copenhagen - 2003 Hotsos Symposium, Feb 9-12 Dallas -Original Message- Ian A. Sent: Tuesday, December 03, 2002 5:02 PM To: Multiple recipients of list ORACLE-L Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and
