Re: [sqlite] SQLite with NAS storage

>>> Each NAS_000 ~ N storages have approximately 300,000 >>> files, the average size of file is few MB (not over GB). >>> The broker servers (with SQLite library) are on the >>> NAS > > It's not clear how many broker servers there are. > One per NAS? 80 ~ 100 servers. and generally one per nas,

Re: [sqlite] SQLite with NAS storage

On Wed, 7 Jan 2009 11:14:11 +0900, "Edward J. Yoon" wrote in General Discussion of SQLite Database : > Thanks for your reply. > >> That's a lot of files. Or did you mean rows? >> Are you sure? There can be many other reasons. > > There is a lot of

Re: [sqlite] SQLite with NAS storage

On Wed, 7 Jan 2009 10:17:06 -0800, "Jim Dodgen" wrote in General Discussion of SQLite Database : > I'm a little worried about how long it takes to open one > of 20,000,000 files in a directory on the NAS? I agree. It would require a very cleverly

Re: [sqlite] SQLite with NAS storage

Edward J. Yoon wrote: >> Is access to *one* of the 20 million different SQLite files getting >> progressively slower? How big is that specific SQLite file? Is that >> the one that is "huge"? I use SQLite over an NAS (at times), and never >> experience any noticeable slowdown. Is access to his NAS

Re: [sqlite] SQLite with NAS storage

I'm a little worried about how long it takes to open one of 20,000,000 files in a directory on the NAS? On Wed, Jan 7, 2009 at 6:36 AM, Edward J. Yoon wrote: > > Is access to *one* of the 20 million different SQLite files getting > > progressively slower? How big is that

Re: [sqlite] SQLite with NAS storage

> Is access to *one* of the 20 million different SQLite files getting > progressively slower? How big is that specific SQLite file? Is that > the one that is "huge"? I use SQLite over an NAS (at times), and never > experience any noticeable slowdown. Is access to his NAS itself slow, > perhaps not

Re: [sqlite] SQLite with NAS storage

I actually thought the original question was perfectly clear. I thought the proposed solution (included in the original post) was perfectly logical too. So what's all the fuss? On Wed, Jan 7, 2009 at 7:28 AM, P Kishor wrote: > On 1/6/09, Edward J. Yoon

Re: [sqlite] SQLite with NAS storage

On 1/6/09, Edward J. Yoon wrote: > Thanks, > > In more detail, SQLite used for user-based applications (20 million is > the size of app-users). and MySQL used for user location (file path on > NAS) addressing. Edward, At least I still don't understand why you have 20

Re: [sqlite] SQLite with NAS storage

Thanks, In more detail, SQLite used for user-based applications (20 million is the size of app-users). and MySQL used for user location (file path on NAS) addressing. On Wed, Jan 7, 2009 at 1:31 PM, P Kishor wrote: > On 1/6/09, Edward J. Yoon wrote:

Re: [sqlite] SQLite with NAS storage

On 1/6/09, Edward J. Yoon wrote: > > Do you have 20 million sqlite databases? > > > Yes. Since all these databases are just files, you should stuff them into a Postgres database, then write an application that extracts the specific row from the pg database with 20 mil rows

Re: [sqlite] SQLite with NAS storage

> Do you have 20 million sqlite databases? Yes. On Wed, Jan 7, 2009 at 12:36 PM, Jim Dodgen wrote: > I think the question was about the structure of your data > > a sqlite database is a file and can contain many tables. tables can contain > many rows. > > Do you have 20 million

Re: [sqlite] SQLite with NAS storage

I think the question was about the structure of your data a sqlite database is a file and can contain many tables. tables can contain many rows. Do you have 20 million sqlite databases? This information can help people formulate an answer. On Tue, Jan 6, 2009 at 6:14 PM, Edward J. Yoon

Re: [sqlite] SQLite with NAS storage

Thanks for your reply. > That's a lot of files. Or did you mean rows? > Are you sure? There can be many other reasons. There is a lot of files. So, I don't know exactly why at this time, But thought network latency can´t be denied. /Edward On Wed, Jan 7, 2009 at 4:07 AM, Kees Nuyt

Re: [sqlite] SQLite with NAS storage

Again, We have a lot of read/write operation. So, I guess the network latency is problem and think about lock management system. On Tue, Jan 6, 2009 at 11:23 AM, Edward J. Yoon wrote: > Hi, I'm newbie in here. > > I'm using SQLite, all data (very huge and 20 million

[sqlite] SQLite with NAS storage

Hi, I'm newbie in here. I'm using SQLite, all data (very huge and 20 million files) stored on NAS storage. Lately my system has been getting progressively slower. Network cost seems too large. To improve its performance, I'm think about local lock file instead of NAS as describe below. char

Re: [sqlite] SQLite for NAS storage - DVR like application

Trilok Soni wrote: Hi John, On Dec 18, 2007 11:53 PM, John Stanton <[EMAIL PROTECTED]> wrote: Sqlite depends upon POSIX file locks. It is no better or no worse than the POSIX lock implementation on your platform. Thanx for the reply. My platform is based on Linux running on ARM9, with

Re: [sqlite] SQLite for NAS storage - DVR like application

Hi Dario, On Dec 19, 2007 6:36 PM, Darío Mariani <[EMAIL PROTECTED]> wrote: > Current implementations of NFS on Linux have a decent lock mechanism > (as far as I know). You'll have to check your NAS device how does it > handle NFS locking. Thanx. I was hoping the same from NFS :) > But, if your

Re: [sqlite] SQLite for NAS storage - DVR like application

Current implementations of NFS on Linux have a decent lock mechanism (as far as I know). You'll have to check your NAS device how does it handle NFS locking. But, if your NFS device is a complete box and not just a disk array with an ethernet card, you will be better creating a small client-server

Re: [sqlite] SQLite for NAS storage - DVR like application

Hi John, On Dec 18, 2007 11:53 PM, John Stanton <[EMAIL PROTECTED]> wrote: > Sqlite depends upon POSIX file locks. It is no better or no worse than > the POSIX lock implementation on your platform. Thanx for the reply. My platform is based on Linux running on ARM9, with having 128MB of the

Re: [sqlite] SQLite for NAS storage - DVR like application

Sqlite depends upon POSIX file locks. It is no better or no worse than the POSIX lock implementation on your platform. We use Sqlite in a multi-user environment without dependency oin the POSIX locks by embedding it in a server using HTTP when it is on a remote machine. We get the small

[sqlite] SQLite for NAS storage - DVR like application

Hi, I am evaluating SQLite for the design of the surveillance camera/DVR product based on Linux, which stores its captured analog/IP camera streams to the NAS storage device(s). To facilitate faster search/scanning of those media files containing many days of videos spread over multiple files, we