Re: [sqlite] Network-based DB performance for Mozilla

2005-09-15 Thread Christian Smith
Sorry to bring this up yet again... On Tue, 13 Sep 2005, D. Richard Hipp wrote: >On Tue, 2005-09-13 at 14:09 -0700, Brett Wilson wrote: >> I'm still concerned about the warnings on the web page about some >> networked file systems not supporting locking. There will be multiple >> DB connections

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-14 Thread Edward Macnaghten
Please excuse me doing another reply to this but Sorry about this daft question, but you have indexed everythink OK and designed the database to a reasonable "BCNF"(Boyce Codd Normal Form) model havn't you? Eddy Brett Wilson wrote: Hi everybody, I'm working on replacing a lot of

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-14 Thread Edward Macnaghten
To add my 2p worth to all this I am not fully aware of the Firefox internals, buts as far as my understanding goes only one process can use any profile at any time. If two "instances" of firefox are fired up for the same user (+ profile) then what happens as far as I understand is

RE: [sqlite] Network-based DB performance for Mozilla

2005-09-14 Thread Brandon, Nicholas
>If you can't tolerate the delays accessing the database over the >network, can you make a copy of the database in a temp directory on the >local machine on startup. If you copy the file after you lock the >profile it should be safe to copy down to local storage. Then use the >local database

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Will Leshner
On Sep 13, 2005, at 3:19 PM, Brett Wilson wrote: The patch says "improve and support locking on the OSX platform (as well as others)". I see at least some enums in there for MSDOS NFS, etc. Well, looking closer at the code, I'm beginning to think it might very well be a generic solution

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Brett Wilson
The patch says "improve and support locking on the OSX platform (as well as others)". I see at least some enums in there for MSDOS NFS, etc. Can anybody clarify whether this works on other platforms as well? Basically, the question I have about this patch is: if I access the DB from more than

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Will Leshner
On Sep 13, 2005, at 2:19 PM, D. Richard Hipp wrote: Apple has contributed patches to SQLite that claim to fix this problem. Those patches may one day find their way into the default release. In the meantime, you can find the patches at: I'm not positive, but I think the Apple patches are

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread D. Richard Hipp
On Tue, 2005-09-13 at 14:09 -0700, Brett Wilson wrote: > I'm still concerned about the warnings on the web page about some > networked file systems not supporting locking. There will be multiple > DB connections from the same process. They might even be > multithreaded. Might we have a problem in

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Brett Wilson
I'm still concerned about the warnings on the web page about some networked file systems not supporting locking. There will be multiple DB connections from the same process. They might even be multithreaded. Might we have a problem in this case? Brett On 9/13/05, Roger Binns <[EMAIL PROTECTED]>

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Joel Lucsy
How about reading it all into :memory: and attaching the network database with a trigger so that when an update is made to the :memory: database the change is reflected to the network copy. -- Joel Lucsy "The dinosaurs became extinct because they didn't have a space program." -- Larry Niven

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Jay Sprenkle
>I don't think it will matter what database you use if you're forced to have > >it hosted over a network. > >As far as I know they all rely on the underlying locking mechanism in the > >OS, which is inherently slow over a network. > > > >Can you avoid multiple locking latency hits by just getting

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Dennis Cote
Brett Wilson wrote: Hi everybody, I'm working on replacing a lot of Firefox's storage systems to use sqlite. It has been going well so far except for one issue. The database file is stored in the user's Mozilla profile directory. In companies and Universities using Linux, this directory is

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Dennis Jenkins
Jay Sprenkle wrote: I'm glad to see someone is working on this since it was a very noticable problem for me. I don't think it will matter what database you use if you're forced to have it hosted over a network. As far as I know they all rely on the underlying locking mechanism in the OS,

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Brett Wilson
This was brought up but seems like a bad idea for several reasons. First, the database could get pretty big. This would really kill start up and shut down times, which is very important to FF users. Second, what would happen if we crash? One possibility is if we think we'll be doing a lot of UI,

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Jay Sprenkle
I'm glad to see someone is working on this since it was a very noticable problem for me. I don't think it will matter what database you use if you're forced to have it hosted over a network. As far as I know they all rely on the underlying locking mechanism in the OS, which is inherently slow

Re: [sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Eric Scouten
To what extent is the database shared (either intentionally or unintentionally)? Or, put another way, do you have an option to cache data locally? Since you are restricting access to the profile to a single process at a time, your best bet is probably to make a local copy of the DB during

[sqlite] Network-based DB performance for Mozilla

2005-09-13 Thread Brett Wilson
Hi everybody, I'm working on replacing a lot of Firefox's storage systems to use sqlite. It has been going well so far except for one issue. The database file is stored in the user's Mozilla profile directory. In companies and Universities using Linux, this directory is often hosted over the