thanks for the replies. understood. i was informed that our underlying (EMMC??) FS will do this w/ FS pages, i.e. read the entire 256k FS page, modify some small portion of it, and write it out again. i'm higher in the stack so i don't understand the underlying FS mechanisms; i've simply asked the lower-level devs for advice about maximizing FS IO.
on this particular FS i'm under the impression that writing a 1k SQLite page requires reading a 256k FS page, modifying the 1k portion and then writing the whole 256k FS page out. i'll verify w/ the lower-level devs next week. yes, this sounds _very_ inefficient, but perhaps this is why our commit times are so poor (i.e. writing each DB page requires reading and writing a much larger FS page). yes - i'm aware that each table and index are in a separate DB page; i saw evidence of this when bumping the page size from 1k to 32k. this DB has only one table w/ a single pkey index. the DB will be relatively large (10MB) compared to the page size and will continue to grow over time, so i'm not concerned. basically i want to know if SQLite will have internal problems running w/ a page_size greater than the recommended (required?) max of 32k. thanks tom On Apr 21, 2010, at 1:39 PM, D. Richard Hipp wrote: > > On Apr 21, 2010, at 4:37 PM, Pavel Ivanov wrote: > >> I don't know anything about internal support of pages bigger than 32k. >> But I want to warn you: each table and each index in SQLite occupy at >> least 1 database page. So let's say you have 4 tables with 1 >> additional index each (besides 'integer primary key' one). With 256k >> pages this schema will result in a database of more than 2 Mb without >> any data stored. Is your embedded FS okay with this storage amount? > > Furthermore, SQLite changes whole pages at a time. So in a database > with 256kB pages, if you change a single byte, you still have to write > 256kB both to the rollback journal and to the database file. > >> >> >> Pavel >> >> On Tue, Apr 20, 2010 at 4:51 PM, Tom Broadbent >> <tom.broadb...@plasticlogic.com> wrote: >>> i've read in the docs that SQLITE_MAX_PAGE_SIZE can't be > 32k >>> (below). is this limitation still valid? >>> >>> we have an embedded FS that is _very_ slow and performs best w/ a >>> write page size of 256k. will bad things happen if i configure >>> SQLite w/ 256k pages? >>> >>> thanks >>> tom >>> >>> Maximum Database Page Size >>> >>> An SQLite database file is organized as pages. The size of each >>> page is a power of 2 between 512 and SQLITE_MAX_PAGE_SIZE. The >>> default value for SQLITE_MAX_PAGE_SIZE is 32768. The current >>> implementation will not support a larger value. >>> >>> It used to be the case that SQLite would allocate some stack >>> structures whose size was proportional to the maximum page size. >>> For this reason, SQLite would sometimes be compiled with a smaller >>> maximum page size on embedded devices with limited stack memory. >>> But more recent versions of SQLite put these large structures on >>> the heap, not on the stack, so reducing the maximum page size is no >>> longer necessary on embedded devices. There is no longer any real >>> reason to lower the maximum page size. >>> >>> ______________________________________________________________________ >>> This email has been scanned by the MessageLabs Email Security System. >>> For more information please visit http://www.messagelabs.com/email >>> ______________________________________________________________________ >>> _______________________________________________ >>> sqlite-users mailing list >>> sqlite-users@sqlite.org >>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users >>> >> _______________________________________________ >> sqlite-users mailing list >> sqlite-users@sqlite.org >> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > D. Richard Hipp > d...@hwaci.com > > > > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________ _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
