Hello,
As mentioned in the subject, our goal is to improve performance regarding
to batch sql updates.
The update sql as follow,
> UPDATE ITEM SET FIELD4 =? WHERE DS=?
We run 100,000 updates in a single transaction. The zipvfs version takes
about 20 min while uncompressed version takes about 7 min.
I've also attached results of "EXPLAIN UPDATE ITEM SET FIELD4 = 10 WHERE DS
= 15;".
Any suggestions is very helpful.
Some background:
We have an app running on Android 4.2 using zipvfs.
The page size is 4096
The cache size is - 2000
Table schema for ITEM
> CREATE TABLE ITEM (
> FIELD0 NUMERIC,
> FIELD1 NUMERIC,
> DS TEXT,
> FIELD2 TEXT,
> FIELD3 TEXT,
> FIELD4 NUMERIC,
> FIELD5 NUMERIC,
> FIELD6 NUMERIC,
> FIELD7 NUMERIC,
> FIELD8 NUMERIC,
> FIELD9 NUMERIC,
> FIELD10 NUMERIC,
> FIELD11 TEXT);
The third column: "DS" is what we query by almost all the time. We also
created index:
> CREATE INDEX DS_INDEX ON ITEM(DS);
There are 30 million records. Zipvfs size of 471mb vs uncompressed db (the
one shipped with Android 4.2) size of 1.39gb.
Zipvfs db using zlib and aes128, which are default.
Thanks
--
Yue Wu | Android Developer
sqlite> EXPLAIN UPDATE ITEM SET FIELD4 = 10 WHERE DS = 15;
addr opcode p1p2p3p4 p5 comment
- - -- -
0 Init 0 28000 Start at 28
1 Null 0 1 200 r[1..2]=NULL
2 OpenWrite 0 3 0 13 08 root=3 iDb=0; ITEM
3 OpenWrite 1 4 0 k(2,,) 02 root=4 iDb=0; ds_index
4 Integer1516000 r[16]=15
5 Affinity 161 0 B 00 affinity(r[16])
6 SeekGE 1 27161 00 key=r[16]
7 IdxGT 1 27161 00 key=r[16]
8 DeferredSeek 1 0 000 Move 0 to 1.rowid
if needed
9 Rowid 0 2 000 r[2]=rowid
10 IsNull 2 27000 if r[2]==NULL goto
27
11 Column 0 0 300 r[3]=ITEM.FIELD0
12 Column 0 1 400 r[4]=ITEM.FIELD1
13 Column 0 2 500 r[5]=ITEM.DS
14 Column 0 3 600 r[6]=ITEM.FIELD2
15 Column 0 4 700 r[7]=ITEM.FIELD3
16 Integer108 000 r[8]=10
17 Column 0 6 900 r[9]=ITEM.FIELD5
18 Column 0 7 10 00 r[10]=ITEM.FIELD6
19 Column 0 8 11 00 r[11]=ITEM.FIELD7
20 Column 0 9 12 00 r[12]=ITEM.FIELD8
21 Column 0 1013 00 r[13]=ITEM.FIELD9
22 Column 0 1114 00 r[14]=ITEM.FIELD10
23 Column 0 1215 00 r[15]=ITEM.FIELD11
24 MakeRecord 3 1317CCBBBCCCB 00
r[17]=mkrec(r[3..15])
25 Insert 0 172 ITEM 07 intkey=r[2]
data=r[17]
26Next 1 7 100
27Halt 0 0 000
28Transaction0 1 4 0 01 usesStmtJournal=0
29Goto 0 1 000___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
