[sqlite] function to intialize the structure member
Hi All, can anybody tell me which function in sqlite3 can help me in initializing the structure member by reading the values from database. currently i m using sqlite3_exec() function to read the values from database and calling callback function to display the values. can i initialize structure members by assigning these values read from databaseentry using sqlite3_exec(). waiting for reply regards Nishit
Re: [sqlite] Multi-User confusion
--- Jeff Godfrey <[EMAIL PROTECTED]> wrote: > Thanks for the pointer. I should have mentioned, my application is > running under Windows (Win2000 and WinXP). A quick look at the > mentioned code makes me believe that it targets Unix-only systems > (though there are a few brief mentions of Windows, such as "The > algorithms are complicated slightly to be compatible with Windows..."). > > Do you know if the mentioned dotLockLockingStyle is compatible with a > Windows environment? Without some code modification, I doubt it. I don't see any mention of "dot" in os_win.c. However, in SQLite 3.5 you can define your own OS Interface File Virtual Methods Object and create your own file lock/unlock routines based on the dot locks in os_unix.c: http://sqlite.org/capi3ref.html#sqlite3_io_methods Looking for a deal? Find great prices on flights and hotels with Yahoo! FareChase. http://farechase.yahoo.com/ - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Multi-User confusion
Joe Wilson wrote: --- Jeff Godfrey <[EMAIL PROTECTED]> wrote: Can you (or anyone else) point me to some web-based information? http://www.sqlite.org/cvstrac/fileview?f=sqlite/src/os_unix.c=1.165 Joe, Thanks for the pointer. I should have mentioned, my application is running under Windows (Win2000 and WinXP). A quick look at the mentioned code makes me believe that it targets Unix-only systems (though there are a few brief mentions of Windows, such as "The algorithms are complicated slightly to be compatible with Windows..."). Do you know if the mentioned dotLockLockingStyle is compatible with a Windows environment? Thanks again. Jeff - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Multi-User confusion
--- Jeff Godfrey <[EMAIL PROTECTED]> wrote: > Joe Wilson wrote: > > If your database storage device cannot guarantee an exclusive file > > lock, then any database write can potentially result in corruption. > > > > If you control all SQLite clients' code, you could recompile sqlite > > to use the file-based dotlockLockingStyle convention via > > > > -DSQLITE_ENABLE_LOCKING_STYLE=1 > > > > But even if a single client does not use that locking convention, > > you still risk corruption. > > > Joe, > > Thanks for the input. I absolutely control all of the SQLite clients - > they are just unique instances of my (Tcl-based) application. I am > unaware of the mentioned "dotlockLockingStyle" convention. Can you (or > anyone else) point me to some web-based information? Use the Source, Luke: http://www.sqlite.org/cvstrac/fileview?f=sqlite/src/os_unix.c=1.165 Shape Yahoo! in your own image. Join our Network Research Panel today! http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Multi-User confusion
[EMAIL PROTECTED] wrote: You may have a look at the dhRCPServer at: http://www.thecommon.net/2.html I am not using it, but it sounds it may do the job. RBS Bart, Thanks for the pointer. It does sound quite interesting, though I don't know if it can (easily?) be used from within a Tcl-based application. I'll have to do some research... Jeff - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Multi-User confusion
Joe Wilson wrote: If your database storage device cannot guarantee an exclusive file lock, then any database write can potentially result in corruption. If you control all SQLite clients' code, you could recompile sqlite to use the file-based dotlockLockingStyle convention via -DSQLITE_ENABLE_LOCKING_STYLE=1 But even if a single client does not use that locking convention, you still risk corruption. Joe, Thanks for the input. I absolutely control all of the SQLite clients - they are just unique instances of my (Tcl-based) application. I am unaware of the mentioned "dotlockLockingStyle" convention. Can you (or anyone else) point me to some web-based information? Thanks again. Jeff - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQL approach to Import only new items, delete other items
You could invert your solution. Create a temporary table which contains all of the existing keys, and every time you insert a new item, delete that item's key from the temporary table. At the end, do something like 'DELETE FROM main_table WHERE key IN (SELECT key FROM tmp_table)'. -scott On 9/6/07, Andre du Plessis <[EMAIL PROTECTED]> wrote: > Im importing data > > The data has a unique value, call it MD5 for now that could be a unique > value for the data. > > > > Each record that gets imported is converted to MD5, a lookup is done on > the table for that MD5, > > if found it must leave it alone, if not found it must insert a new > record... > > > > All the items in the table that was not imported must be deleted... > > > > The only feasible approach I have is to add a column to the table, like > UPDATE_FLAG for example, > > During the import update_flag gets set to 0, > > Once a record is found update_flag gets set to 1, > > > > At the end of the import all records with update_flag = 0 gets > deleted... > > > > However I did not want to add a column to the table, REASON being, I'm > also version controlling the DB, and when an import occurs and nothing > > Has been added or removed, I don't want modifications to the DB, as the > import can run many times over. > > > > I was considering the following: > > Create a temp table call it, > > Temp_ids for example > > Then insert into the temp table for each record that was found... > > > > At the end do something like > > Delete from imports where import_id not in (select id from temp_ids) > > > > But that might be horribly slow and memory expensive remembering that > the import table may have millions of records.. > > > > What could the people here suggest to me, > > > > Thanks. > > - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Multi-User confusion
If your database storage device cannot guarantee an exclusive file lock, then any database write can potentially result in corruption. If you control all SQLite clients' code, you could recompile sqlite to use the file-based dotlockLockingStyle convention via -DSQLITE_ENABLE_LOCKING_STYLE=1 But even if a single client does not use that locking convention, you still risk corruption. --- Jeff Godfrey <[EMAIL PROTECTED]> wrote: > I currently have a single-user SQLite-based application that, due to > customer need, is being pushed toward multi-user access. I've done some > research on the multi-user capabilities of SQLite. It seems the general > consensus is that when the database file is stored on a network drive > (as is my case), the integrity of the stored data becomes questionable > (apparently due to bugs in the various NFS file locking protocols). > > Fortunately, my application is designed such that (generally speaking) > each User of the system will be working within their own SQLite > database. However, there are a few select places in the code where a > User could trigger an action that would cause the storage of data to a > common, upper-level SQLite database. > > I think I can change portions of the application to ensure that these > common writes never happen concurrently, but I'd like to understand the > underlying situations and dangers that can occur in this environment. > So, what are the cases that could cause database corruption? > > 1. Multiple Users writing to the same table of the same open database at > the same time? > 2. Multiple Users writing to two different tables of the same open > database at the same time? > 3. Multiple Users writing to the same table of the same open database at > different times? > 4. Other cases I haven't thought about? > > I realize there also some dangers with regard to potentially writing > "stale" data to the database thus losing someone else's updates. I > still have some thinking to do in that regard, but for now I'd like to > understand the situations that could compromise the integrity of the > underlying database file itself. Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. http://smallbusiness.yahoo.com/webhosting - To unsubscribe, send email to [EMAIL PROTECTED] -
RE: [sqlite] SQL approach to Import only new items, delete other items
> Because my concern is this, I don't know how SQLite will do > > Delete from table where col not in (select from large temp dataset) > > How the delete will actually be walked, if it will create a serverside > cursor and walk the values in the in statement then it will > be fine and > fast, > If however it loads all the values into memory and then walk > the dataset > it would require a large amount of memory, I believe that it will write the results of the subquery to a temporary table, so I doubt that memory usage will be a problem. I'm only about 98% sure though, so no guarantees. :) EXPLAIN will tell you for sure though. > Your suggestion to drop the table and recreate from temp > although a good > idea will probably modify the database which means that if I version > control it, it will create a large amount of changes each > time an import > is run, even though nothing might have been changed. I think you have issues here either way though - one way or another you need to know that nothing changed. You may be able to determine that for free, depending on your approach, or you may need to go out of your way to determine that, it depends on your approach. But I wouldn't assume that you can know this for free and that any solution that requires work to know that is inherently bad. -Tom - To unsubscribe, send email to [EMAIL PROTECTED] -
RE: [sqlite] SQL approach to Import only new items, delete other items
Thank you for the suggestion, Im about to set up a test as soon as I have the time where I want to create a test table and populate it with a high number of rows, then perform an update to a temp table, and then delete with an in statement Because my concern is this, I don't know how SQLite will do Delete from table where col not in (select from large temp dataset) How the delete will actually be walked, if it will create a serverside cursor and walk the values in the in statement then it will be fine and fast, If however it loads all the values into memory and then walk the dataset it would require a large amount of memory, Your suggestion to drop the table and recreate from temp although a good idea will probably modify the database which means that if I version control it, it will create a large amount of changes each time an import is run, even though nothing might have been changed. I guess I could write another framework for versioning the database, .dump would probably work better but it requires that user intervention. But thanks for the suggestion I might be able to implement something that works. -Original Message- From: Tom Briggs [mailto:[EMAIL PROTECTED] Sent: 06 September 2007 03:33 PM To: sqlite-users@sqlite.org Subject: RE: [sqlite] SQL approach to Import only new items, delete other items Your suggested temp table approach is how I would solve this; if everything is properly indexed it won't be too bad. Even if it is bad, it'll be better than updating columns within the table and then deleting rows based on that. Another potential alternative is to: 1. Load all new rows into a temp table 2. Select the old matching rows into a second temp table 3. Insert all the remaining new rows to that second temp table 4. Drop the original table and rename the second temp table That's likely to be slower on small data sets and faster on larger datasets, I think. Depends on how much data is already in the database vs. the amount of data being loaded. -Tom > -Original Message- > From: Andre du Plessis [mailto:[EMAIL PROTECTED] > Sent: Thursday, September 06, 2007 5:41 AM > To: sqlite-users@sqlite.org > Subject: [sqlite] SQL approach to Import only new items, > delete other items > > Im importing data > > The data has a unique value, call it MD5 for now that could > be a unique > value for the data. > > > > Each record that gets imported is converted to MD5, a lookup > is done on > the table for that MD5, > > if found it must leave it alone, if not found it must insert a new > record... > > > > All the items in the table that was not imported must be deleted... > > > > The only feasible approach I have is to add a column to the > table, like > UPDATE_FLAG for example, > > During the import update_flag gets set to 0, > > Once a record is found update_flag gets set to 1, > > > > At the end of the import all records with update_flag = 0 gets > deleted... > > > > However I did not want to add a column to the table, REASON being, I'm > also version controlling the DB, and when an import occurs and nothing > > Has been added or removed, I don't want modifications to the > DB, as the > import can run many times over. > > > > I was considering the following: > > Create a temp table call it, > > Temp_ids for example > > Then insert into the temp table for each record that was found... > > > > At the end do something like > > Delete from imports where import_id not in (select id from temp_ids) > > > > But that might be horribly slow and memory expensive remembering that > the import table may have millions of records.. > > > > What could the people here suggest to me, > > > > Thanks. > > - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
RE: [sqlite] SQL approach to Import only new items, delete other items
Your suggested temp table approach is how I would solve this; if everything is properly indexed it won't be too bad. Even if it is bad, it'll be better than updating columns within the table and then deleting rows based on that. Another potential alternative is to: 1. Load all new rows into a temp table 2. Select the old matching rows into a second temp table 3. Insert all the remaining new rows to that second temp table 4. Drop the original table and rename the second temp table That's likely to be slower on small data sets and faster on larger datasets, I think. Depends on how much data is already in the database vs. the amount of data being loaded. -Tom > -Original Message- > From: Andre du Plessis [mailto:[EMAIL PROTECTED] > Sent: Thursday, September 06, 2007 5:41 AM > To: sqlite-users@sqlite.org > Subject: [sqlite] SQL approach to Import only new items, > delete other items > > Im importing data > > The data has a unique value, call it MD5 for now that could > be a unique > value for the data. > > > > Each record that gets imported is converted to MD5, a lookup > is done on > the table for that MD5, > > if found it must leave it alone, if not found it must insert a new > record... > > > > All the items in the table that was not imported must be deleted... > > > > The only feasible approach I have is to add a column to the > table, like > UPDATE_FLAG for example, > > During the import update_flag gets set to 0, > > Once a record is found update_flag gets set to 1, > > > > At the end of the import all records with update_flag = 0 gets > deleted... > > > > However I did not want to add a column to the table, REASON being, I'm > also version controlling the DB, and when an import occurs and nothing > > Has been added or removed, I don't want modifications to the > DB, as the > import can run many times over. > > > > I was considering the following: > > Create a temp table call it, > > Temp_ids for example > > Then insert into the temp table for each record that was found... > > > > At the end do something like > > Delete from imports where import_id not in (select id from temp_ids) > > > > But that might be horribly slow and memory expensive remembering that > the import table may have millions of records.. > > > > What could the people here suggest to me, > > > > Thanks. > > - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] Patch for lemon issue #2550
Hi all,
Please find attached a patch for ticket 2550: "Lemon parser may accept
input before EOF is seen."
I'm sending it to the list as I do not seem able to pass the abbreviated
Turing test that guards attaching stuff to the ticket. :(
It appears to work over here where several lemon generated parsers are
used in a context with heavy regression tests.
Patch hereby released to the public.
Enjoy. Vincent.
Index: lemon.c
===
--- lemon.c (revision 3121)
+++ lemon.c (working copy)
@@ -42,6 +42,7 @@
/** From the file "build.h" /
void FindRulePrecedences();
void FindFirstSets();
+void FindStartSymbol();
void FindStates();
void FindLinks();
void FindFollowSets();
@@ -244,6 +245,7 @@
char *tokentype; /* Type of terminal symbols in the parser stack */
char *vartype; /* The default type of non-terminal symbols */
char *start; /* Name of the start symbol for the grammar */
+ struct symbol *startsym; /* Start symbol for the grammar */
char *stacksize; /* Size of the parser stack */
char *include; /* Code to put at the start of the C file */
int includeln; /* Line number for start of include code */
@@ -660,19 +662,12 @@
return;
}
-/* Compute all LR(0) states for the grammar. Links
-** are added to between some states so that the LR(1) follow sets
-** can be computed later.
-*/
-PRIVATE struct state *getstate(/* struct lemon * */); /* forward reference */
-void FindStates(lemp)
+void FindStartSymbol(lemp)
struct lemon *lemp;
{
struct symbol *sp;
struct rule *rp;
- Configlist_init();
-
/* Find the start symbol */
if( lemp->start ){
sp = Symbol_find(lemp->start);
@@ -704,10 +699,23 @@
}
}
+ lemp->startsym = sp;
+}
+
+/* Compute all LR(0) states for the grammar. Links
+** are added to between some states so that the LR(1) follow sets
+** can be computed later.
+*/
+PRIVATE struct state *getstate(/* struct lemon * */); /* forward reference */
+void FindStates(lemp)
+struct lemon *lemp;
+{
+ struct rule *rp;
+
/* The basis configuration set for the first state
** is all rules which have the start symbol as their
** left-hand side */
- for(rp=sp->rule; rp; rp=rp->nextlhs){
+ for(rp=lemp->startsym->rule; rp; rp=rp->nextlhs){
struct config *newcfp;
newcfp = Configlist_addbasis(rp,0);
SetAdd(newcfp->fws,0);
@@ -921,7 +929,6 @@
int i,j;
struct config *cfp;
struct state *stp;
- struct symbol *sp;
struct rule *rp;
/* Add all of the reduce actions
@@ -944,16 +951,10 @@
}
/* Add the accepting token */
- if( lemp->start ){
-sp = Symbol_find(lemp->start);
-if( sp==0 ) sp = lemp->rule->lhs;
- }else{
-sp = lemp->rule->lhs;
- }
/* Add to the first state (which is always the starting state of the
** finite state machine) an action to ACCEPT if the lookahead is the
** start nonterminal. */
- Action_add(>sorted[0]->ap,ACCEPT,sp,0);
+ Action_add(>sorted[0]->ap,ACCEPT,lemp->startsym,0);
/* Resolve conflicts */
for(i=0; instate; i++){
@@ -1461,6 +1462,11 @@
** nonterminal */
FindFirstSets();
+Configlist_init();
+
+/* Determine the start symbol */
+FindStartSymbol();
+
/* Compute all LR(0) states. Also record follow-set propagation
** links so that the follow-set can be computed later */
lem.nstate = 0;
@@ -3941,6 +3947,7 @@
}
if( ap->type!=REDUCE ) continue;
rp = ap->x.rp;
+ if (rp->lhs == lemp->startsym) continue; /* do not default a start rule to ensure reduction only on EOF */
if( rp==rbest ) continue;
n = 1;
for(ap2=ap->next; ap2; ap2=ap2->next){
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
[sqlite] SQL approach to Import only new items, delete other items
Im importing data The data has a unique value, call it MD5 for now that could be a unique value for the data. Each record that gets imported is converted to MD5, a lookup is done on the table for that MD5, if found it must leave it alone, if not found it must insert a new record... All the items in the table that was not imported must be deleted... The only feasible approach I have is to add a column to the table, like UPDATE_FLAG for example, During the import update_flag gets set to 0, Once a record is found update_flag gets set to 1, At the end of the import all records with update_flag = 0 gets deleted... However I did not want to add a column to the table, REASON being, I'm also version controlling the DB, and when an import occurs and nothing Has been added or removed, I don't want modifications to the DB, as the import can run many times over. I was considering the following: Create a temp table call it, Temp_ids for example Then insert into the temp table for each record that was found... At the end do something like Delete from imports where import_id not in (select id from temp_ids) But that might be horribly slow and memory expensive remembering that the import table may have millions of records.. What could the people here suggest to me, Thanks.
