Re: [sqlite] memory leak in transactions
Thanks for your reply.
I am using version 1.0.79.0 of System.Data.sqlite
Here is how i am using it, i only use executescalar and
executenonquery, let me know if you want to see how i pass the
parameters etc.
RunBigtransaction()
{
using (IDbTransaction tran = Connection.BeginTransaction())
{
foreach (Message msg in messages)
{
CreateMessage(msg);
}
tran.Commit();
}
}
void CreateMessage()
{
string tableName = gettablename();
// here i check if a table exists using a select query
string query = select "name" from "sqlite_master" where
"type=table", name=tablename";
string tablename = ExecuteQuery(query)
if (tablename == null)
{
// here i do a query on another table in schema to read
a script to
create a new table, which uses the ExecuteGetTableQuery.
string script = getscript();
Exceute(script);
}
// here i execute an insert and update
// insert uses the same pattern like ExecuteGetTableQuery but
with
query changed to insert into with 15 parameters.
RawInsertMessage(tableName, msg);
// insert or update where update does just Execute
function.
SetMessageFolderId(msg.StoreId, msg.Id, msg.ParentId);
}
public void Execute(string script)
{
using (IDbCommand cmd = Connection.CreateCommand())
{
cmd.CommandText = script;
cmd.ExecuteNonQuery();
}
}
public string ExecuteGetTableQuery(string b)
{
using (IDbCommand cmd = Connection.CreateCommand())
{
cmd.CommandText = b.ToString();
object r = cmd.ExecuteScalar();
if (r == null || r is DBNull) return default(T);
if (!(r is T)) throw new
InvalidCastException("cannot cast " +
r.GetType() + " to " + typeof (T));
return (T) r;
}
}
Regards,
Raj
On Thu, Nov 15, 2012 at 8:52 PM, Joe Mistachkin wrote:
>
> rui wrote:
>>
>> I am seeing explosive memory growth when i am using transactions using
>> System.Data.SQLite.
>>
>
> The best way to address this issue is to utilize "using" blocks for any
> SQLiteCommand, SQLiteDataReader, and SQLiteTransaction objects used. That
> way, you won't have to wait until they are garbage collected later on by the
> CLR.
>
>>
>> All the object are properly disposed from trasnsaction to command etc.
>>
>
> Could you provide some example C# code that demonstrates the behavior you are
> seeing?
>
>>
>> The SQLiteConnection is kept open for the life time of the session,
>> which could span hours.
>>
>
> This means the memory associated with the connection will be kept around;
> however, this should be OK and should not result in the memory usage numbers
> you are seeing.
>
> --
> Joe Mistachkin
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] memory leak in transactions
rui wrote: > > I am seeing explosive memory growth when i am using transactions using > System.Data.SQLite. > The best way to address this issue is to utilize "using" blocks for any SQLiteCommand, SQLiteDataReader, and SQLiteTransaction objects used. That way, you won't have to wait until they are garbage collected later on by the CLR. > > All the object are properly disposed from trasnsaction to command etc. > Could you provide some example C# code that demonstrates the behavior you are seeing? > > The SQLiteConnection is kept open for the life time of the session, > which could span hours. > This means the memory associated with the connection will be kept around; however, this should be OK and should not result in the memory usage numbers you are seeing. -- Joe Mistachkin ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] memory leak in transactions
Do you have your inserts wrapped in a single transaction? It used to be that I wasn't worrying about transactions in my projects, but I noticed things were very slow. I realized that it was creating and committing one transaction for each insert I was doing. When I wrapped all inserts into a single transaction, speed jumped dramatically. But, of course, this doesn't address your memory issue. Unless the increased memory is somewhere in the transaction management logic. RobR -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of rui Sent: Thursday, November 15, 2012 12:21 PM To: sqlite-users@sqlite.org Subject: [sqlite] memory leak in transactions Hi, I am seeing explosive memory growth when i am using transactions using System.Data.SQLite. I see this when the transactions which were before taking fraction of second, taking 15 sec for 50 row insert into a table. I have tried to use sqlite3_db_release_memory but that only frees upto 2mb after every transaction. After some time say half an hour - i do see memory usage drop but that's not acceptable. All the object are properly disposed from trasnsaction to command etc. The SQLiteConnection is kept open for the life time of the session, which could span hours. I would really appreciate if somebody can help me in getting the reason for such excessive memory usage, i have seen working set(memory)go up from 70 mb to 400 mb in 1 minute where three transactions only doing 50 insert in a table. Regards, Raj ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] memory leak in transactions
Hi, I am seeing explosive memory growth when i am using transactions using System.Data.SQLite. I see this when the transactions which were before taking fraction of second, taking 15 sec for 50 row insert into a table. I have tried to use sqlite3_db_release_memory but that only frees upto 2mb after every transaction. After some time say half an hour - i do see memory usage drop but that's not acceptable. All the object are properly disposed from trasnsaction to command etc. The SQLiteConnection is kept open for the life time of the session, which could span hours. I would really appreciate if somebody can help me in getting the reason for such excessive memory usage, i have seen working set(memory)go up from 70 mb to 400 mb in 1 minute where three transactions only doing 50 insert in a table. Regards, Raj ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
