[sqlite] Fwd: signal 6 during sqlite_step in WAL mode

[Mattan Shalev]

Hey guys,
I'm getting signal 6 during sqlite_step in WAL mode. Working on Ubuntu
12.04, sqlite3 3.7.9.
One process is the writing continuously,  while other process reads from
the DB in a multi threaded access. I made sure that sqlite is configured to
serialised mode.

Here is the backtrace:

#0 0x7f4f78f0d08d in nanosleep () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x7f4f78f0cf2c in sleep () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x7f4f7982d881 in signal_catch (signo=6) at signal_masker.c:35
#3 
#4 0x7f4f78e83425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#5 0x7f4f78e86b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#6 0x7f4f78ec139e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#7 0x7f4f78ecbb96 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#8 0x7f4f77661690 in sqlite3_free () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#9 0x7f4f77664b96 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#10 0x7f4f77664c7c in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#11 0x7f4f77664cb0 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#12 0x7f4f7768b477 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#13 0x7f4f7769a249 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#14 0x7f4f776a9689 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#15 0x7f4f776ae451 in sqlite3_step () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#16 0x7f4f79416557 in get_down_intervals_size (start_time=1403711040,
end_time=1577836800,
size=0x7f3c49c68958) at stats_ext.c:202
#17 0x00407f80 in ns__get_stats_performance (s=0x1e4cb00,
start_time=1403711040, end_time=1577836800,
time_unit=NS_MINUTES, resource_id=0, ret=0x7f3c49c68990) at
msg_handlers/capi_wrappers.c:636
#18 0x004461c1 in soap_serve_ns__get_stats_performance
(soap=0x1e4cb00) at soapServer.c:1294
#19 0x004415ae in soap_serve_request (soap=0x1e4cb00) at
soapServer.c:119
#20 0x00440f3e in soap_serve (soap=0x1e4cb00) at soapServer.c:39
#21 0x00403c90 in process_request (s=0x1e4cb00) at be_server.c:111
#22 0x7f4f79a3de9a in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#23 0x7f4f78f413fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#24 0x in ?? ()

Does anyone has an idea?


Thanks!

Mattan.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] think I need better error-handling guidance in the C API

[Keith Medcalf]

IF ResultCode == A_OK YipeeKiAiii else OhShitItDidntWork;

Seems pretty straightforward to me.  You handle the result codes you know what 
to do with, and everything that remains means your program should explode 
immediately and very very loudly dumping ALL relevant information (ie, error 
code, statement, inout data, output data, etc.)

In other words, there is no point checking for error conditions which you 
cannot handle.  Instead you just die.  Quickly, noisily, and immediately.  
There is no point checking for the error message 
SQLITE3_CPU_REGISTER_HIT_BY_COSMIC_RAY if there is nothing that you can do 
about it.  Every error that you cannot handle is a fatal error, and you should 
die accordingly.

>-Original Message-
>From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-
>boun...@sqlite.org] On Behalf Of Eric Rubin-Smith
>Sent: Monday, 30 June, 2014 12:47
>To: General Discussion of SQLite Database
>Subject: [sqlite] think I need better error-handling guidance in the C
>API
>
>I've been using the sqlite Tcl API for about 3 million years, with much
>delight.  I'm venturing now into the C API -- hoping to put sqlite into a
>large monolithic "always on" C++ daemon.  I would like to use the API in
>a
>way perfectly in line with how the authors intended.
>
>But it's sort of hard to figure out exactly what the authors intended in
>some cases.
>
>Consider the man page for sqlite3_step()
>(http://sqlite.org/c3ref/step.html
>):
>
>"With the "v2" interface, any of the other result codes
> or extended result codes
> might be returned as
>well SQLITE_ERROR  means that a
>run-time error (such as a constraint violation) has occurred.
>sqlite3_step() should not be called again on the VM. More information may
>be found by calling sqlite3_errmsg()
>.
>With the legacy interface, a more specific error code (for example,
>SQLITE_INTERRUPT , SQLITE_SCHEMA
>, SQLITE_CORRUPT
>, and so forth) can be obtained by
>calling sqlite3_reset()  on the
>prepared
>statement . In the "v2" interface, the
>more specific error code is returned directly by sqlite3_step()."
>
>I am a big fan generally of the clarity of the sqlite docs.  But this
>page
>could use another section detailing all the possible result codes'
>specific
>semantics under sqlite3_step(), and in particular what the client should
>do
>in those circumstances.  Similar to how your normal linux system call man
>page details such things.
>
>The page bites a bit of that off, as you see above.  But consider e.g.
>(from http://sqlite.org/c3ref/c_abort.html):
>
>#define SQLITE_NOTICE  27   /* Notifications from sqlite3_log() */
>
>Am I supposed to look for that?  What am I supposed to do with it if I
>get
>it?  How about SQLITE_CANTOPEN?  Seems like that shouldn't be a
>possibility
>by the time we hit sqlite3_step(), but the docs are silent on the matter.
>So what am I to do?
>
>If I get SQLITE_IOERR, is that always permanent, or do I need to look for
>things like SQLITE_IOERR_LOCK and if that bit is set then rollback and
>retry?
>
>
>sqlite3_exec() has the same issue: it's hard to say from reading the docs
>exactly which of the error codes might be returned, and what the caller
>should do in those cases.
>
>sqlite3_prepare_v2() again has the same issue:
>
>"On success, the sqlite3_prepare() family of routines return SQLITE_OK
>; otherwise an error code
> is returned."
>
>Again, can *any* of those error codes be returned?  Which of them is
>permanent and which are temporary in the case of prepare_v2()?
>
>And so on.
>
>Apologies if I have missed some of the docs.  Any pointers, or especially
>example C code that the SQLite devs consider perfectly robust and
>complete
>API usage, are much appreciated.
>
>Eric
>___
>sqlite-users mailing list
>sqlite-users@sqlite.org
>http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] xCommit is called without a prior xBegin when creating a virtual table.

[Peter Aronson]

According to the documentation for the xCommit virtual table method "A call to 
this method always follows a prior call to xBegin and xSync."  However, this 
does not seem to be the case when actually creating a virtual table.  The 
post-create xCommit call is made without any prior xBegin call.  Stepping 
through sqlite3VtabBegin(), it looks like pVTab is always NULL, so no call is 
made to xBegin.

SQLite 3.8.4.3
Solaris 9

Is there anyone out there using the xBegin, xCommit,. xRollback methods in 
their virtual table?  If so, do you pay attention to errors, or only return 
success?

Peter
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Multiple SQLiteDataReader objects against a single connection

[Joseph L. Casale]

Hey guys,
How does one accomplish this in the case where I iterate over a long result set
with the first reader open, then open a new reader against a prepared statement
and pass in a value derived from the first reader.

Thanks,
jlc
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] think I need better error-handling guidance in the C API

[Simon Slavin]

On 30 Jun 2014, at 7:47pm, Eric Rubin-Smith  wrote:

> I am a big fan generally of the clarity of the sqlite docs.  But this page
> could use another section detailing all the possible result codes' specific
> semantics under sqlite3_step(), and in particular what the client should do
> in those circumstances.

I agree with this, and want to extend it to other calls.  Many of the API calls 
could usefully do with an explanation of non _OK results, what they mean, and 
how a typical app might react to them (including other sqlite3_ calls that 
might be used).

I have tried three times to explain the combinations of things that might lead 
to ROLLBACK, which calls to use, and how to react to failure of the ROLLBACK 
command.  I failed, because I lack detailed understanding of what might cause 
ROLLBACK to fail and what it's still safe to try.

There's also an unfortunate consequence of SQLite trying to be helpful.  In 
strict SQL the programmer has to issue BEGIN before any operation and ROLLBACK 
or END after it.  So if you issue ROLLBACK the SQL engine can safely ignore 
anything until it sees another BEGIN.  But SQLite automatically wraps 
individual operations in transactions.  So it can see another command before 
the END and it has to do it.  Which can be nasty.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] think I need better error-handling guidance in the C API

[Eric Rubin-Smith]

I've been using the sqlite Tcl API for about 3 million years, with much
delight.  I'm venturing now into the C API -- hoping to put sqlite into a
large monolithic "always on" C++ daemon.  I would like to use the API in a
way perfectly in line with how the authors intended.

But it's sort of hard to figure out exactly what the authors intended in
some cases.

Consider the man page for sqlite3_step() (http://sqlite.org/c3ref/step.html
):

"With the "v2" interface, any of the other result codes
 or extended result codes
 might be returned as
well SQLITE_ERROR  means that a
run-time error (such as a constraint violation) has occurred.
sqlite3_step() should not be called again on the VM. More information may
be found by calling sqlite3_errmsg() .
With the legacy interface, a more specific error code (for example,
SQLITE_INTERRUPT , SQLITE_SCHEMA
, SQLITE_CORRUPT
, and so forth) can be obtained by
calling sqlite3_reset()  on the prepared
statement . In the "v2" interface, the
more specific error code is returned directly by sqlite3_step()."

I am a big fan generally of the clarity of the sqlite docs.  But this page
could use another section detailing all the possible result codes' specific
semantics under sqlite3_step(), and in particular what the client should do
in those circumstances.  Similar to how your normal linux system call man
page details such things.

The page bites a bit of that off, as you see above.  But consider e.g.
(from http://sqlite.org/c3ref/c_abort.html):

#define SQLITE_NOTICE  27   /* Notifications from sqlite3_log() */

Am I supposed to look for that?  What am I supposed to do with it if I get
it?  How about SQLITE_CANTOPEN?  Seems like that shouldn't be a possibility
by the time we hit sqlite3_step(), but the docs are silent on the matter.
So what am I to do?

If I get SQLITE_IOERR, is that always permanent, or do I need to look for
things like SQLITE_IOERR_LOCK and if that bit is set then rollback and
retry?


sqlite3_exec() has the same issue: it's hard to say from reading the docs
exactly which of the error codes might be returned, and what the caller
should do in those cases.

sqlite3_prepare_v2() again has the same issue:

"On success, the sqlite3_prepare() family of routines return SQLITE_OK
; otherwise an error code
 is returned."

Again, can *any* of those error codes be returned?  Which of them is
permanent and which are temporary in the case of prepare_v2()?

And so on.

Apologies if I have missed some of the docs.  Any pointers, or especially
example C code that the SQLite devs consider perfectly robust and complete
API usage, are much appreciated.

Eric
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] hoe to create index to a big table?

[GB]

Do you possibly have redirected temporary storage to memory, either by 
compile switch or #pragma temp_store? If so, try explicitly setting 
#pragma temp_store=1, this will force temporary data to be stored on disk.


Gerd.

Hadashi, Rinat schrieb am 30.06.2014 10:30:

I work with a very big table, on Linux.
I fail to create index getting the following error:
SQL error near line 1: database or disk is full

Any advice?

Rinat Hadashi


-
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] JDBC and savepoints

[gwenn]

Hello,
Are you sure?
http://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html#setSavepoint()
Regards.

On Sun, Jun 29, 2014 at 12:15 PM, hala  wrote:
> JDBC does not support savepoints from SQLite
>
> is there any replacement for savepoints?
>
> if not what to use for bulk inserts to ensure the possibility of rolling
> back without losing much data?
>
>
>
> --
> View this message in context: 
> http://sqlite.1065341.n5.nabble.com/JDBC-and-savepoints-tp76304.html
> Sent from the SQLite mailing list archive at Nabble.com.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Is 32bit SQLite limited to 1900mb RAM in windows?

[Richard Hipp]

On Mon, Jun 30, 2014 at 12:14 PM, Nick Eubank  wrote:

> Sorry, I wasn't clear: I have 64 bit R and Windows, but since there are no
> 64 bit binaries for SQLite I started with 32 bit SQLite.
>
> So now I'm wondering if I could fix this memory problem by installing the
> 64-bit version of SQLite.
>

Probably so, yes.  We are working on providing a 64-bit DLL on the download
site now.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Is 32bit SQLite limited to 1900mb RAM in windows?

[Richard Hipp]

A 64-bit Windows DLL is now available at http://www.sqlite.org/download.html


On Mon, Jun 30, 2014 at 12:26 PM, Richard Hipp  wrote:

>
>
>
> On Mon, Jun 30, 2014 at 12:14 PM, Nick Eubank 
> wrote:
>
>> Sorry, I wasn't clear: I have 64 bit R and Windows, but since there are no
>> 64 bit binaries for SQLite I started with 32 bit SQLite.
>>
>> So now I'm wondering if I could fix this memory problem by installing the
>> 64-bit version of SQLite.
>>
>
> Probably so, yes.  We are working on providing a 64-bit DLL on the
> download site now.
>
> --
> D. Richard Hipp
> d...@sqlite.org
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Is 32bit SQLite limited to 1900mb RAM in windows?

[Nick Eubank]

Richard, you are my hero. :)


On Mon, Jun 30, 2014 at 9:31 AM, Richard Hipp  wrote:

> A 64-bit Windows DLL is now available at
> http://www.sqlite.org/download.html
>
>
> On Mon, Jun 30, 2014 at 12:26 PM, Richard Hipp  wrote:
>
> >
> >
> >
> > On Mon, Jun 30, 2014 at 12:14 PM, Nick Eubank 
> > wrote:
> >
> >> Sorry, I wasn't clear: I have 64 bit R and Windows, but since there are
> no
> >> 64 bit binaries for SQLite I started with 32 bit SQLite.
> >>
> >> So now I'm wondering if I could fix this memory problem by installing
> the
> >> 64-bit version of SQLite.
> >>
> >
> > Probably so, yes.  We are working on providing a 64-bit DLL on the
> > download site now.
> >
> > --
> > D. Richard Hipp
> > d...@sqlite.org
> >
>
>
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Is 32bit SQLite limited to 1900mb RAM in windows?

[Nelson, Erik - 2]

Simon Slavin wrote on Monday, June 30, 2014 12:21 PM
> 
> SQLite isn't a thing, it's an API.  There's no SQLite server.  There's
> no particular installation of SQLite on a computer that Windows expects
> to be in a particular place.  You can have many copies of many
> different versions of SQLite in different folders on your computer.
> You can have twenty different apps on your computer with twenty
> different versions of SQLite compiled into them, some 32-bit, some 64-
> bit, and they'll all work fine.
> 

I'd say it's a 'thing'... it's something you can find somewhere on your hard 
drive, maybe twenty different things, in the case.

I think what the OP is trying to find out is if a 64-bit compiled version of 
SQLite properly accesses the 64-bit (ish) memory space, or is it still 
effectively limited to 32-bit addressing.

In my experience, 64-bit SQLite is cleanly addressing the larger space.

Erik

--
This message, and any attachments, is for the intended recipient(s) only, may 
contain information that is privileged, confidential and/or proprietary and 
subject to important terms and conditions available at 
http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended 
recipient, please delete this message.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Is 32bit SQLite limited to 1900mb RAM in windows?

[Simon Slavin]

On 30 Jun 2014, at 5:14pm, Nick Eubank  wrote:

> So now I'm wondering if I could fix this memory problem by installing the
> 64-bit version of SQLite.

SQLite isn't a thing, it's an API.  There's no SQLite server.  There's no 
particular installation of SQLite on a computer that Windows expects to be in a 
particular place.  You can have many copies of many different versions of 
SQLite in different folders on your computer.  You can have twenty different 
apps on your computer with twenty different versions of SQLite compiled into 
them, some 32-bit, some 64-bit, and they'll all work fine.

What you're concerned about is what your copy of R is using to talk to your 
SQLite databases.  Which is probably an ODBC driver.  I don't know much about 
them, but I found this page which is relevant:



Maybe someone who uses ODBC drivers more than me can suggest a better source.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Is 32bit SQLite limited to 1900mb RAM in windows?

[Nick Eubank]

Sorry, I wasn't clear: I have 64 bit R and Windows, but since there are no
64 bit binaries for SQLite I started with 32 bit SQLite.

So now I'm wondering if I could fix this memory problem by installing the
64-bit version of SQLite.

(I know that this wouldn't fix the problem in postgres, for example,
because they never updated the addressing code when they built a 64bit
version -- a lesson I learned the hard way. )

On Monday, June 30, 2014, Simon Slavin  wrote:

>
> On 30 Jun 2014, at 4:58pm, Nick Eubank  > wrote:
>
> > Thanks Cory -- is this fixed in 64 bit versions of SQLite? I know
> postgres
> > never changed memory address variable storage in the 64 bit so the
> problem
> > persists.
>
> You are misunderstanding the problem.  There is no bug to be fixed.  It is
> in the nature of 32-bit Windows that no process has access to more than
> 4Gig of memory.  In your case, with your version of Windows, it's 2Gig.
>  Only Microsoft can fix this and they did that by releasing a 64-bit
> version of Windows.
>
> So you need a 64-bit version of R, 64-bit ODBC drivers (which means 64-bit
> version of Office if you have it), and a 64-bit version of Windows.
>
> > Also: any advice on getting a 64bit installation for someone who doesn't
> > really know how to compile C?
>
> I don't know much about that, but this page might be relevant:
>
> 
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org 
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Is 32bit SQLite limited to 1900mb RAM in windows?

[Simon Slavin]

On 30 Jun 2014, at 4:58pm, Nick Eubank  wrote:

> Thanks Cory -- is this fixed in 64 bit versions of SQLite? I know postgres
> never changed memory address variable storage in the 64 bit so the problem
> persists.

You are misunderstanding the problem.  There is no bug to be fixed.  It is in 
the nature of 32-bit Windows that no process has access to more than 4Gig of 
memory.  In your case, with your version of Windows, it's 2Gig.  Only Microsoft 
can fix this and they did that by releasing a 64-bit version of Windows.

So you need a 64-bit version of R, 64-bit ODBC drivers (which means 64-bit 
version of Office if you have it), and a 64-bit version of Windows.

> Also: any advice on getting a 64bit installation for someone who doesn't
> really know how to compile C?

I don't know much about that, but this page might be relevant:



Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Is 32bit SQLite limited to 1900mb RAM in windows?

[Nick Eubank]

Thanks Cory -- is this fixed in 64 bit versions of SQLite? I know postgres
never changed memory address variable storage in the 64 bit so the problem
persists.

Also: any advice on getting a 64bit installation for someone who doesn't
really know how to compile C?

Thanks!

Nick

On Monday, June 30, 2014, Cory Nelson  wrote:

> Without special handling, any 32-bit Windows process is limited to a 2GB
> user address space. Due to fragmentation during allocation, you'll never
> reach a full 2GB.
>
>
> On Mon, Jun 30, 2014 at 10:31 AM, Nick Eubank  > wrote:
>
> > Hi All,
> >
> > I'm a social scientist wrestling with SQLite in Windows 8 (through R
> using
> > the RSQLite library) for some data manipulation and querying.
> >
> > No matter what I do to cache_size (or R's memory settings), SQLite never
> > seems to be using more than about ~1900 mb of RAM. Is that a result of
> the
> > 32 bit build (on another project I discovered 32bit windows is limited to
> > 1900 mb per thread due to how memory addresses are stored)?
> >
> > If so, any advice on installing 64bit version on Windows 8 for someone
> with
> > zero experience compiling C?
> >
> > Thank you so much!
> >
> > Nick
> > ___
> > sqlite-users mailing list
> > sqlite-users@sqlite.org 
> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> >
>
>
>
> --
> Cory Nelson
> http://int64.org
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org 
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Is 32bit SQLite limited to 1900mb RAM in windows?

[Cory Nelson]

Without special handling, any 32-bit Windows process is limited to a 2GB
user address space. Due to fragmentation during allocation, you'll never
reach a full 2GB.


On Mon, Jun 30, 2014 at 10:31 AM, Nick Eubank  wrote:

> Hi All,
>
> I'm a social scientist wrestling with SQLite in Windows 8 (through R using
> the RSQLite library) for some data manipulation and querying.
>
> No matter what I do to cache_size (or R's memory settings), SQLite never
> seems to be using more than about ~1900 mb of RAM. Is that a result of the
> 32 bit build (on another project I discovered 32bit windows is limited to
> 1900 mb per thread due to how memory addresses are stored)?
>
> If so, any advice on installing 64bit version on Windows 8 for someone with
> zero experience compiling C?
>
> Thank you so much!
>
> Nick
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Cory Nelson
http://int64.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Is 32bit SQLite limited to 1900mb RAM in windows?

[Nick Eubank]

Hi All,

I'm a social scientist wrestling with SQLite in Windows 8 (through R using
the RSQLite library) for some data manipulation and querying.

No matter what I do to cache_size (or R's memory settings), SQLite never
seems to be using more than about ~1900 mb of RAM. Is that a result of the
32 bit build (on another project I discovered 32bit windows is limited to
1900 mb per thread due to how memory addresses are stored)?

If so, any advice on installing 64bit version on Windows 8 for someone with
zero experience compiling C?

Thank you so much!

Nick
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Merge join in SQLite

[Richard Hipp]

On Mon, Jun 30, 2014 at 9:37 AM, Eleytherios Stamatogiannakis <
est...@gmail.com> wrote:

> Thank you for the answer as well as the merge-join complexity remark. What
> about the union-all with ordered input and an order-by on the whole query?
> Does SQLite use a "merge" algorithm for that case?
>

Yes.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Merge join in SQLite

[Eleytherios Stamatogiannakis]

Thank you for the answer as well as the merge-join complexity remark. 
What about the union-all with ordered input and an order-by on the whole 
query? Does SQLite use a "merge" algorithm for that case?


Kind regards,

l.

On 26/06/14 21:45, Richard Hipp wrote:




On Thu, Jun 26, 2014 at 11:41 AM, Eleytherios Stamatogiannakis
> wrote:

Hi,

I've seen that SQLite supports the "SorterOpen" Op, to do merge
sorting. So based on that, does SQLite support merge joins when both
inputs of the join are sorted?


No, not at this time.  Note that a merge-join is more complicated than
it appears at first glance for the common case where the join key is not
unique in one or the other of the two tables being joined.


--
D. Richard Hipp
d...@sqlite.org 


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Building for vxWorks

[Andy Ling]

Hi,

I am trying to build sqlite 3.8.5 for vxWorks 6.9 and I'm after a bit of help.

To be more precise, I'm building an RTP for 64bit NEHALEM on vxWorks 6.9.3

Straight out of the box it won't build, I get the following errors

sqlite3.c:24997: error: expected specifier-qualifier-list before 'sem_t'
sqlite3.c: In function 'fileHasMoved':
sqlite3.c:25237: error: 'struct unixFileId' has no member named 'ino'
sqlite3.c: In function 'semCheckReservedLock':

The first I have fixed by adding a #include  and the second I've 
fixed by bodging fileHasMoved to always return false.
Is there a compile option I should have used that would have fixed these?

It then builds, but has a few unresolved symbols. These I have fixed by adding 
the compile options

-DHAVE_UTIME
-DSQLITE_OMIT_LOAD_EXTENSION

It now runs, but anything that tries to modify a database file generates a Disk 
I/O error.

It lets me create a database and a file gets created on disk. I can do simple 
stuff like

SELECT date('now')

And that returns sensible stuff. So lots of it is working. But as soon as I try 
to create a table I get a disk I/O error. So I can't get much further.

Does anyone have any ideas about what I need to do to make this work.

Thanks

Andy Ling


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] JDBC and savepoints

[hala]

JDBC does not support savepoints from SQLite

is there any replacement for savepoints?

if not what to use for bulk inserts to ensure the possibility of rolling
back without losing much data?



--
View this message in context: 
http://sqlite.1065341.n5.nabble.com/JDBC-and-savepoints-tp76304.html
Sent from the SQLite mailing list archive at Nabble.com.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Adding WHERE to query --> database disk image is malformed

[James K. Lowden]

On Tue, 24 Jun 2014 22:06:15 -0700
Jerry Krinock  wrote:

> > The most likely explanation is that it got processed by something
> > which thought it should be treated as ASCII text and was doing a
> > spurious LF-to-CR translation. If there was only one 0x0A byte in
> > the "good" file, then that is the only one which would have been
> > modified.
> 
> Yes, that makes sense, David.  I don?t think that happened.  But I?m
> still thinking :)

Like perhaps maybe the file's git properties use native line endings,
and you're programming on a Mac?  

It seems unlikely, though, that the entire database would contain only
a single 10.  

--jkl

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Sequential numbers

[James K. Lowden]

On Tue, 24 Jun 2014 21:02:22 +0100
"Dave Wellman"  wrote:

> 2)  Assuming that my processing follows this pattern: empty table
> T1 completely, insert a number of rows, insert/select from T1 into
> T2. On the 'select' processing will the 'rowid' 
> ** always ** start at 1?

http://www.schemamania.org/sql/#rank.rows

If the use of the sequential number is ephemeral,  why  store it?  Just
generate it with a self-join.  

If the use of the sequential number is to identify the row in
perpetuity, you may find it's more trouble than it's worth.  

--jkl

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] hoe to create index to a big table?

[RSmith]

On 2014/06/30 10:30, Hadashi, Rinat wrote:

I work with a very big table, on Linux.
I fail to create index getting the following error:
SQL error near line 1: database or disk is full

Any advice?


Hi Rinat - I was going to simply say "Get a bigger harddrive?" with a smile attached. Obviously you have already established that 
your Hard drive is in fact not full, so let me try a more useful approach


When you create a new Index it requires a lot of extra space, both for the Index itself and the process of making the Index with 
sorting, possibly uniqueness checking and the like. We cannot tell you exactly how much memory is needed because nobody knows since 
it depends very much on the data shape and size in the table and column for which you are making an Index - all I can say with 
reasonable certainty is that it typically requires a lot of space - very much in proportion to the size of the full DB, but the 
exact proportion depends on the data.


To elaborate, it is possible for an index and process of creating an index to require as little as 10% of the full DB size 
(typically for a table where the data is very wide, not many rows but lots of columns with lots of fat data), and it might require 
almost double the size of the original DB (on top of the actual DB's filesize) to create and store the Index initially if the data 
is thin, 2 or 3 columns containing millions of rows with little data per row, such as a logger.


if however you have more than double the free space on your drive (let's say 1GB data file and you have 2GB+ free space) then I 
would start to look elsewhere for a problem. It is hard to conceive an Index creation using more than that, I think a hard upper 
limit % do exist but maybe someone else here might know exactly as I don't.


In most flavours of Linux though, you can just "grow" a volume onto some other attached drive and then release space again 
afterwards - which might solve your problem. (Backup data before attempting this) - a discussion on that is probably not suitable to 
this forum but a simple google will get you all kinds of instructions in this regard.


Cheers!
Ryan
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] hoe to create index to a big table?

[Dan Kennedy]

On 06/30/2014 03:30 PM, Hadashi, Rinat wrote:

I work with a very big table, on Linux.
I fail to create index getting the following error:
SQL error near line 1: database or disk is full


A CREATE INDEX on a large table uses temporary files to sort the data 
before creating the actual index b-tree. So even if you have enough 
space in the file-system that the database is stored in, you might still 
be running out of temp space. The maximum amount of temp space required 
by a CREATE INDEX is roughly twice the size of the final index on disk.


To set the directory used for temporary files on Linux, set the TMPDIR 
(or SQLITE_TMPDIR) environment variable.


Dan.





Any advice?

Rinat Hadashi


-
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] hoe to create index to a big table?

[Hadashi, Rinat]

I work with a very big table, on Linux.
I fail to create index getting the following error:
SQL error near line 1: database or disk is full

Any advice?

Rinat Hadashi


-
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using sqlite3_file in shims.

[Klaas V]

Renji Panicker wrote to this fine list:

>
>- As per the sqlite3PagerOpen function, the size of the memory allocated to
>this pointer is ROUND8(pVfs->szOsFile).

What configuration you're using, Renji?
On some OS/browser/SupportSoftware etc. combinations this function may crash

https://crash-stats.mozilla.com/report/list?signature=sqlite3PagerOpen

There seem to be version differences as well.

 

Kind regards | Cordiali saluti | Vriendelijke groeten | Freundliche Grüsse,
Klaas `Z4us` V  - OrcID -0001-7190-2544
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users