Re: [sqlite] sqlite3 tool bug

[Scott Robison]

On Jan 17, 2015 7:29 PM, "Dave Dyer" 
> Here in the real world, when everything is working, we ask "why upgrade".

But it wasn't working correctly so the statement doesn't really answer the
question asked. :)
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Dave Dyer]

>
>The  has been in shell.c since 3.8.6.  We are on 3.8.8.  Why
>not upgrade?
>-- 

Here in the real world, when everything is working, we ask "why upgrade".

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Richard Hipp]

On 1/17/15, Dave Dyer  wrote:
>
>>
>>
>>OK.  Dave, please try this patch at let us know if it works better for
>>you:  https://www.sqlite.org/src/info/80541e8b94b7
>>
>
> It needs #include  to compile in my sources.
> With that, it seems to fix the problem.
>

The  has been in shell.c since 3.8.6.  We are on 3.8.8.  Why
not upgrade?
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Dave Dyer]

>
>
>OK.  Dave, please try this patch at let us know if it works better for
>you:  https://www.sqlite.org/src/info/80541e8b94b7
>

It needs #include  to compile in my sources.
With that, it seems to fix the problem.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Richard Hipp]

On 1/17/15, Roger Binns  wrote:
>>
> The bug in the SQLite shell is that it tries to manage the encoding
> itself, which is fine if the file is in binary mode.  But with
> stdin/out in text mode doing so will lead to extra data mangling.  The
> shell needs to change stdin/out to binary mode:
>

OK.  Dave, please try this patch at let us know if it works better for
you:  https://www.sqlite.org/src/info/80541e8b94b7


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/17/2015 12:20 PM, Graham Holden wrote:
> I would echo this: it's good at mangling the command-line, but
> I've not been aware of it ever mangling data sent to a file/stream
> (other than the binary/text mode conversions).

In text mode (the default for stdin/out on Windows) control Z (ascii
26) is also considered end of file.

Microsoft don't document everything that is done to text streams, but
there is likely to also be some conversions for the current code page
and MBCS.  Here for example is what is done with stdio when using the
wide character methods:

  http://msdn.microsoft.com/en-us/library/c4cy2b8e.aspx

The bug in the SQLite shell is that it tries to manage the encoding
itself, which is fine if the file is in binary mode.  But with
stdin/out in text mode doing so will lead to extra data mangling.  The
shell needs to change stdin/out to binary mode:

  http://msdn.microsoft.com/en-us/library/tw4k6df8.aspx

Roger

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iEYEARECAAYFAlS68dsACgkQmOOfHg372QSWRQCfeBr7J/p0VhqsDwRAhDcSDq3d
MuYAoLG9R5Z3DiEHQgYTY/Ulpu7ilgIi
=/lr/
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug summary

[Dave Dyer]

>
> .once '| sqlite3 new.db'
> .dump

.Once is not a command in the version of sqlite3 I use.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug summary

[Richard Hipp]

On 1/17/15, Dave Dyer  wrote:
>
>>
>>> But that doesn't explain the difference between redirecting to a file
>>> and redirecting to a pipe.
> using .output file works
> using > to direct stdout to a file works and produces the same file as
> .output
> using .read file works
> using < file does not work.
> using | to shortcut > and < doesn't work.

Thanks for the excellent summary.

But what about this case:

 .once '| sqlite3 new.db'
 .dump


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug summary

[Dave Dyer]

>
>> But that doesn't explain the difference between redirecting to a file
>> and redirecting to a pipe.
using .output file works
using > to direct stdout to a file works and produces the same file as .output
using .read file works
using < file does not work.
using | to shortcut > and < doesn't work.


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Graham Holden]

> I'm skeptical of the notion that cmd.exe is diddling with your data en
> route to the pipe.  I can't think of a time Windows munged my data in
> that particular way despite more years using that lousy tool than I
> care to remember.  Quotes and escapes, sure, don't get me started.

I would echo this: it's good at mangling the command-line, but I've
not been aware of it ever mangling data sent to a file/stream (other
than the binary/text mode conversions).

> Shot in the dark: maybe a string is being continued by ending the
> line with a backslash.  If the output handle is opened as text with
> fopen, the sequence would be
> 5c 0d 0a
> which the escape-reader wouldn't recognize, expecting only
> 5c 0a
> leading to a noncontinued, incomplete line.

I don't believe this would be the case, if both stdin and stdout have
been left in "text" mode: '5c 0a' sent to stdout would be converted
into '5c 0d 0a' but when this was read in "text" mode it would get
converted back to '5c 0a' which would be what the "escape-reader" of
SQLite would see.

> But that doesn't explain the difference between redirecting to a file
> and redirecting to a pipe.

Even if the data coming from the first command contained characters
that "confused" the "text" mode of Windows, I would expect things to
work or fail the same whether the output's being piped or sent to a
file.

My "shot in the dark" would be that some buffer-size limit in the
piping process is being exceeded.

Graham Holden



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Dave Dyer]

>
>Not, at least, when your database contains string data with unusual
>characters that Windows feels like it should translate for you...

Who can guarantee what characters are used in all their text strings,
much less guarantee what unnamed transformations windows is helpfully
doing to pipe data.  

Saying "it probably will work" is not very satisfactory.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Dave Dyer]

I have a class of database for which using sqlite3 to create
a copy via the "pipe" method fails.   Using an explicit intermediate
file seems to work ok.

I can supply a sample database to anyone interested in investigating.

--

F:\2013 YearTech\Yearbook Tools\Resource>sqlite3 -version
3.7.3

F:\2013 YearTech\Yearbook Tools\Resource>sqlite3 po.sqlite .dump | sqlite3 
po2.sqlite
Error: incomplete SQL: INSERT INTO "imageblob" VALUES(1,'G:\share\e ...

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Scott Robison]

On Fri, Jan 16, 2015 at 3:48 PM, Keith Medcalf  wrote:

>
> >You have a system with a bunch of apps installed.  You then upgrade to
> >a new version of the operating system and a whole bunch of the apps
> >break.  Do you think people blame the apps or the operating system?
> >Do you think anyone takes the apps apart and blames them for using the
> >wrong apis despite documentation to the contrary?
>
> This is kind of begging the question.  There is lots of software that runs
> perfectly well on *all* 32-bit capable versions of Windows.  This is not
> particularly hard to do, you just have to not do stupid things.  If the
> application has problems that is an application problem, or an application
> design issue.  It has nothing whatsoever to do with the Operating System.
>
> Quite frankly, it is a Microsoft strategy to force application
> incompatibility, and something Microsoft sells as a "feature" to their
> ISVs.  Each new Microsoft "thing" is sold to ISVs with the promise that "if
> you use this new buzzword technology, which we have paid handsomely to have
> the pundit and magazine writers claim is the greatest thing since sliced
> bread" we can guarantee you that your customers will keep coming back and
> buying the same product over and over and over again because we will
> "embrace, extend, and make incompatible", deliberately, with each version
> and patch, in order to bolster your revenue stream.  It is also sold to
> Enterprises on the same basis, because that ensures that the IT Department
> will has to be continually staffed (big = more underthings = higher paid
> execs = bigger budgets = bigger bonusses) with throngs of people to keep
> running along the Microsoft designed incompatibility treadmill.
>

While there is some truth to this, it is more nuanced. There are bugs &
misfeatures that Microsoft maintains specifically not to annoy paying
customers, and there are undocumented APIs that developers have discovered
and used (not just in Microsoft, but certainly they have the best chance of
accessing those undocumented APIs) that Microsoft maintains some level of
compatibility with just to avoid breaking selected apps some customers
depend on.

Linux, BSD, whatever, caters to a technical crowd. If those developers
decide they want to change the kernel API / ABI at every release, it is not
the end of the world. They hold less than 10% of desktop systems (depending
on the source). Windows caters to a much later audience, many who have no
technical expertise, much less source code to recompile software when
operating system changes break applications. Even if the applications are
the real source of the problem (using undocumented APIs or depending on
bugs) that doesn't matter to customers.

Frankly, given the vast array of hardware that Windows tries to be
compatible with, and all the permutations of hardware & drivers & such, I
think it is a miracle they that succeed as often as they do. Do they make
stupid decisions / choices at times? Sure. Everybody does. But that doesn't
mean they are all incompetent or stupid. I dare say DRH would like to (and
has said as much IIUC) go back in time and make changes that were set in
stone years ago, but he specifically does not do it to maintain backward
compatibility. Heck, if nothing else, I'm sure he'd love to go back and
eliminate every bug that has forced a patch update.

-- 
Scott Robison
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Simon Slavin]

On 16 Jan 2015, at 10:27pm, Keith Medcalf  wrote:

> [snip] The long and the short of it is that the interprocess pipe in Windows 
> connects to cooked channels because it never occurred to anyone at Microsoft 
> that this was undesirable and irrational.

Thanks for this long explanation which I found interesting and useful.  And 
annoying in that it decreases my appreciation of Windows still further.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/16/2015 02:35 PM, James K. Lowden wrote:
> I'm skeptical of the notion that cmd.exe is diddling with your data
> en route to the pipe.

Almost certainly the reason is that stdout and stdin are in character
mode.  It requires extra code to put them into binary mode:

  http://support.microsoft.com/kb/58427

Roger

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iEYEARECAAYFAlS5lz4ACgkQmOOfHg372QR6mACg4MmvKwP3pvZ0AJwLyKl08sGm
Z/0An050kV+FhIUtHnyZzKYGYEiK836s
=aG9W
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Keith Medcalf]

>You have a system with a bunch of apps installed.  You then upgrade to
>a new version of the operating system and a whole bunch of the apps
>break.  Do you think people blame the apps or the operating system?
>Do you think anyone takes the apps apart and blames them for using the
>wrong apis despite documentation to the contrary?

This is kind of begging the question.  There is lots of software that runs 
perfectly well on *all* 32-bit capable versions of Windows.  This is not 
particularly hard to do, you just have to not do stupid things.  If the 
application has problems that is an application problem, or an application 
design issue.  It has nothing whatsoever to do with the Operating System.

Quite frankly, it is a Microsoft strategy to force application incompatibility, 
and something Microsoft sells as a "feature" to their ISVs.  Each new Microsoft 
"thing" is sold to ISVs with the promise that "if you use this new buzzword 
technology, which we have paid handsomely to have the pundit and magazine 
writers claim is the greatest thing since sliced bread" we can guarantee you 
that your customers will keep coming back and buying the same product over and 
over and over again because we will "embrace, extend, and make incompatible", 
deliberately, with each version and patch, in order to bolster your revenue 
stream.  It is also sold to Enterprises on the same basis, because that ensures 
that the IT Department will has to be continually staffed (big = more 
underthings = higher paid execs = bigger budgets = bigger bonusses) with 
throngs of people to keep running along the Microsoft designed incompatibility 
treadmill.

---
Theory is when you know everything but nothing works.  Practice is when 
everything works but no one knows why.  Sometimes theory and practice are 
combined:  nothing works and no one knows why.




___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[James K. Lowden]

On Fri, 16 Jan 2015 10:38:54 -0800
Dave Dyer  wrote:

> [$] sqlite3 po.sqlite .dump | sqlite3 po2.sqlite
> Error: incomplete SQL: INSERT INTO "imageblob" VALUES(1,'G:\share

Perhaps try -echo, to display the incomplete SQL?  

I'm skeptical of the notion that cmd.exe is diddling with your data en
route to the pipe.  I can't think of a time Windows munged my data in
that particular way despite more years using that lousy tool than I
care to remember.  Quotes and escapes, sure, don't get me started. 

Shot in the dark: maybe a string is being continued by ending the
line with a backslash.  If the output handle is opened as text with
fopen, the sequence would be 

5c 0d 0a

which the escape-reader wouldn't recognize, expecting only 

5c 0a

leading to a noncontinued, incomplete line.  

But that doesn't explain the difference between redirecting to a file
and redirecting to a pipe.  

HTH.  

--jkl


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Keith Medcalf]

On Friday, 16 January, 2015 14:05, Simon Slavin  said:
>On 16 Jan 2015, at 9:01pm, Keith Medcalf  wrote:

>>> Not, at least, when your database contains string data with unusual
>>> characters that Windows feels like it should translate for you...

>> I think that pretty much limits one to the 7-bit ASCII character set
>> ...

>Why on earth would an operating system programmer bother to put any
>translation into piping ? Piping is really just a demonstration of how to
>use stdin and stdout.  I'm mystified.

They wouldn't.  

However, we are not talking about a real Operating System designed by real 
System Programmers -- we are talking about Microsoft Windows, which contains a 
lot of disparate code sources all stuck together with LePages White Glue which 
maintains backwards comptibility with, oh, DOS. 

DOS implemented the pipe "|" operator in "a | b" as "a > tempfile" followed by 
"b < tempfile" -- because it could not actually run two processes at the same 
time.  This meant that the pipe was 8-bit clean.  This method of implementation 
of pipes stayed the same until OS/2, which was the first truly multitasking 
descendant of DOS (more or less).  It implemented the interprocess pipe as one 
would expect -- by opening an OS 8-bit clean anonymous pipe, then cloning the 
input/output handles onto stdout and stdin of the two processes respectively.

Then along came "OS/2 New Technology" in which Microsoft renaged on the Joint 
Development agreement between IBM and Microsoft, and renamed the product 
"Windows NT".  Under the termination agreement, the codebase became the 
property of both IBM and Microsoft.  However, under the JDA the codebase 
contained quite a bit of patented and proprietary IBM code and technology.  
Microsoft was given a few years to replace all the IBM technology with their 
own (so, for example, the "Microsoft" re-written versions of all the DDE and 
IPC mechanisms, which were patented and proprietary IBM based on their 
experience with "real computing systems" where renamed COM/DCOM/ActiveX) in 
various flavours and re-implemented by Microsoft.  This eventually became 
Windows NT 4.0 (which is I believe the first version of "OS/2 New Technology" 
which was devoid of all the IBM code and partial workarounds.  Microsoft 
programmers, being the rather simple folks that they are, re-implemented many 
of those IBM bits with
  rather simplistic and crude implementations and others, that they couldn't 
figure out how to do at all such as priority-based pre-emptive multitasking, 
they simply replaced with much simpler implementations derived from their DOS 
and Windows 3.1 GUI.  Without actually redesigning and fixing those broken 
design decisions (like the one where they decided that the screensaver -- when 
triggered -- should be the highest priority task in the system), current 
versions of "Microsoft Stuff" still operates from this old ill-concieved design 
basis.  They still haven't quite figured out how to do the "pre-emption" part 
of multitasking yet, but they are getting (very slightly) better.  If Microsoft 
eventually gets around to fixing their busted dispatcher, their busted I/O 
systems, and their busted process isolation (which they deliberately bust in 
order for their other software to work) then they will have a "real" Operating 
System -- probably somewhere around Windows 25.6 at their present rate
  of progress (although, of course, that Ballmer idiot really screwed things 
up, and their propensity for doing really assinine things, makes it more likely 
that they will never be able to produce anything that would qualify as an 
"Operating System" in the true sense of that word).

Anyway ...

The pipe operator connects the "output" from one program to the "input" of the 
next.  On Windows both stdout (to the console) and stdin (from the keyboard) 
are "cooked" streams.  Using the "|" operator connects these two cooked streams 
together (meaning that Windows will perform the same output translation as if 
it were outputting to the console, and the same input cooking as if the input 
wre coming from the keyboard.

Contrast this with the "file pipe" operators < and > which cause a file to be 
opened and then that file handle to be "cloned" over top of the stdout or stdin 
handle -- meaning that the "console cookery" is no longer in effect -- you now 
have an 8-bit clear channel.

Real Operating Systems written by real System Programmers implement the pipe | 
operator by opening a pipe (which is an 8-bit clean communication channel, like 
a file) and then cloning the handles to the appropriate process stdin/stdout 
handles, thus the kitchen is disabled and the I/O is not cooked.

It would be nice if Windows had a "treat all I/O as 8-bit clean" switch and you 
could disable the cook/kitchen, but you still cannot.  Just as there is no 
switch to let you, with one go, switch Windows into "Intelligent Person using a 
Computer to Compute" 

Re: [sqlite] sqlite3 tool bug

[Random Coder]

On Fri, Jan 16, 2015 at 2:02 PM, Dave Dyer  wrote:
> the input side of the pipe.  Perhaps there is some windows conditioning
> that ought to be done by sqlite, on STDIN, to make it into a binary data
> source ?

You should be able to do a freopen(NULL, "rb", stdin); to change stdin
to be in binary mode.  I can't test that it'll help in this case,
however.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/16/2015 01:05 PM, Simon Slavin wrote:
> Why on earth would an operating system programmer bother to put any
> translation into piping

You have a system with a bunch of apps installed.  You then upgrade to
a new version of the operating system and a whole bunch of the apps
break.  Do you think people blame the apps or the operating system?
Do you think anyone takes the apps apart and blames them for using the
wrong apis despite documentation to the contrary?

Microsoft puts a phenomenal amount of effort into backwards
compatibility.  So the question is actually why Windows behaves the
way it does.

Pipes are done differently on Windows for historical reasons.  DOS
actually did them by writing to a file and not by running the commands
simultaneously.  Operating systems have always done something with
I/O.  C libraries (fopen etc) also do things.  Heck a good question
might be why does Unix not have a separate text type of file?  BTW
SQLite shell uses fopen but claiming binary mode.

Raymond Chen - a Microsoft employee who does a lot of work in this
area has many good articles:

http://blogs.msdn.com/b/oldnewthing/archive/2003/12/24/45779.aspx
http://blogs.msdn.com/b/oldnewthing/archive/2007/07/23/4003873.aspx
http://blogs.msdn.com/b/oldnewthing/archive/2015/01/07/10584656.aspx
http://blogs.msdn.com/b/oldnewthing/archive/2010/03/11/9976571.aspx

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iEYEARECAAYFAlS5jEsACgkQmOOfHg372QQt8gCg4Lu0r7I5eg8B4vZUygPGczxt
SdwAniolznwWWxogG0NHFX3yHkfI3SoB
=VgJ4
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Dave Dyer]

>
>Not, at least, when your database contains string data with unusual
>characters that Windows feels like it should translate for you...

Who can guarantee what characters are used in all their text strings,
much less guarantee what unnamed transformations windows is helpfully
doing to pipe data.  

Saying "it probably will work" is not very satisfactory.

--

I did a little more experimentation, and found that the problem is on
the input side of the pipe.  Perhaps there is some windows conditioning
that ought to be done by sqlite, on STDIN, to make it into a binary data 
source ?

succeeds:
   cat < pipe.txt > pipe-out.txt 
   sqlite3 database.sqlite
   .read pipe-out.txt

fails:

   sqlite3 database.sqlite < pipe.txt 




___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Simon Slavin]

On 16 Jan 2015, at 9:01pm, Keith Medcalf  wrote:

>> Not, at least, when your database contains string data with unusual
>> characters that Windows feels like it should translate for you...
> 
> I think that pretty much limits one to the 7-bit ASCII character set ...

Why on earth would an operating system programmer bother to put any translation 
into piping ? Piping is really just a demonstration of how to use stdin and 
stdout.  I'm mystified.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Keith Medcalf]

>>>I'm guess this is a case of the windows command-line shell doing some
>>>character translations in the pipe, rather than just shipping the
>>>bytes through the pipe unaltered.

>> Ouch.  That basically means the "pipe" method shouldn't ever be
>> used on windows.

>Not, at least, when your database contains string data with unusual
>characters that Windows feels like it should translate for you...

I think that pretty much limits one to the 7-bit ASCII character set ...




___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Richard Hipp]

On 1/16/15, Dave Dyer  wrote:
>
>>
>>The pipe method works fine for me on Linux.
>>
>>I'm guess this is a case of the windows command-line shell doing some
>>character translations in the pipe, rather than just shipping the
>>bytes through the pipe unaltered.
>
> Ouch.  That basically means the "pipe" method shouldn't ever be
> used on windows.
>

Not, at least, when your database contains string data with unusual
characters that Windows feels like it should translate for you...

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Dave Dyer]

>
>The pipe method works fine for me on Linux.
>
>I'm guess this is a case of the windows command-line shell doing some
>character translations in the pipe, rather than just shipping the
>bytes through the pipe unaltered.

Ouch.  That basically means the "pipe" method shouldn't ever be 
used on windows.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Dave Dyer]

>
>The pipe method works fine for me on Linux.
>
>I'm guess this is a case of the windows command-line shell doing some
>character translations in the pipe, rather than just shipping the
>bytes through the pipe unaltered.

Ouch.  That basically means the "pipe" method shouldn't ever be 
used on windows.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Richard Hipp]

On 1/16/15, Dave Dyer  wrote:
>
> I have a class of database for which using sqlite3 to create
> a copy via the "pipe" method fails.   Using an explicit intermediate
> file seems to work ok.
>

The pipe method works fine for me on Linux.

I'm guess this is a case of the windows command-line shell doing some
character translations in the pipe, rather than just shipping the
bytes through the pipe unaltered.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Dave Dyer]

>
>Rather than the full database, can you show us the full schema of this
>database, including triggers?

It's a very simple database, no triggers or coalitions.  The
problem is most likely a buffer overrun because of a very long
literal string field.

I've sent a minimal sample to drh



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Simon Slavin]

On 16 Jan 2015, at 6:38pm, Dave Dyer  wrote:

> I have a class of database for which using sqlite3 to create
> a copy via the "pipe" method fails.   Using an explicit intermediate
> file seems to work ok.

Which version of Windows are you using ?  You can type 'ver' at the prompt.

Can you reduce your database down to just a few lines which cause the problem, 
then post those  lines here ?

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Zsbán Ambrus]

On 1/16/15, Dave Dyer  wrote:
> I have a class of database for which using sqlite3 to create
> a copy via the "pipe" method fails.   Using an explicit intermediate
> file seems to work ok.
>
> I can supply a sample database to anyone interested in investigating.

Rather than the full database, can you show us the full schema of this
database, including triggers?

Does this database have any custom collations (mentioned after the
COLLATE keyword in the schema), or custom SQL functions mentioned in
the schema (such as in DEFAULT or CHECK constraint clauses)?
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Richard Hipp]

On 1/16/15, Dave Dyer  wrote:
>
> I have a class of database for which using sqlite3 to create
> a copy via the "pipe" method fails.   Using an explicit intermediate
> file seems to work ok.
>
> I can supply a sample database to anyone interested in investigating.

Is the database small enough to sent to me via email?

>
> --
>
> F:\2013 YearTech\Yearbook Tools\Resource>sqlite3 -version
> 3.7.3
>
> F:\2013 YearTech\Yearbook Tools\Resource>sqlite3 po.sqlite .dump | sqlite3
> po2.sqlite
> Error: incomplete SQL: INSERT INTO "imageblob" VALUES(1,'G:\share\e ...
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3 tool bug

[Dave Dyer]

I have a class of database for which using sqlite3 to create
a copy via the "pipe" method fails.   Using an explicit intermediate
file seems to work ok.

I can supply a sample database to anyone interested in investigating.

--

F:\2013 YearTech\Yearbook Tools\Resource>sqlite3 -version
3.7.3

F:\2013 YearTech\Yearbook Tools\Resource>sqlite3 po.sqlite .dump | sqlite3 
po2.sqlite
Error: incomplete SQL: INSERT INTO "imageblob" VALUES(1,'G:\share\e ...

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users