[I got an unexpected _"message's content type was not explicitly allowed"_ bounce message after my first email, so I'm re-sending it in plain text. I apologize in advance for the subsequent lack of formatting, and am sorry for the double post if the first one actually reached you.]
Hi, I've been experiencing an issue with SQLite's CLI tool on Windows 7, when used through a script or spawned process (not when used directly in the console). For a bit more context, this problem namely affects the way the *dblite* library ([https://github.com/WebReflection/dblite], an SQLite wrapper for Node.js which spawns the CLI tool behind the scenes) works. The problem lies in the fact that, as opposed to Linux where stderr is always unbuffered, Windows behaves differently depending on whether the output is an interactive console or a pipe: on an interactive console, stderr is unbuffered, but when piped it becomes buffered (which I personally think is overcomplicated, but meh, that's the way it is, although I don't know if it is a general rule on Windows, or specific to GCC's implementation, or whatever, but I don't think it matters here). This difference in I/O buffering happens to create a discrepancy in the behavior of the CLI tool, between on one side Linux and Windows' interactive console, and on the other side piped output on Windows. Here is a tiny piece of shell script (working on both Windows' cmd and Bash) that can be used to see the difference: (echo fail\; && echo .version) | sqlite3 2>&1 This is the non-piped version. Simply add `| more` at the end to get a piped output. The sections below show the result. Windows 7 x64: C:\>(echo fail\; && echo .version) | sqlite3 2>&1 Error: near line 1: near "fail": syntax error SQLite 3.8.7.4 2014-12-09 01:34:36 f66f7a17b78ba617acde90fc810107f34f1a1f2e C:\>(echo fail\; && echo .version) | sqlite3 2>&1 | more SQLite 3.8.7.4 2014-12-09 01:34:36 f66f7a17b78ba617acde90fc810107f34f1a1f2e Error: near line 1: near "fail": syntax error Note the swapped output lines: the error shows up after the version when piping the output. Lubuntu 14.10: gmal@VLubuntuGM:~$ (echo fail\; && echo .version) | sqlite3 2>&1 Error: near line 1: near "fail": syntax error SQLite 3.8.6 2014-08-15 11:46:33 9491ba7d738528f168657adb43a198238abde19e gmal@VLubuntuGM:~$ (echo fail\; && echo .version) | sqlite3 2>&1 | more Error: near line 1: near "fail": syntax error SQLite 3.8.6 2014-08-15 11:46:33 9491ba7d738528f168657adb43a198238abde19e All is fine here. (Unfortunately I don't have an OSX machine, so I don't know how it behaves.) This issue basically makes it impossible to use the CLI tool the way *dblite* does, i.e. a long-living child process with piped input and output. Indeed, the error messages come in too late, at seemingly random times (in fact, they show up whenever the system flushes the buffer, which is usually completely out of context). I am not sure this way of using the CLI tool is actually meant to be supported by SQLite, but I'm pretty sure this discrepancy between the different OSes is not there on purpose, which is the reason why I'm submitting this as a potential bug to you guys. Additional info: After quickly looking at the code (sorry if I'm breaking the _"tachyon modulation must be wrongly polarised"_ rule), I found out that SQLite's shell tool explicitly flushes stdout after each query but leaves stderr untouched, probably trusting it to be unbuffered. I tried to compile a modified version of shell.c (just adding a few calls to `fflush(stderr)` along those to `fflush(p->out)` ), and it appears to work like a charm so far. But there may be compatibility issues with other usages, so I'm not going to pretend this is an easy fix. We all know there is no such thing as an easy fix in projects with a large-enough user base. Thanks a lot for your patience and congratulations for the awesome work done on SQLite. I'm quite new to it but it really seems to be a GREAT piece of software. Guilhem _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
