Re: [sqlite] SQLite 3.6.17

[Alberto Simões]

On Thu, Sep 3, 2009 at 10:30 PM, Nicolas Williams
wrote:

> On Thu, Sep 03, 2009 at 03:27:38PM +0100, Simon Slavin wrote:
> > On 3 Sep 2009, at 1:38am, P Kishor wrote:
> > > well, I think the problem is with the sqlite3 command line tool.
> > I agree.  I just checked it with OS X 10.6, which comes with SQLite
> > version 3.6.12, and got the same problem: you can't type accented
> > characters into the sqlite3 tool.  So I guess the sqlite3 tool uses a
> > version of readline (or whatever) which doesn't do this properly.
> > However, the SQLite database system does handle unicode characters
> > just fine.
>
> It's almost certainly readline.  I tried SQLite3 on Solaris without
> readline and it handles non-ASCII UTF-8 input just fine.
>

It's all Mac readline fault. Compile a new one, and recompile sqlite with it
(check another e-mail mine about how to compile sqlite with a different
readline version).
It will work :)


-- 
Alberto Simões
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[Nicolas Williams]

On Thu, Sep 03, 2009 at 03:27:38PM +0100, Simon Slavin wrote:
> On 3 Sep 2009, at 1:38am, P Kishor wrote:
> > well, I think the problem is with the sqlite3 command line tool.
> I agree.  I just checked it with OS X 10.6, which comes with SQLite  
> version 3.6.12, and got the same problem: you can't type accented  
> characters into the sqlite3 tool.  So I guess the sqlite3 tool uses a  
> version of readline (or whatever) which doesn't do this properly.   
> However, the SQLite database system does handle unicode characters  
> just fine.

It's almost certainly readline.  I tried SQLite3 on Solaris without
readline and it handles non-ASCII UTF-8 input just fine.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[Simon Slavin]

On 3 Sep 2009, at 1:38am, P Kishor wrote:

> well, I think the problem is with the sqlite3 command line tool.

I agree.  I just checked it with OS X 10.6, which comes with SQLite  
version 3.6.12, and got the same problem: you can't type accented  
characters into the sqlite3 tool.  So I guess the sqlite3 tool uses a  
version of readline (or whatever) which doesn't do this properly.   
However, the SQLite database system does handle unicode characters  
just fine.

> I can
> type accented characters in the Terminal.app using bash and no fancy
> setting just fine.

Yes, OS X 10.6 seems to have a version of bash which reads the  
accented characters fine without any changes to the environment.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[P Kishor]

On Wed, Sep 2, 2009 at 6:14 PM, Simon Slavin wrote:
>
> On 2 Sep 2009, at 8:27pm, P Kishor wrote:
>
>> Simon
>> Slavin wrote:
>>>
>>> On 2 Sep 2009, at 2:39pm, Alberto Simões wrote:
>>>
 Can you please send me your env?
>>>
>>> It works fine for me on Leopard, and I have done no special
>>> environment setting at all: I use the default Unix settings and the
>>> default SQLite settings for 10.5.
>>>
>>> You might want to check which shell you're using (csh ?  bash ?) to
>>> see if the shell is filtering out your funny characters for you.
>>
>> actually, does not work for me.
>
> I apologise.  You're quite right: you can't type non-ASCII characters into
> the sqlite3 command-line tool.  What I'd done when I was testing was to put
> commands into a text file and use the '.read' command.  I just tested this:
> make a text file with the following in:
>
> CREATE TABLE myTab (myCol TEXT);
> INSERT INTO myTab (myCol) VALUES ('pub');
> INSERT INTO myTab (myCol) VALUES ('café');
>
> Then I did this:
>
> SimonsMBP2009:Desktop simon$ sqlite3 testchars.sql
> SQLite version 3.4.0
> Enter ".help" for instructions
> sqlite> .read commands.txt
> sqlite> SELECT * FROM myTab;
> pub
> café
> sqlite> .dump
> BEGIN TRANSACTION;
> CREATE TABLE myTab (myCol TEXT);
> INSERT INTO "myTab" VALUES('pub');
> INSERT INTO "myTab" VALUES('café');
> COMMIT;
> sqlite>
>
> As you can see, the accented character went in fine, and sqlite retained and
> displays it fine.  The problem is you just can't type such a character, or
> you can't using an unchanged bash shell.  The setting for the bash
> command-line would be
>
> set input-meta on
>
> This allows bash to handle the accented e correctly.  But it's not passed
> into the command-line tool that comes with OS X 10.5.  I'll be able to test
> this with the version of sqlite3 that comes with 10.6 soon.
>

well, I think the problem is with the sqlite3 command line tool. I can
type accented characters in the Terminal.app using bash and no fancy
setting just fine. See below

punk...@lucknow ~$I ate at el niño café çirca 1984 with ümlaut and hei∫e in øslo
-bash: I: command not found
punk...@lucknow ~$sqlite3
SQLite version 3.6.11
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> create table t1(x);
sqlite> insert into t1 values ('I ate at el nio caf irca 1984 with
mlaut e in Heislo');
sqlite> select * from t1;
I ate at el nio caf irca 1984 with mlaut e in Heislo
sqlite>


setting input-meta on doesn't change the above behavior at all.


> Simon.



-- 
Puneet Kishor http://www.punkish.org
Carbon Model http://carbonmodel.org
Charter Member, Open Source Geospatial Foundation http://www.osgeo.org
Science Commons Fellow, http://sciencecommons.org/about/whoweare/kishor
Nelson Institute, UW-Madison http://www.nelson.wisc.edu
---
Assertions are politics; backing up assertions with evidence is science
===
Sent from Madison, WI, United States
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[Simon Slavin]

On 2 Sep 2009, at 8:27pm, P Kishor wrote:

> Simon
> Slavin wrote:
>>
>> On 2 Sep 2009, at 2:39pm, Alberto Simões wrote:
>>
>>> Can you please send me your env?
>>
>> It works fine for me on Leopard, and I have done no special
>> environment setting at all: I use the default Unix settings and the
>> default SQLite settings for 10.5.
>>
>> You might want to check which shell you're using (csh ?  bash ?) to
>> see if the shell is filtering out your funny characters for you.
>
> actually, does not work for me.

I apologise.  You're quite right: you can't type non-ASCII characters  
into the sqlite3 command-line tool.  What I'd done when I was testing  
was to put commands into a text file and use the '.read' command.  I  
just tested this: make a text file with the following in:

CREATE TABLE myTab (myCol TEXT);
INSERT INTO myTab (myCol) VALUES ('pub');
INSERT INTO myTab (myCol) VALUES ('café');

Then I did this:

SimonsMBP2009:Desktop simon$ sqlite3 testchars.sql
SQLite version 3.4.0
Enter ".help" for instructions
sqlite> .read commands.txt
sqlite> SELECT * FROM myTab;
pub
café
sqlite> .dump
BEGIN TRANSACTION;
CREATE TABLE myTab (myCol TEXT);
INSERT INTO "myTab" VALUES('pub');
INSERT INTO "myTab" VALUES('café');
COMMIT;
sqlite>

As you can see, the accented character went in fine, and sqlite  
retained and displays it fine.  The problem is you just can't type  
such a character, or you can't using an unchanged bash shell.  The  
setting for the bash command-line would be

set input-meta on

This allows bash to handle the accented e correctly.  But it's not  
passed into the command-line tool that comes with OS X 10.5.  I'll be  
able to test this with the version of sqlite3 that comes with 10.6 soon.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[P Kishor]

2009/9/2 Alberto Simões :
> Simon, try to compile a gnu version of readline, and then compile sqlite
> againt it. It should do the trick.


actually, it is me, not Simon, who is experiencing this problem.
Before I jump into compiling a new readline lib, I want to ascertain
if that is what others (such as Simon or DRH) did who are *not* having
any such issues on the Mac.

>
> All the best,
> Alberto
>
> On Wed, Sep 2, 2009 at 8:27 PM, P Kishor  wrote:
>>
>> On Wed, Sep 2, 2009 at 10:55 AM, Simon
>> Slavin wrote:
>> >
>> > On 2 Sep 2009, at 2:39pm, Alberto Simões wrote:
>> >
>> >> Can you please send me your env?
>> >
>> > It works fine for me on Leopard, and I have done no special
>> > environment setting at all: I use the default Unix settings and the
>> > default SQLite settings for 10.5.
>> >
>> > You might want to check which shell you're using (csh ?  bash ?) to
>> > see if the shell is filtering out your funny characters for you.
>>
>>
>> actually, does not work for me. I am using bash, and while both in
>> Terminal.app and in iTerm.app, the character encoding is set to
>> Unicode (UTF-8), none of the accented characters work. When I type
>> 'café', for example, upon typing Option-e, I get the accent mark, but
>> when I type e, instead of getting é, I get nothing. It wipes the
>> accent, and the cursor doesn't move.
>>
>>
>> >
>> > Simon.
>> > --
>> >  http://www.hearsay.demon.co.uk | I'd expect if a computer was involved
>> >                                 | it all would have been much worse.
>> >    No Buffy for you.            |                -- John "West" McKenna
>> >    Leave quickly now. -- Anya   |          THE FRENCH WAS THERE
>> >
>> > ___
>> > sqlite-users mailing list
>> > sqlite-users@sqlite.org
>> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>> >
>>
>>
>>
>> --
>> Puneet Kishor http://www.punkish.org
>> Carbon Model http://carbonmodel.org
>> Charter Member, Open Source Geospatial Foundation http://www.osgeo.org
>> Science Commons Fellow, http://sciencecommons.org/about/whoweare/kishor
>> Nelson Institute, UW-Madison http://www.nelson.wisc.edu
>> ---
>> Assertions are politics; backing up assertions with evidence is science
>> ===
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
>
>
> --
> Alberto Simões
>
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[Alberto Simões]

Simon, try to compile a gnu version of readline, and then compile sqlite
againt it. It should do the trick.

All the best,
Alberto

On Wed, Sep 2, 2009 at 8:27 PM, P Kishor  wrote:

> On Wed, Sep 2, 2009 at 10:55 AM, Simon
> Slavin wrote:
> >
> > On 2 Sep 2009, at 2:39pm, Alberto Simões wrote:
> >
> >> Can you please send me your env?
> >
> > It works fine for me on Leopard, and I have done no special
> > environment setting at all: I use the default Unix settings and the
> > default SQLite settings for 10.5.
> >
> > You might want to check which shell you're using (csh ?  bash ?) to
> > see if the shell is filtering out your funny characters for you.
>
>
> actually, does not work for me. I am using bash, and while both in
> Terminal.app and in iTerm.app, the character encoding is set to
> Unicode (UTF-8), none of the accented characters work. When I type
> 'café', for example, upon typing Option-e, I get the accent mark, but
> when I type e, instead of getting é, I get nothing. It wipes the
> accent, and the cursor doesn't move.
>
>
> >
> > Simon.
> > --
> >  http://www.hearsay.demon.co.uk | I'd expect if a computer was involved
> > | it all would have been much worse.
> >No Buffy for you.|-- John "West" McKenna
> >Leave quickly now. -- Anya   |  THE FRENCH WAS THERE
> >
> > ___
> > sqlite-users mailing list
> > sqlite-users@sqlite.org
> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> >
>
>
>
> --
> Puneet Kishor http://www.punkish.org
> Carbon Model http://carbonmodel.org
> Charter Member, Open Source Geospatial Foundation http://www.osgeo.org
> Science Commons Fellow, http://sciencecommons.org/about/whoweare/kishor
> Nelson Institute, UW-Madison http://www.nelson.wisc.edu
> ---
> Assertions are politics; backing up assertions with evidence is science
> ===
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Alberto Simões
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[P Kishor]

On Wed, Sep 2, 2009 at 10:55 AM, Simon
Slavin wrote:
>
> On 2 Sep 2009, at 2:39pm, Alberto Simões wrote:
>
>> Can you please send me your env?
>
> It works fine for me on Leopard, and I have done no special
> environment setting at all: I use the default Unix settings and the
> default SQLite settings for 10.5.
>
> You might want to check which shell you're using (csh ?  bash ?) to
> see if the shell is filtering out your funny characters for you.


actually, does not work for me. I am using bash, and while both in
Terminal.app and in iTerm.app, the character encoding is set to
Unicode (UTF-8), none of the accented characters work. When I type
'café', for example, upon typing Option-e, I get the accent mark, but
when I type e, instead of getting é, I get nothing. It wipes the
accent, and the cursor doesn't move.


>
> Simon.
> --
>  http://www.hearsay.demon.co.uk | I'd expect if a computer was involved
>                                 | it all would have been much worse.
>    No Buffy for you.            |                -- John "West" McKenna
>    Leave quickly now. -- Anya   |          THE FRENCH WAS THERE
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Puneet Kishor http://www.punkish.org
Carbon Model http://carbonmodel.org
Charter Member, Open Source Geospatial Foundation http://www.osgeo.org
Science Commons Fellow, http://sciencecommons.org/about/whoweare/kishor
Nelson Institute, UW-Madison http://www.nelson.wisc.edu
---
Assertions are politics; backing up assertions with evidence is science
===
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[Simon Slavin]

On 2 Sep 2009, at 2:39pm, Alberto Simões wrote:

> Can you please send me your env?

It works fine for me on Leopard, and I have done no special  
environment setting at all: I use the default Unix settings and the  
default SQLite settings for 10.5.

You might want to check which shell you're using (csh ?  bash ?) to  
see if the shell is filtering out your funny characters for you.

Simon.
-- 
  http://www.hearsay.demon.co.uk | I'd expect if a computer was involved
 | it all would have been much worse.
No Buffy for you.|-- John "West" McKenna
Leave quickly now. -- Anya   |  THE FRENCH WAS THERE

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[Alberto Simões]

On Wed, Sep 2, 2009 at 2:52 PM, D. Richard Hipp  wrote:

>
> On Sep 2, 2009, at 9:46 AM, D. Richard Hipp wrote:
> >>>
> >>> It worked for me before. No idea what is happenning.
> >> Can you please send me your env?
> >
> >
>
>
> sqlite-imac:bld drh$ ./sqlite3
> SQLite version 3.6.17
> Enter ".help" for instructions
> Enter SQL statements terminated with a ";"
> sqlite> create table t1(x);
> sqlite> insert into t1 values('Alberto Simões ξ €  夷');
> sqlite> select * from t1;
> Alberto Simões ξ € 夷
> sqlite> select hex(x) from t1;
> 416C626572746F2053696DC3B5657320CEBE20E282AC20E5A4B7
> sqlite> sqlite-imac:bld drh$ env
> MANPATH=/usr/share/man:/usr/local/share/man:/usr/X11/man
> TERM_PROGRAM=Apple_Terminal
> TERM=xterm-color
> SHELL=/bin/bash
> TMPDIR=/var/folders/9H/9H+KCJI3F5K1xNlxD8BM7TI/-Tmp-/
> Apple_PubSub_Socket_Render=/tmp/launch-CFpjwW/Render
> TERM_PROGRAM_VERSION=240.2
> USER=drh
> COMMAND_MODE=unix2003
> SSH_AUTH_SOCK=/tmp/launch-FWX7Up/Listeners
> __CF_USER_TEXT_ENCODING=0x1F5:0:0
> PATH=/opt/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/
> X11/bin
> PWD=/Users/drh/sqlite/bld
> LANG=en_US.UTF-8
> SHLVL=1
> HOME=/Users/drh
> LOGNAME=drh
> VISUAL=e
> DISPLAY=/tmp/launch-GNoo4x/:0
> SECURITYSESSIONID=a8c400
> _=/usr/bin/env
> OLDPWD=/Users/drh/sqlite/sqlite
> sqlite-imac:bld drh$
>

Thank you. Nothing easy to detect.
I'll dig a little more on it.

-- 
Alberto Simões
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[D. Richard Hipp]

On Sep 2, 2009, at 9:46 AM, D. Richard Hipp wrote:
>>>
>>> It worked for me before. No idea what is happenning.
>> Can you please send me your env?
>
>


sqlite-imac:bld drh$ ./sqlite3
SQLite version 3.6.17
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> create table t1(x);
sqlite> insert into t1 values('Alberto Simões ξ €  夷');
sqlite> select * from t1;
Alberto Simões ξ € 夷
sqlite> select hex(x) from t1;
416C626572746F2053696DC3B5657320CEBE20E282AC20E5A4B7
sqlite> sqlite-imac:bld drh$ env
MANPATH=/usr/share/man:/usr/local/share/man:/usr/X11/man
TERM_PROGRAM=Apple_Terminal
TERM=xterm-color
SHELL=/bin/bash
TMPDIR=/var/folders/9H/9H+KCJI3F5K1xNlxD8BM7TI/-Tmp-/
Apple_PubSub_Socket_Render=/tmp/launch-CFpjwW/Render
TERM_PROGRAM_VERSION=240.2
USER=drh
COMMAND_MODE=unix2003
SSH_AUTH_SOCK=/tmp/launch-FWX7Up/Listeners
__CF_USER_TEXT_ENCODING=0x1F5:0:0
PATH=/opt/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/ 
X11/bin
PWD=/Users/drh/sqlite/bld
LANG=en_US.UTF-8
SHLVL=1
HOME=/Users/drh
LOGNAME=drh
VISUAL=e
DISPLAY=/tmp/launch-GNoo4x/:0
SECURITYSESSIONID=a8c400
_=/usr/bin/env
OLDPWD=/Users/drh/sqlite/sqlite
sqlite-imac:bld drh$

D. Richard Hipp
d...@hwaci.com



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[D. Richard Hipp]

On Sep 2, 2009, at 9:39 AM, Alberto Simões wrote:


Hello, drh.

On Wed, Sep 2, 2009 at 2:28 PM, D. Richard Hipp  wrote:





Works for me:

sqlite-imac:bld drh$ ./sqlite3
SQLite version 3.6.17
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> create table t1(x);
sqlite> insert into t1 values('Alberto Simões ξ €  夷');
sqlite> select * from t1;
Alberto Simões ξ € 夷
sqlite>

It worked for me before. No idea what is happenning.

Can you please send me your env?






Thank you
Alberto
--
Alberto Simões
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


D. Richard Hipp
d...@hwaci.com



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[Alberto Simões]

Hello, drh.

On Wed, Sep 2, 2009 at 2:28 PM, D. Richard Hipp  wrote:

>
>
>
> Works for me:
>
> sqlite-imac:bld drh$ ./sqlite3
> SQLite version 3.6.17
> Enter ".help" for instructions
> Enter SQL statements terminated with a ";"
> sqlite> create table t1(x);
> sqlite> insert into t1 values('Alberto Simões ξ €  夷');
> sqlite> select * from t1;
> Alberto Simões ξ € 夷
> sqlite>
>
> It worked for me before. No idea what is happenning.
Can you please send me your env?

Thank you
Alberto
-- 
Alberto Simões
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite 3.6.17

[D. Richard Hipp]

On Sep 2, 2009, at 9:06 AM, Alberto Simões wrote:

> Hellows
>
> Noticed today (with SQLite 3.6.17) that I can't insert non latin1  
> character
> using the sqlite3 shell.
> I am running under Mac OS Leopard (not yet the white one), and using  
> an
> unicode terminal.


Works for me:

sqlite-imac:bld drh$ ./sqlite3
SQLite version 3.6.17
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> create table t1(x);
sqlite> insert into t1 values('Alberto Simões ξ €  夷');
sqlite> select * from t1;
Alberto Simões ξ € 夷
sqlite>


D. Richard Hipp
d...@hwaci.com



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users