Re: ncurses getch unicode (was: decoding keyboard input when using curses)
On Aug 20, 6:12 pm, Iñigo Serna inigose...@gmail.com wrote: Hi again, 2009/8/20 Iñigo Serna inigose...@gmail.com I have the same problem mentioned inhttp://groups.google.com/group/comp.lang.python/browse_thread/thread/...some months ago. Python 2.6 program which usesncursesmodule in a terminal configured to use UTF-8 encoding. When trying to get input from keyboard, a non-ascii character (like ç) is returned as 2 integers 255, needing 2 calls to getch method to get both. These two integers \xc3 \xa7 forms the utf-8 encoded representation of ç character. ncursesget_wch documentation states the function should return an unique integer 255 with the ordinal representation of that unicode char encoded in UTF-8, \xc3a7. Answering myself, I've copied at the bottom of this email a working solution, but the question still remains: why win.getch() doesn't return the correct value? The code looks consistent with the curses functions... Kind regards, Iñigo Serna ## # test.py import curses import locale locale.setlocale(locale.LC_ALL, '') print locale.getpreferredencoding() def get_char(win): def get_check_next_byte(): c = win.getch() You're using getch, not get_wch (Python's ncurses binding may/may not have the latter). curses getch returns 8-bit values, get_wch would return wider values. -- http://mail.python.org/mailman/listinfo/python-list
Re: ncurses getch unicode (was: decoding keyboard input when using curses)
On Fri, 21 Aug 2009, Iñigo Serna wrote: 2009/8/21 Thomas Dickey dic...@his.com: On Aug 20, 6:12 pm, Iñigo Serna inigose...@gmail.com wrote: c = win.getch() You're using getch, not get_wch (Python's ncurses binding may/may not have the latter). curses getch returns 8-bit values, get_wch would return wider values. you are right, ncurses binding does not have get_wch, only getch, and this last is the only one called in ncurses library bindings. Anyway, I've written a patch to include the get_wch method in the bindings. See http://bugs.python.org/issue6755 Thanks for the clarification, no problem (report bugs) Iñigo -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net-- http://mail.python.org/mailman/listinfo/python-list
Re: web page text extractor
Miki [EMAIL PROTECTED] wrote: (You can find lynx at http://lynx.browser.org/) not exactly - The current version of lynx is 2.8.6 It's available at http://lynx.isc.org/lynx2.8.6/ 2.8.7 Development patches: http://lynx.isc.org/current/index.html -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Curses and resizing windows
Nick ! [EMAIL PROTECTED] wrote: http://web.cs.mun.ca/~rod/ncurses/ncurses.html#xterm says The ncurses library does not catch [the SIGWINCH aka resizing] signal, because it cannot in general know how you want the screen re-painted. First, is this really true? When I make my xterms smaller they clip what is no - given that particular url is a file dating from 1995, it's something that I overlooked in making corrections here: http://invisible-island.net/ncurses/ncurses-intro.html But also note where I link it from: http://invisible-island.net/ncurses/ncurses.faq.html#additional_reading displayed--is that a function of curses or the xterm? both - xterm sends the signal, and curses receives it. Second, if true, it explains /what/ is going on--stdscr, only--, but isn't really satisfactory; it doesn't solve the original poster's bug. #!/usr/bin/env python curses_resize.py - run the program without hooking SIGWINCH curses_resize.py 1 - run the program with hooking SIGWINCH import sys, curses, signal, time def sigwinch_handler(n, frame): curses.endwin() curses.initscr() Actually I'd expect to see curses.refresh() here - but that may be a quirk of the python code. In general, ncurses passes a KEY_RESIZE via the getch() call that the application (such as python) should process. If it's not reading from getch(), ncurses' repainting/resizing won't happen. Also, if you add your own signal handler, ncurses' SIGWINCH catcher won't work. (I'd recommend chaining the signal handlers, but don't know if python supports that ;-). def main(stdscr): just repeatedly redraw a long string to reveal the window boundaries while 1: stdscr.insstr(0,0,abcd*40) time.sleep(1) if __name__=='__main__': if len(sys.argv)==2 and sys.argv[1]==1: signal.signal(signal.SIGWINCH, sigwinch_handler) curses.wrapper(main) -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python+ncurses: I can't display accents
Fabrice DELENTE [EMAIL PROTECTED] wrote: My system is Linux, and the distribution is Slackware 10.1. I have /lib/libncurses.so.5.4 /lib/libncursesw.so.5.4 so I even have the wide-chars version available. Any hint on the python configuration? I didn't find any function that would allow the unrestricted display of 8-bit chars. It's more complicated than that: python's loading ncurses dynamically. It doesn't matter much (to python) which one it loads, but it's specifying the library name explicitly. At the same time, some other packages (such as readline) are _separately_ loading the ncurses library - and they also specify a library name. Rather than abstracting that stuff out to another (more easily changed) level, it's embedded in the code. If the initialization script is changed to load ncursesw then python can use ncursesw without further change. There are a few patches to the configuration that I've seen mentioned in the bug reports to enable python to do this. Those are patches to python of course... (this was a topic of discussion on this newsgroup about a year ago). -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python+ncurses: I can't display accents
Neil Cerutti [EMAIL PROTECTED] wrote: I don't really expect it to work, but if anything will, that is it. Curses supports only ASCII and a some special symbol codes defined by curses. un - no. Curses supports whatever the flavor of curses you have does. Often that's the 8-bit flavor of ncurses, but the limitation is definitely in the python configuration. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: curses problem reading cursor keys
Simon Morgan [EMAIL PROTECTED] wrote: I'd also appreciate any pointers to good tutorials on curses, I've read the one by awk and esr but found it rather brief and lacking in detail. esr only contributed his name - awk wrote the rest. (When I asked why, he only said it sounded like a good idea ;-) -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: need all python dialog equivalent
Eric S. Johansson [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: dialog binary is 110 KB. Won't it fit ? missing library. I have ncurses and newt and dialog seems to require something called ncursesw. I've been trying to find the Python newt seems to require if you're installing some package. dialog builds/works against either ncurses/ncursesw -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Need pixie dust for building Python 2.4 curses module on Solaris 8
[EMAIL PROTECTED] wrote: I'm having no success building the curses module on Solaris 8 (yes, I know it's ancient - advancing the state-of-the-art is not yet an option) for Python 2.4. Sun provides an apparently ancient version of curses in /usr/lib, so I downloaded and installed ncurses 5.5, both using default settings and using --with-shared. When the curses module is linked against for Solaris, you also should add --enable-rpath -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Urwid 0.9.0 - Console UI library
Ian Ward [EMAIL PROTECTED] wrote: - New raw_display module that handles console display without relying on external libraries. This module was written as a work around for the lack of UTF-8 support in the standard version of ncurses. The standard version of ncurses has supported UTF-8 for the past few years. You may perhaps mean default configuration, which has a different connotation. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Replacing curses
Ian Ward [EMAIL PROTECTED] wrote: I'm going to look at the Mined text editor for some terminal behavior mined_2000 (there's more than one program named mined, and the other doesn't do UTF-8). detection code. Mined is able to produce good UTF-8 output on a variety of terminals, and it links agains ncurses, not ncursesw... Interesting. It's probably using termcap (and the wide-character functions declared in wchar.h). -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with curses and UTF-8
Martin v. Löwis [EMAIL PROTECTED] wrote: Ian Ward wrote: Any Ideas? I think there is one or more ncurses bugs somewhere. indeed. It might be nice to report them rather than jawing about it. The ncurses documentation suggests that you should link with ncurses_w instead of linking with ncurses - you might try that as well. If it helps, please do report back. ncursesw Ultimately, somebody will need to debug ncurses to find out what precisely happens, and why. no need for debugging - it's a well-known problem. UTF-8 uses more than one byte per cell, normal curses uses one byte per cell. To handle UTF-8, you need ncursesw. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Replacing curses
Ian Ward [EMAIL PROTECTED] wrote: I've had to work around many curses issues while developing Urwid (a hmm - I've read Urwid, and most of the comments I've read in that regard reflect problems in Urwid. Perhaps it's time for you to do a little analysis. (looking forward to bug reports, rather than line noise) -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with curses and UTF-8
Martin v. Löwis [EMAIL PROTECTED] wrote: I'll test it if someone would dumb down link with ncursesw instead of ncurses a little for me. I tried: ./configure --with-libs=ncursesw5 and it failed saying: checking size of wchar_t... configure: error: cannot compute sizeof (wchar_t), 77 If that was Python's configure: don't do that. Instead, hack setup.py yes - python's configure script needs a lot of work (alternatively, it is not the sort of script I would write). to make it change the compiler/linker settings, or even edit the compiler/linker line manually at first. that works -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with curses and UTF-8
Ian Ward [EMAIL PROTECTED] wrote:
Martin v. Löwis wrote:
If that was Python's configure: don't do that. Instead, hack setup.py
to make it change the compiler/linker settings, or even edit the
compiler/linker line manually at first.
Ok, that compiled.
same here - though it was not immediately not clear which copy of ncurses it's
using (not the shared libraries since I installed those with tracing - a
little odd for it to use the static library, but that's what the access time
tells me).
To check on that (since I wanted to read the ncurses trace),
I ran strace and ltrace to look for clues.
Now when I run the same test:
import curses
s = curses.initscr()
s.addstr('\xc3\x85 U+00C5 LATIN CAPITAL LETTER A WITH RING ABOVE\n')
s.addstr('\xc3\xa5 U+00F5 LATIN SMALL LETTER O WITH TILDE')
s.refresh()
s.getstr()
curses.endwin()
Testing this, and looking to see what's going on, I notice that python
is doing a
setlocale(LC_ALL, C);
before the addstr is actually called. (ncurses never sets the locale;
it calls setlocale in one place to ask what it is).
That makes ncurses think it's not really doing UTF-8, of course. What I
see on the screen is the U+00C5 comes out with a box and a ~E (the
latter being ncurses' representation in POSIX for \0x85).
This is what I see:
+00C5 LATIN CAPITAL LETTER A WITH RING ABOVE
+00F5 LATIN SMALL LETTER O WITH TILDE
so, the UTF-8 characters didn't appear and the U at the beginning
became just .
well - running in uxterm I see the second line properly. But some more
tinkering is needed to make python work properly.
--
Thomas E. Dickey
http://invisible-island.net
ftp://invisible-island.net
--
http://mail.python.org/mailman/listinfo/python-list
Re: Replacing curses
Grant Edwards [EMAIL PROTECTED] wrote: Depending on what you're tring to do, slang might be an option, perhaps not - he's trying to use UTF-8. I haven't seen any plausible comment that indicates John Davis is interested in updating newt to work with slang2 (though of course he's welcome to show the code ;-) -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Replacing curses
Ian Ward [EMAIL PROTECTED] wrote: Grant Edwards wrote: Depending on what you're tring to do, slang might be an option, I've looked at newt and snack, but all I really need is: - a way to position the cursor at (0,0) - a way to hide and show the cursor - a way to detect when the terminal is resized - a way to query the terminal size ...and send UTF-8 text, keeping track of where you really are on the screen. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Replacing curses
Ian Ward [EMAIL PROTECTED] wrote: Thomas Dickey wrote: hmm - I've read Urwid, and most of the comments I've read in that regard reflect problems in Urwid. Perhaps it's time for you to do a little analysis. (looking forward to bug reports, rather than line noise) A fair request. My appologies for the inflammatory subject :-) When trying to check for user input without waiting I use code like: window_object.nodelay(1) curses.cbreak() input = window_object.getch() Occasionally (hard to reproduce reliably) the cbreak() call will raise an exception, but if I call it a second time before calling getch the code will work properly. This problem might be related to a signal interrupting the function call, I'm not sure. perhaps a more complete test-case would let me test it and see. Also, screen resizing only seems to be reported once by getch() even if the user continues to resize the window. I have worked around this by calling curses.doupdate() between calls to getch(). Maybe this is by design? Or perhaps it's some interaction with python - I don't know. The applications that I use with resizing (and ncurses' test programs) work smoothly enough. Finally, the curses escape sequence detection could be broadened. The top part of the curses_display module in Urwid defines many escape sequences I've run into that curses doesn't detect. That's data (terminfo). ncurses is data-driven, doesn't detect features of the terminal (though it does of course use environment variables for locale, etc.). xterm's terminfo lists a lot of function keys, for instance. The limit for predefined function-key names for terminfo is 60, but ncurses can accept extended terminfo descriptions (but I like to limit the length and style of names so it's possible to access them from termcap). One could define names like shift_f1, but then termcap applications couldn't see them. (The last I knew, slang doesn't either, but that's a different thread). That's been true for about 6 years. Current xterm's terminfo includes these names which apply to your comment: The ones on the end are extended names that ncurses' tic deduces from the terminfo file when it compiles it: comparing xterm-new to xterm-xf86-v44. comparing booleans. comparing numbers. comparing strings. kf49: '\EO3P', NULL. kf50: '\EO3Q', NULL. kf51: '\EO3R', NULL. kf52: '\EO3S', NULL. kf53: '\E[15;3~', NULL. kf54: '\E[17;3~', NULL. kf55: '\E[18;3~', NULL. kf56: '\E[19;3~', NULL. kf57: '\E[20;3~', NULL. kf58: '\E[21;3~', NULL. kf59: '\E[23;3~', NULL. kf60: '\E[24;3~', NULL. kf61: '\EO4P', NULL. kf62: '\EO4Q', NULL. kf63: '\EO4R', NULL. kind: '\E[1;2B', NULL. kri: '\E[1;2A', NULL. kDN: '\E[1;2B', NULL. kDN5: '\E[1;5B', NULL. kDN6: '\E[1;6B', NULL. kLFT5: '\E[1;5D', NULL. kLFT6: '\E[1;6D', NULL. kRIT5: '\E[1;5C', NULL. kRIT6: '\E[1;6C', NULL. kUP: '\E[1;2A', NULL. kUP5: '\E[1;5A', NULL. kUP6: '\E[1;6A', NULL. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Is the Python binding for ncurses unicode capable?
Martin v. Löwis [EMAIL PROTECTED] wrote: Thomas Dickey wrote: ncurses expects byte strings (although I'm uncertain as to what impact multi-byte encodings have in ncurses). It depends on whether python's curses binding is linked with the wide-character flavor (ncursesw) or the normal one. If it's linked with ncursesw, strings are interpreted according to the locale settings. Are you sure? According to ncurses' INSTALL file, the purpose of ncursesw is to provide wide character support, not locale support. It's two libraries just for binary compatibility issues. Well, of course I'm _sure_. But my explanation may be poor. Both libraries respond to locale. But ncurses only deals in single-byte encodings, e.g., ISO-8859-1 through ISO-8859-15. ncursesw supports that, but adds support for multi-byte encodings, e.g., UTF-8. For the latter, one can also have characters that combine (a printable character combined with other characters that overlay it or combine to form a new character). ncurses stores the character data in one byte per cell. ncursesw requires more than one byte per cell. That's the reason for two libraries. But since ncurses (not ncursesw) doesn't know about multibyte encoding, it can't do anything related to the locale for those. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Is the Python binding for ncurses unicode capable?
Martin v. Löwis [EMAIL PROTECTED] wrote: ncurses expects byte strings (although I'm uncertain as to what impact multi-byte encodings have in ncurses). It depends on whether python's curses binding is linked with the wide-character flavor (ncursesw) or the normal one. If it's linked with ncursesw, strings are interpreted according to the locale settings. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Python and curses
linuxfreak [EMAIL PROTECTED] wrote: Was wanting to write a text based application in python seems curses module is the way to go... anyone knows of any good tutorials apart from the one written by esr It wasn't written by esr. He only has his name on it, did none of the work. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Converting HTML to ASCII
Grant Edwards [EMAIL PROTECTED] wrote: First, make it work. Then make it work right. Then worry about how fast it is. Premature optimization... That could be - but then again, most of the comments I've seen for that particular issue are for rather old releases. It seems to use a quadratic algorithm for remembering where the links point, or something. I wrote a very crude but very fast renderer in C that I can post if someone wants it, which is what I use for this purpose. If lynx really is too slow, try w3m or links. Both do a better job of rendering anyway. They lay out tables more/less as expected (though navigation in tables for links seems to be an afterthought). -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net -- http://mail.python.org/mailman/listinfo/python-list
