[issue4815] idle 3.1a1 utf8

2009-01-18 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

Committed as r68732, r68733.

--
resolution:  - fixed
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-04 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

 utf8 has a special flag

No, it doesn't.

 as a special mark for editors.

That's the BOM, or UTF-8 signature. It's optional, and UTF-8-encoded
files typically do *not* have the UTF-8 signature.

 Even after making change in 
 it in IDLE, it does not became real utf8. 

Your understanding of UTF-8 is incorrect.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-04 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

With this file  - hello.py (attached) - I should be also asked for
converting to utf8. When I open it, nothing changes, after making
changes and saving then the encodings is my windows standard cp1250 

Added file: http://bugs.python.org/file12582/hello.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-04 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

 With this file  - hello.py (attached) - I should be also asked for
 converting to utf8.

Why that? This file is already encoded in utf-8 just fine. It is,
simultaneously, also encoded in ASCII, cp1250, cp1252, and nearly
any other encoding in use (as long as it is ASCII-based).

 When I open it, nothing changes, after making
 changes and saving then the encodings is my windows standard cp1250 

What did you do to find that out?

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

You can open script made in python 2.x and it stops immediately working
after saving, if it is coding-aware. You can have bigger project and use
idle for editing config and text files from this project too. It is
unfair to change without notification the encodings. Or do you
consider IDLE just for beginners for learning?

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

I forgot about Perhaps IDLE should offer to convert it on opening.
That would be nice, too.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

Here is a patch to provide an explicit message that the file will be
converted when the file is opened (also querying what encoding should be
converted from), answering the complaint that the conversion is without
notice.

If you want to edit Python 2.x scripts with IDLE 3, you need to add
encoding declarations to the files. For bigger projects, it is
reasonable to expect that they do add these declarations.

Again, please only report one issue at the time. You chose to make
Python source files the topic of this issue, so please don't divert into
config or text files.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

Sorry, where is the patch?

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Martin v. Löwis

Changes by Martin v. Löwis mar...@v.loewis.de:


--
keywords: +patch
Added file: http://bugs.python.org/file12561/conv.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Martin v. Löwis

Changes by Martin v. Löwis mar...@v.loewis.de:


--
keywords: +needs review
priority:  - release blocker

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

OK, I got it. 

In my opinion it would nice if user can either convert file to utf8 or
to do nothing and add new encodings declaration or cancel. Current
Cancel gives an Decoding error. If you give an encodings that doesn't
exist, it shouldn't  destroy IDLE. Hoping its not my mistake, cause I do
not have all files from 3.1a  - just those from idlelib.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

 In my opinion it would nice if user can either convert file to utf8 or
 to do nothing and add new encodings declaration or cancel.

Ypu can still add an encoding declaration after the file got converted.
Cancelling is also possible.

 If you give an encodings that doesn't exist, it shouldn't  destroy IDLE.

Right. Here is a patch that fixes that.

Added file: http://bugs.python.org/file12566/conv.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Martin v. Löwis

Changes by Martin v. Löwis mar...@v.loewis.de:


Removed file: http://bugs.python.org/file12561/conv.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

seems to be working.

Seems to me now I get it. The file encoding is ruled by the encoding
declaration. When I stated 

# -*- coding: cp1250 -*-

then the file would be saved in cp1250. 

Now hoping that I would keep this issue, cause it comes with this
patches: when I open file *with* say # -*- coding: cp1250 -*-, I am
asked to change to utf8. This behaviour was not before and is probably
unwanted.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

 Now hoping that I would keep this issue, cause it comes with this
 patches: when I open file *with* say # -*- coding: cp1250 -*-, I am
 asked to change to utf8. This behaviour was not before and is probably
 unwanted.

Actually, the behavior was there before - it's just that conversion was
silent. If you put an empty line after the coding declaration, it should
work well. issue 4008 contains a patch for that problem.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

Well, thanks a lot. 

(aware this is really off this issue): Now I even get the system of
patches  - issue 4008 solved the inconvenience in print Unicode signs
inside IDLE. Still not sure how works patches for Python versions. I
vote for including this a that patch about IDLE even in some 3.0.1, not
only in branch 3.1. In non-English speaking countries troubles with
encodings are more common, more beginners-frustrating than you can believe.

HNY 2009
Pavel Kosina

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

 I
 vote for including this a that patch about IDLE even in some 3.0.1, not
 only in branch 3.1.

This is my plan, yes - hence I marked them all release-critical. They
still need review. I agree that IDLE in 3.0 is fairly broken wrt.
non-ASCII characters. I knew that before the release, but there is so
much work and only so little time. FWIW, my own language (German) also
uses (some) non-ASCII characters.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

I might have another problem with this patch and maybe also that one in
issue 4008. Having a file with

print (ěščřžýáíé)
# saved in cp1250

Open - confirm converting to utf8 - F5 - error: see attached file
idleunicode1.jpg

Added file: http://bugs.python.org/file12576/idleunicode1.jpg

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

 print (ěščřžýáíé)
 # saved in cp1250

I can't reproduce the problem. Can you please attach the
exact file that failed to work?

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

Martin v. Löwis napsal(a), dne 3.1.2009 22:24:
 I can't reproduce the problem. Can you please attach the
 exact file that failed to work?
   

You can use that one that is already here: cp1250.py. It is the same 
error with me.

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

 You can use that one that is already here: cp1250.py. It is the same 
 error with me.

Ok, then what are the exact steps to reproduce? What code base, what
patches applied, what user interaction in what order?

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

Microsoft Windows XP [Verze 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\prg\Python30\Lib\idlelibsvn update# from 
http://svn.python.org/projects/python/branches/py3k/Lib/idlelib
Restored 'AutoCompleteWindow.py'
Restored 'ToolTip.py'
Restored 'UndoDelegator.py'
Restored 'Bindings.py'
Restored '__init__.py'
Restored 'AutoComplete.py'
Restored 'configHandler.py'
Restored 'HyperParser.py'
Restored 'ColorDelegator.py'
Restored 'Delegator.py'
Restored 'ObjectBrowser.py'
Restored 'testcode.py'
Restored 'configSectionNameDialog.py'
Restored 'ZoomHeight.py'
Restored 'PyShell.py'
Restored 'ParenMatch.py'
Restored 'config-keys.def'
Restored 'Debugger.py'
Restored 'CREDITS.txt'
Restored 'configDialog.py'
Restored 'StackViewer.py'
Restored 'HISTORY.txt'
Restored 'SearchEngine.py'
Restored 'ReplaceDialog.py'
Restored 'ScriptBinding.py'
Restored 'ChangeLog'
Restored 'tabbedpages.py'
Restored 'keybindingDialog.py'
Restored 'configHelpSourceEdit.py'
Restored 'WidgetRedirector.py'
Restored 'GrepDialog.py'
Restored 'FormatParagraph.py'
Restored 'EditorWindow.py'
Restored 'help.txt'
Restored 'config-highlight.def'
Restored 'PyParse.py'
Restored 'README.txt'
Restored 'rpc.py'
Restored 'OutputWindow.py'
Restored 'aboutDialog.py'
Restored 'idle.bat'
Restored 'TODO.txt'
Restored 'config-main.def'
Restored 'IdleHistory.py'
Restored 'PathBrowser.py'
Restored 'IOBinding.py'
Restored 'WindowList.py'
Restored 'ScrolledList.py'
Restored 'ClassBrowser.py'
Restored 'FileList.py'
Restored 'CallTips.py'
Restored 'idle.py'
Restored 'CodeContext.py'
Restored 'textView.py'
Restored 'SearchDialogBase.py'
Restored 'CallTipWindow.py'
Restored 'SearchDialog.py'
Restored 'RemoteObjectBrowser.py'
Restored 'idlever.py'
Restored 'RemoteDebugger.py'
Restored 'TreeWidget.py'
Restored 'NEWS.txt'
Restored 'idle.pyw'
Restored 'run.py'
Restored 'config-extensions.def'
Restored 'AutoExpand.py'
Restored 'Percolator.py'
Restored 'dynOptionMenuWidget.py'
Restored 'extend.txt'
Restored 'MultiStatusBar.py'
Restored 'MultiCall.py'
Restored 'macosxSupport.py'
At revision 68230.

C:\prg\Python30\Lib\idlelibpatch  conv.diff
patching file IOBinding.py

C:\prg\Python30\Lib\idlelibpatch  idle_encoding_4.patch
patching file ScriptBinding.py
patching file IOBinding.py

-

Run IDLE - Open cp1250.py - confirm converting to utf8 - F5 (immediately, no 
change in code!) - error: see attached file
idleunicode1.jpg

All the other python code base is clean 3.0.
xpsp3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

I see. Here is a revised patch. This sets the modified flag on the
buffer after conversion, so that you get asked to save it before running it.

Added file: http://bugs.python.org/file12578/conv.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-03 Thread Pavel Kosina

Pavel Kosina g...@post.cz added the comment:

Yes. God job. ;-)

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-02 Thread Pavel Kosina

New submission from Pavel Kosina g...@post.cz:

When you open file without encoding declaration, make changes and save,
then IDLE changes without any question encodings to utf8. You can try it
on attached file that is cp1250 now. 

It could be that at first saving we are asked to use 
*utf8 
*current one.

--
components: IDLE
files: cp1250.py
messages: 78932
nosy: geon
severity: normal
status: open
title: idle 3.1a1 utf8
versions: Python 3.1
Added file: http://bugs.python.org/file12559/cp1250.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue4815] idle 3.1a1 utf8

2009-01-02 Thread Martin v. Löwis

Martin v. Löwis mar...@v.loewis.de added the comment:

IDLE is right to save the file as UTF-8; the file is invalid Python 3.0
code. In Python 3.0, the source encoding *is* UTF-8; nothing else is
allowed unless you have an encoding declaration.

Perhaps IDLE should offer to convert it on opening.

--
nosy: +loewis

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4815
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com