[issue6822] Error calling .storlines from ftplib

[Roundup Robot]

Roundup Robot added the comment:

New changeset c78dfc6ce37a by Victor Stinner in branch '3.3':
Close #6822: ftplib.FTP.storlines() expects a binary file, not a text file
http://hg.python.org/cpython/rev/c78dfc6ce37a

New changeset 9328e2b8a397 by Victor Stinner in branch 'default':
(Merge 3.3) Close #6822: ftplib.FTP.storlines() expects a binary file, not a 
text file
http://hg.python.org/cpython/rev/9328e2b8a397

--
nosy: +python-dev
resolution:  -> fixed
stage:  -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Giampaolo Rodola']

Giampaolo Rodola' added the comment:

Apparently the doc has already been changed and mentions that a binary file is 
necessary. The unit test is legitimate and can be committed though (Victor, you 
want to do it?).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Mark Lawrence]

Mark Lawrence added the comment:

ftplib_doc.patch is so simple I'm assuming this has slipped under the radar, so 
would someone like to do the honours and commit the patch please.

--
nosy: +BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[STINNER Victor]

STINNER Victor  added the comment:

pitrou> The FTP class is not supposed to guess in which charset 
pitrou> your data should be encoded.
pitrou> (the "encoding" argument on the FTP class is meant 
pitrou> for protocol commands (such as file names), not for file
pitrou> contents)

I agree, my patch is completly wrong :-) You should open your file in binary 
mode, not in text (unicode) mode. The issue is more a documentation bug, than a 
bug in ftplib behaviour.

Here is a new patch: very short patch on the doc (just add "binary" to "open 
binary file object *file*").

--
components: +Documentation
Added file: http://bugs.python.org/file18663/ftplib_doc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> But I can tell you that I've never had another FTP program fail like
> Python 3 is failing for me trying to what seems like a simple transfer
> of a text file (expecting the line endings to be adjusted).  

Python 3 is not failing, you are just using it the wrong way. Open your
file in binary mode (as opposed to text mode) and it should work. "Text
files" in Python 3 are files decoded to unicode while reading (and
encoded from unicode while writing); they are much more than simply
binary files with adaptive line endings. This is more or less documented
here (although I agree it would perhaps need a better introductory
paragraph):
http://docs.python.org/py3k/library/io.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Ayman]

Ayman  added the comment:

My other 2 cents worth.

Actually that is my main issue.  Things used to work in 2.x, and
"suddenly" refused to work after my 3.x upgrade.

I'm not saying it worked means it is correct, but the Exception being
thrown does not look right.  I think we should always have core
libraries work out of the box, and have workarounds even if it's not
"our" error.

 Ayman

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[danonymous]

danonymous  added the comment:

One other observation on this issue.

I just 'backleveled' the python on my PC from 3.x to 2.7.  My test script works 
fine on 2.7.

- Dan

--
versions: +Python 3.1, Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[d nazario]

d nazario  added the comment:

There seems to be a lot of confusion on this thread.  So I'll add mine and 
hopefully clear things up a bit.In short I think aymanhs has it right.

This is what I get when trying to send a simple text file on a Windows Xp PC 
using python 3.x
-
ftp.storlines('STOR ' + localFile, fd)  # Send the file by 
issuing the STOR command.
  File "C:\Python30\lib\ftplib.py", line 477, in storlines
if buf[-1] in B_CRLF: buf = buf[:-1]
TypeError: Type str doesn't support the buffer API

-

getting up on my soap box  :-)

FTP has two modes binary and ascii.  When doing binary you expect your data to 
be sent as is with NO changes.  When doing ascii, you expect to send a text 
file with your line endings to be adjusted if they are not the same on the two 
platforms involved in the FTP transfer (MAC, Linux, PC, etc.).  

So I do think it's an bug.  Every other FTP program I've used, and it's a long 
list, has two distinct modes ASCII and Binary.  When using ASCII, the line 
endings are translated.  I don't know enough to comment on whether the ascii 
code for every character is supposed to be 127 or less.  But I can tell you 
that I've never had another FTP program fail like Python 3 is failing for me 
trying to what seems like a simple transfer of a text file (expecting the line 
endings to be adjusted).  

I think the goal here should be (must be) to make it work like it is expected 
to work, based on the expectations of millions of people who have been using 
FTP programs which have come before python for several decades.  The other 
finer points are really not important.

stepping off the soap box.

--
nosy: +danonymous

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[d nazario]

Changes by d nazario :


--
versions:  -Python 3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

Just my 2 cents to add that FTP ASCII mode should consists in nothing but 
replacing os.sep in "\r\n" before sending file data.
The opposite operation must be done by the receiving peer which has to read the 
data from the socket and replace "\r\n" with os.sep.
That should be all: no further conversion should be done against the data.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> I do not agree with Antoine.  For binary transfer, another method is
> called, and it does work fine.  However, storelines would be called
> for ASCII mode, in which encoding and decoding should be done by the
> FTP program.  ASCII mode would translate files, so a file loaded on a
> PC may not be identical to the same file when loaded on a Mac.

Well if it's really "ASCII" mode, it should refuse any character > 127,
in which case I would agree to hardcode ascii as an encoding.

But, in any case, storing "lines" has a meaning in binary mode, as
witnessed by the fact that binary files in Python 3 have a readline()
method. This means storlines should still succeed for binary input, even
if it is made compatible with text input.

I would personally argue that the text transfer mode in FTP is an
ill-conceived feature from the start and ftplib should do whatever is
reasonable nowadays, rather than what looked reasonable 30 years ago.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Ayman]

Ayman  added the comment:

I do not agree with Antoine.  For binary transfer, another method is called, 
and it does work fine.  However, storelines would be called for ASCII mode, in 
which encoding and decoding should be done by the FTP program.  ASCII mode 
would translate files, so a file loaded on a PC may not be identical to the 
same file when loaded on a Mac.

--
nosy: +aymanhs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

I agree with Antoine. Maybe it would make sense to check the file object passed 
to storlines() and raise an exception if mode != binary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Antoine Pitrou]

Antoine Pitrou  added the comment:

To me this isn't a bug, and the patch introduces incorrect behaviour. If you 
want to store data on an FTP server, you have to provide binary data, not text 
data. The FTP class is not supposed to guess in which charset your data should 
be encoded.

(the "encoding" argument on the FTP class is meant for protocol commands (such 
as file names), not for file contents)

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

Attached is a patch which fixes FTP_TLS class as well and changes the test 
server a little bit.
Shouldn't retrlines() suffer the same issue?

--
assignee:  -> giampaolo.rodola
nosy: +giampaolo.rodola
versions: +Python 3.1, Python 3.2 -Python 3.0
Added file: http://bugs.python.org/file17170/ftplib.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file14866/ftplib.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[STINNER Victor]

STINNER Victor  added the comment:

@amaury.forgeotdarc: Oops, no, it doesn't.

The new patch includes a new test (for the unicode file).

--
Added file: http://bugs.python.org/file14867/ftplib-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

Does this still pass the test suite? in test_ftplib, storlines() is given 
a BytesIO object.

--
nosy: +amaury.forgeotdarc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[STINNER Victor]

STINNER Victor  added the comment:

Oh yes, FTP.storlines() fails if the file is a text file. Here is a
patch. My patch encodes each line with self.encoding, which is latin1 by
default.

--
keywords: +patch
nosy: +haypo
Added file: http://bugs.python.org/file14866/ftplib.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Benjamin Peterson]

Changes by Benjamin Peterson :


--
type: compile error -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6822] Error calling .storlines from ftplib

[Robert DeVaughn]

New submission from Robert DeVaughn :

When attempting to store a file via FTP, the following error occurs. See 
attached code.

Traceback (most recent call last):
  File "C:\Documents and 
Settings\rdevaughn\Desktop\HTTP\src\FTP_directory.py", line 14, in 

ftp.storlines("STOR source.txt",f)
  File "C:\Python30\lib\ftplib.py", line 477, in storlines
if buf[-1] in B_CRLF: buf = buf[:-1]
TypeError: Type str doesn't support the buffer AP

--
components: Library (Lib)
files: module.txt
messages: 92166
nosy: rdevaughn
severity: normal
status: open
title: Error calling .storlines from ftplib
type: compile error
versions: Python 3.0
Added file: http://bugs.python.org/file14820/module.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com