Changes by Yury Selivanov yseliva...@gmail.com:
--
nosy: +Yury.Selivanov
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10882
___
___
Stefan Krah stefan-use...@bytereef.org added the comment:
test_os fails on the Fedora bot (--without-threads):
test test_os crashed -- Traceback (most recent call last):
File ./Lib/test/regrtest.py, line 1037, in runtest_inner
File
Roundup Robot devnull@devnull added the comment:
New changeset c45e92bd4d81 by Giampaolo Rodola' in branch 'default':
os.sendfile() test: fix AttributeError: 'NoneType' object has no attribute
'Thread' when running tests with --without-threads option.
Giampaolo Rodola' g.rod...@gmail.com added the comment:
OSX failure is tracked in issue 11351.
Closing this out as fixed.
--
resolution: accepted - fixed
status: open - closed
___
Python tracker rep...@bugs.python.org
Antoine Pitrou pit...@free.fr added the comment:
It seems we have a failure on Leopard:
I think the issue is you request a 4096 transfer near the end of file, but
there aren't that many bytes remaining. Apparently OS X then doesn't transfer
anything and returns 0. Hint:
10485760. / 4096
Giampaolo Rodola' g.rod...@gmail.com added the comment:
Committed in r88580.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10882
___
___
Georg Brandl ge...@python.org added the comment:
Three comments:
* When changing configure.in, you also should regenerate configure. (Now done
in r88584).
* PyParse_off_t is a bad name for this function. It is not a new C API, so it
should be static, and therefore there is no need for the
Giampaolo Rodola' g.rod...@gmail.com added the comment:
Thanks Georg.
It seems we have a failure on Leopard:
http://www.python.org/dev/buildbot/all/builders/PPC%20Leopard%203.x/builds/1411/steps/test/logs/stdio
Also, I think I can add support for AIX if someone gives me SSH access over an
AIX
Giampaolo Rodola' g.rod...@gmail.com added the comment:
I'm going to commit the patch and then watch whether some of the buildbots turn
red.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10882
Ross Lagerwall rosslagerw...@gmail.com added the comment:
Yes I agree it can go in now. Unless someone wants to do some tests on more
OS's like FreeBSD 7.2, Solaris, etc. (I've only checked on Linux 2.6, FreeBSD
8.1, OpenIndiana and OS X 10.5).
--
Giampaolo Rodola' g.rod...@gmail.com added the comment:
Patch in attachment removes offset and return just the number of bytes sent as
an integer.
Tests and documentation are updated accordingly.
--
Added file: http://bugs.python.org/file20733/sendfile_v8.patch
Giampaolo Rodola' g.rod...@gmail.com added the comment:
On a second thought I have two complaints.
There is no reason to return the file offset, also because this is only
supported on Linux. On all other platforms we are calculating the file offset
by making a sum of offset + number of bytes
Ross Lagerwall rosslagerw...@gmail.com added the comment:
OK, I'm happy to not return the file offset. However, I still think that
headers and trailers should remain as is since this matches the native
interface very closely.
--
___
Python tracker
Giampaolo Rodola' g.rod...@gmail.com added the comment:
I was testing against FreeBSD 7.0 RC1 and I confirm the problem doesn't occur
on 8.1 version. Patch in attachment adds test for flags argument and skips
headers/trailers tests on linux and solaris.
I'm ok for committing this as-is in 3.3.
Antoine Pitrou pit...@free.fr added the comment:
Please leave open until a patch is actually committed ;)
--
status: pending - open
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10882
___
Giampaolo Rodola' g.rod...@gmail.com added the comment:
You're right, that line is certainly wrong as it should compare for bytes but
the trailer data still doesn't get appended.
Do you have a BSD box in order to figure out what's wrong?
--
___
Ross Lagerwall rosslagerw...@gmail.com added the comment:
With no changes, I get:
==
FAIL: test_trailers (test.test_os.TestSendfile)
--
Traceback (most recent
Giampaolo Rodola' g.rod...@gmail.com added the comment:
Patch in attachment provides a complete test suite.
It also fixes a problem which occurred on BSD platforms when using non-blocking
sockets: EAGAIN/EBUSY are now raised if the transmitted data == 0 bytes
reflecting socket's send()
Ross Lagerwall rosslagerw...@gmail.com added the comment:
For trailers to work, I think the line:
self.assertEqual(data, abcde12345)
should be:
self.assertEqual(data, babcde12345)
Also not that tests like this:
if not sys.platform.startswith('linux'):
perhaps should also include solaris
Giampaolo Rodola' g.rod...@gmail.com added the comment:
In case someone is interested in statistics, I wrote a sendfile() wrapper by
using ctypes for pyftpdlib and benchmark results are quite impressive:
http://code.google.com/p/pyftpdlib/issues/detail?id=152#c5
--
Ross Lagerwall rosslagerw...@gmail.com added the comment:
Attached is a new sendfile patch which fixes the issue with FreeBSD (and Mac OS
X DragonFly BSD from what I can see).
With regards to anacrolix's request, I think what Martin said in msg126049.
i.e. if we want to provide a unifying
Giampaolo Rodola' g.rod...@gmail.com added the comment:
Could you please also add support for offset argument on Linux?
Also, headers, trailers and flags could be turned in keyword args for
simplicity.
--
___
Python tracker rep...@bugs.python.org
Ross Lagerwall rosslagerw...@gmail.com added the comment:
Attached is an updated patch that uses keyword arguments.
Using an offset with Linux was always supported although I have cleaned up the
documentation a bit to make that clearer.
E.g. the following script sends part of a file over a
Giampaolo Rodola' g.rod...@gmail.com added the comment:
Copy *count* bytes from file descriptor *in* to file descriptor
*out*, starting at *offset* and continuing for *count* bytes.
The latter part is incorrect as it is not guaranteed that all bytes specified
in count argument are going to
Matt Joiner anacro...@gmail.com added the comment:
I have a few problems with these parts of the latest patch:
+ The second case may be used on Mac OS X and FreeBSD where *headers*
+ and *trailers* are arbitrary sequences of buffers that are written before
and
+ after the data from *in*
Giampaolo Rodola' g.rod...@gmail.com added the comment:
I'm pretty sure that Solaris isn't the only platform that supports non-
socket file descriptors here, Linux (the platform I'm using), is one
such case.
No, sendfile() on Linux supports only socket file descriptors:
Matt Joiner anacro...@gmail.com added the comment:
Thanks for catching that:
Presently (Linux 2.6.9): in_fd, must correspond to a file which sup‐
ports mmap(2)-like operations (i.e., it cannot be a socket); and out_fd
must refer to a socket.
Despite the fact the manpage hasn't
Ross Lagerwall rosslagerw...@gmail.com added the comment:
OK, updated documentation and tests.
Why special case these? Why can't Mac OS X and FreeBSD write those manually
into the output file descriptor.
These can be a crucial part of certain protocols such as HTTP to ensure that a
minimal
Giampaolo Rodola' g.rod...@gmail.com added the comment:
Please note that on FreeBSD things work a little bit differently for
non-blocking sockets:
http://www.freebsd.org/cgi/man.cgi?query=sendfilesektion=2
In details I'm talking about:
When using a socket marked for non-blocking I/O,
Matt Joiner anacro...@gmail.com added the comment:
I notice Linux is described as not taking count=0 to mean to send until the end
of in is reached. Is it possible to pass (size_t)-1 to this field if None is
given, or do a loop on non-zero counts from sendfile to emulate this?
I poked around
Giampaolo Rodola' g.rod...@gmail.com added the comment:
Thanks for writing this.
Follows my comments.
I would focus on trying to provide a unique interface across all platforms.
Being sendfile() not a standard POSIX I think we should not worry about
providing a strict one-to-one interface.
Giampaolo Rodola' g.rod...@gmail.com added the comment:
Also, I think it should be optional since when it's NULL, sendfile()
implicitly assumes the current offset (file's tell() return value).
This is true on Linux, at least. Not sure about other platforms but my
best guess is that it
Ross Lagerwall rosslagerw...@gmail.com added the comment:
Just to be clear:
There are 3 different interfaces.
The basic one with the offset included no headers/trailers is supported by
all the platforms, including Linux.
The one with offset as None is only supported by Linux.
The one with
Giampaolo Rodola' g.rod...@gmail.com added the comment:
I agree then, although I'm not sure there are other functions in the os module
(or anywhere else) having a variable number of args depending on the platform.
I wanted to try your patch but it does not apply cleanly (python 3.2, revision
Giampaolo Rodola' g.rod...@gmail.com added the comment:
I agree then, although I'm not sure having a function with a variable number of
args depending on the platform is acceptable.
I wanted to try your patch but it does not apply cleanly (python 3.2, revision
87930).
--
Changes by Giampaolo Rodola' g.rod...@gmail.com:
--
Removed message: http://bugs.python.org/msg126007
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10882
___
Ross Lagerwall rosslagerw...@gmail.com added the comment:
I've just tried it against r87935 and it applies cleanly.
Perhaps you didn't apply the patch correctly (it requires -p1 since it was a
Mercurial diff), try:
patch -p1 sendfile_v2.patch
With regards to the different arguments, I
Giampaolo Rodola' g.rod...@gmail.com added the comment:
The patch as-is did not work on Linux.
I had to add entries in pyconfig.h.in and configure files in order to make
os.sendfile available.
Patch is in attachment.
--
Added file: http://bugs.python.org/file20357/sendfile_v2.patch
Martin v. Löwis mar...@v.loewis.de added the comment:
I would focus on trying to provide a unique interface across all
platforms. Being sendfile() not a standard POSIX I think we should
not worry about providing a strict one-to-one interface.
We absolutely need to expose sendfile as-is. If
Ross Lagerwall rosslagerw...@gmail.com added the comment:
Oh sorry, that was because it changed configure.in so autoreconf needs to be
run to regenerate configure pyconfig.h.in.
I thought that patches weren't meant to include the regenerated files.
Especially since differences in the
Martin v. Löwis mar...@v.loewis.de added the comment:
I thought that patches weren't meant to include the regenerated files.
Correct. Not including them is perfectly fine.
--
___
Python tracker rep...@bugs.python.org
New submission from Ross Lagerwall rosslagerw...@gmail.com:
Attached is a patch which implements os.sendfile for unix systems (linux,
freebsd, apple, solaris, dragonfly).
It takes the iov initialization code and off_t parsing from i10812.
It encapsulates all the functionality from the various
Antoine Pitrou pit...@free.fr added the comment:
Tested on Linux FreeBSD - it should work on solaris but since it
needs to link with the sendfile library and I have no idea how to link
the posix module with the sendfile library only on Solaris, i couldn't
test it.
Since the posix module
Ross Lagerwall rosslagerw...@gmail.com added the comment:
Ok, I figured it out to link with sendfile on solaris. Here is the updated
patch.
--
Added file: http://bugs.python.org/file20352/sendfile_v2.patch
___
Python tracker rep...@bugs.python.org
44 matches
Mail list logo
