Re: [Python-Dev] LZMA compression support in 3.3

Another update - I've added proper documentation. Now the code should be pretty much complete - all that's missing is the necessary bits and pieces to build it on Windows. Cheers, Nadeem ___ Python-Dev mailing list Python-Dev@python.org http://mail.pytho

Re: [Python-Dev] LZMA compression support in 3.3

I've posted an updated patch to the bug tracker, with a complete implementation of the lzma module, including 100% test coverage for the LZMAFile class (which is implemented entirely in Python). It doesn't include ReST documentation (yet), but the docstrings are quite detailed. Please take a look

Re: [Python-Dev] LZMA compression support in 3.3

I've updated the issue with a patch containing my work so far - the LZMACompressor and LZMADecompressor classes, along with some tests. These two classes should provide a fairly complete interface to liblzma; it will be possible to implement LZMAFile on top of the

Re: [Python-Dev] LZMA compression support in 3.3

On Aug 27, 2011, at 10:36 PM, Nadeem Vawda wrote: >I talked to Antoine about this on IRC; he didn't seem to think a PEP would be >necessary. But a summary of the discussion on the tracker issue might still >be a useful thing to have, given how long it's gotten. I agree with Antoine - no PEP shoul

Re: [Python-Dev] LZMA compression support in 3.3

Guido van Rossum wrote: On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg wrote: IMO, we really, really need some common way of accessing C libraries that works for all major Python variants. We have one. It's called writing an extension module. I think Dan means some way of doing this withou

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 10:36 PM, Dan Stromberg wrote: > > On Sat, Aug 27, 2011 at 8:57 PM, Guido van Rossum wrote: >> >> On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg >> wrote: >> > IMO, we really, really need some common way of accessing C libraries >> > that >> > works for all major Python v

Re: [Python-Dev] LZMA compression support in 3.3

Dan Stromberg, 27.08.2011 21:58: On Sat, Aug 27, 2011 at 9:04 AM, Nick Coghlan wrote: On Sun, Aug 28, 2011 at 1:58 AM, Nadeem Vawda wrote: On Sat, Aug 27, 2011 at 5:52 PM, Nick Coghlan wrote: It's acceptable for the Python version to use ctypes in the case of wrapping an existing library, but

Re: [Python-Dev] LZMA compression support in 3.3

> I just want to talk about it - for now. python-ideas is a better place to just talk than python-dev. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 8:57 PM, Guido van Rossum wrote: > On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg > wrote: > > IMO, we really, really need some common way of accessing C libraries that > > works for all major Python variants. > > We have one. It's called writing an extension module. > A

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg wrote: > IMO, we really, really need some common way of accessing C libraries that > works for all major Python variants. We have one. It's called writing an extension module. ctypes is a crutch because it doesn't realistically have access to the he

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 4:27 PM, Antoine Pitrou wrote: > On Sat, 27 Aug 2011 16:19:01 -0700 > Dan Stromberg wrote: > > 2) It's a rather arbitrary distinction that's being drawn between dev and > > nondev today. There's no particular reason why the line couldn't be > drawn > > somewhere else. >

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 4:27 PM, Antoine Pitrou wrote: > > Sure. Now please convince Linux distributions first, because this > particular subthread is going nowhere. > I hope you're not a solipsist. Anyway, if the mere -discussion- of embracing a standard and safe way of making C libraries call

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, 27 Aug 2011 16:19:01 -0700 Dan Stromberg wrote: > 2) It's a rather arbitrary distinction that's being drawn between dev and > nondev today. There's no particular reason why the line couldn't be drawn > somewhere else. Sure. Now please convince Linux distributions first, because this part

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 3:26 PM, Antoine Pitrou wrote: > On Sat, 27 Aug 2011 15:14:15 -0700 > Dan Stromberg wrote: > > > On Sat, Aug 27, 2011 at 2:38 PM, Nadeem Vawda >wrote: > > > > > On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg > > > wrote: > > > > It seems like there should be some way o

Re: [Python-Dev] LZMA compression support in 3.3

> Why -can't- we expect the user to have liblzma headers installed? > Couldn't it just be a dependency in the package management system? Please give it up. You just won't convince that list that ctypes is a viable approach for the standard library. Regards, Martin ___

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, 27 Aug 2011 15:14:15 -0700 Dan Stromberg wrote: > On Sat, Aug 27, 2011 at 2:38 PM, Nadeem Vawda wrote: > > > On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg > > wrote: > > > It seems like there should be some way of coming up with an xml file > > > describing the types of the various bi

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 2:38 PM, Nadeem Vawda wrote: > On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg > wrote: > > It seems like there should be some way of coming up with an xml file > > describing the types of the various bits of data and formal arguments - > > perhaps using gccxml or somethin

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg wrote: > It seems like there should be some way of coming up with an xml file > describing the types of the various bits of data and formal arguments - > perhaps using gccxml or something like it. The problem is that you would need to do this check

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 1:21 PM, "Martin v. Löwis" wrote: > > I'd like to better understand why ctypes is (sometimes) frowned upon. > > > > Is it the brittleness? Tendency to segfault? > > That, and Python should work completely if ctypes is not available. > What are the most major platforms cty

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 9:47 PM, Terry Reedy wrote: > On 8/27/2011 9:47 AM, Nadeem Vawda wrote: >> I'd like to propose the addition of a new module in Python 3.3. The 'lzma' >> module will provide support for compression and decompression using the >> LZMA >> algorithm, and the .xz and .lzma file

Re: [Python-Dev] LZMA compression support in 3.3

> I'd like to better understand why ctypes is (sometimes) frowned upon. > > Is it the brittleness? Tendency to segfault? That, and Python should work completely if ctypes is not available. > FWIW, I have a partial implementation of a module that does xz from > Python using ctypes. So does it w

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 9:04 AM, Nick Coghlan wrote: > On Sun, Aug 28, 2011 at 1:58 AM, Nadeem Vawda > wrote: > > On Sat, Aug 27, 2011 at 5:52 PM, Nick Coghlan > wrote: > >> It's acceptable for the Python version to use ctypes in the case of > >> wrapping an existing library, but the Python ver

Re: [Python-Dev] LZMA compression support in 3.3

On 8/27/2011 9:47 AM, Nadeem Vawda wrote: I'd like to propose the addition of a new module in Python 3.3. The 'lzma' module will provide support for compression and decompression using the LZMA algorithm, and the .xz and .lzma file formats. The matter has already been discussed on the tracker

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, 27 Aug 2011 18:50:40 +0200 Antoine Pitrou wrote: > On Sun, 28 Aug 2011 01:52:51 +1000 > Nick Coghlan wrote: > > > > > > The plausible story being that we basically wrap an existing library? > > > I don't think PyPy et al have pure Python versions of the zlib or > > > OpenSSL, do they? >

Re: [Python-Dev] LZMA compression support in 3.3

>>> PEP 399 also comes into play - we need a pure Python version for PyPy >>> et al (or a plausible story for why an exception should be granted). No, we don't. We can grant an exception, which I'm very willing to do. The PEP lists wrapping a specific C-based library as a plausible reason. > It's

Re: [Python-Dev] LZMA compression support in 3.3

On Sun, 28 Aug 2011 01:52:51 +1000 Nick Coghlan wrote: > > > > The plausible story being that we basically wrap an existing library? > > I don't think PyPy et al have pure Python versions of the zlib or > > OpenSSL, do they? > > > > If we start taking PEP 399 conformance to such levels, we might a

Re: [Python-Dev] LZMA compression support in 3.3

On Sun, Aug 28, 2011 at 1:58 AM, Nadeem Vawda wrote: > On Sat, Aug 27, 2011 at 5:52 PM, Nick Coghlan wrote: >> It's acceptable for the Python version to use ctypes in the case of >> wrapping an existing library, but the Python version should still >> exist. > > I'm not too sure about that - PEP 3

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 5:42 PM, "Martin v. Löwis" wrote: > Not sure whether you already have this: supporting the tarfile module > would be nice. Yes, got that - issue 5689. Also of interest is issue 5411 - adding .xz support to distutils. But I think that these are separate projects that should

Re: [Python-Dev] LZMA compression support in 3.3

On Sun, Aug 28, 2011 at 1:40 AM, Antoine Pitrou wrote: > On Sun, 28 Aug 2011 01:36:50 +1000 > Nick Coghlan wrote: >> On Sun, Aug 28, 2011 at 1:15 AM, "Martin v. Löwis" >> wrote: >> > This is exactly what I worry about. I think adding file I/O to bz2 was a >> > mistake, as this doesn't integrate

Re: [Python-Dev] LZMA compression support in 3.3

On Sun, 28 Aug 2011 01:36:50 +1000 Nick Coghlan wrote: > On Sun, Aug 28, 2011 at 1:15 AM, "Martin v. Löwis" wrote: > > This is exactly what I worry about. I think adding file I/O to bz2 was a > > mistake, as this doesn't integrate with Python's IO library (it used > > to, but now after dropping s

Re: [Python-Dev] LZMA compression support in 3.3

> It is not my intention for the _lzma C module to do I/O - that will be done by > the LZMAFile class, which will be written in Python. My comparison with bz2 > was > in reference to the state of the module after it was rewritten for issue 5863. Ok. I'll defer my judgement then until actual code

Re: [Python-Dev] LZMA compression support in 3.3

On Sun, Aug 28, 2011 at 1:15 AM, "Martin v. Löwis" wrote: > This is exactly what I worry about. I think adding file I/O to bz2 was a > mistake, as this doesn't integrate with Python's IO library (it used > to, but now after dropping stdio, they were incompatible. Indeed, for > Python 3.2, BZ2File

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 5:15 PM, "Martin v. Löwis" wrote: >> As for file formats, these are handled by liblzma itself; the extension >> module >> just selects which compressor/decompressor initializer function to use >> depending >> on the value of the "format" argument. Our code won't contain a

Re: [Python-Dev] LZMA compression support in 3.3

> As for file formats, these are handled by liblzma itself; the extension module > just selects which compressor/decompressor initializer function to use > depending > on the value of the "format" argument. Our code won't contain anything along > the > lines of GzipFile; all of that work is done

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 4:50 PM, "Martin v. Löwis" wrote: >> The implementation will also be similar to bz2 - basic compressor and >> decompressor classes written in C, with convenience functions and a file >> interface implemented on top of those in Python. > > When I reviewed lzma, I found that

Re: [Python-Dev] LZMA compression support in 3.3

> The implementation will also be similar to bz2 - basic compressor and > decompressor classes written in C, with convenience functions and a file > interface implemented on top of those in Python. When I reviewed lzma, I found that this approach might not be appropriate. lzma has many more option

Re: [Python-Dev] LZMA compression support in 3.3

> I'd like to propose the addition of a new module in Python 3.3. The 'lzma' > module will provide support for compression and decompression using the LZMA > algorithm, and the .xz and .lzma file formats. The matter has already been > discussed on the tracker , whe

[Python-Dev] LZMA compression support in 3.3

Hello all, I'd like to propose the addition of a new module in Python 3.3. The 'lzma' module will provide support for compression and decompression using the LZMA algorithm, and the .xz and .lzma file formats. The matter has already been discussed on the tracker ,