Re: [Python-ideas] tarfile.extractall progress

[Tarek Ziadé]

On Fri, Sep 1, 2017, at 02:04 PM, Oleg Broytman wrote:
> Hi!
> 
> On Fri, Sep 01, 2017 at 01:50:13PM +0200, Tarek Ziad?? 
> wrote:
> > Hey,
> > 
> > For large archives, I want to display a progress bar while the archive
> > is being extracted with:
> > 
> > https://docs.python.org/3/library/tarfile.html#tarfile.TarFile.extractall
> > 
> > I could write my own version of extractall() to do this, or maybe we
> > could introduce a callback option that gets called
> > everytime .extract() is called in extractall()
> > 
> > The callback can receive the tarinfo object and where it's being
> > extracted. This is enough to plug a progress bar
> > and avoid reinventing .extractall()
> 
>What is "where" here? I think it should be 2 parameters -- position
> in the file (in bytes) and total file size; the total could be None if
> the size is unknown (the tar is piped from network or a (g/bz)zip
> subprocess).

Interesting. In my mind, I was thinking about a high level callable that
would just let me count the files and directory that are being
extracted,

my hackish implementation with clint:

with tarfile.open(file, "r:gz") as tar:
size = len(list(tar))
with progress.Bar(expected_size=size) as bar:
def _extract(self, *args, **kw):
bar.show(bar.last_progress + 1)
return self.old(*args, **kw)

tar.old = tar.extract
tar.extract = functools.partial(_extract, tar)
tar.extractall(profile_dir)

What I would expect to be able to do with the new option, something
like:

with tarfile.open(file, "r:gz") as tar:
size = len(list(tar))
with progress.Bar(expected_size=size) as bar:
def _progress(tarinfo):
bar.show(bar.last_progress + 1)
tar.extractall(profile_dir, onextracted=_progress)



> 
> > I can add a ticket and maybe a patch if people think this is a good
> > little enhancement
> 
>Definitely a good idea!
> 
> > Cheers
> > Tarek
> > 
> > -- 
> > 
> > Tarek Ziad?? | coding: https://ziade.org | running: https://foule.es |
> > twitter: @tarek_ziade
> 
> Oleg.
> -- 
>  Oleg Broytmanhttp://phdru.name/   
>  p...@phdru.name
>Programmers don't die, they just GOSUB without RETURN.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] tarfile.extractall progress

[Tarek Ziadé]

On Fri, Sep 1, 2017, at 02:18 PM, Paul Moore wrote:
[..]
> 
> Sounds like a reasonable enhancement, but for your particular use
> couldn't you just subclass TarFile and call your progress callback at
> the end of the extract method after the base class extract?

Yes that's what I ended up doing.  But a callable in extractall() sounds
like a simpler way to do it.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] tarfile.extractall progress

[Tarek Ziadé]

Hey,

For large archives, I want to display a progress bar while the archive
is being extracted with:

https://docs.python.org/3/library/tarfile.html#tarfile.TarFile.extractall

I could write my own version of extractall() to do this, or maybe we
could introduce a callback option that gets called
everytime .extract() is called in extractall()

The callback can receive the tarinfo object and where it's being
extracted. This is enough to plug a progress bar
and avoid reinventing .extractall()

I can add a ticket and maybe a patch if people think this is a good
little enhancement

Cheers
Tarek

-- 

Tarek Ziadé | coding: https://ziade.org | running: https://foule.es |
twitter: @tarek_ziade
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Argparse argument deprecation

[Tarek Ziadé]

> Another note about the proposal: calling it "deprecated" seems odd,
> since the proposal is really just a general-purpose callback.  argparse
> isn't generating the warning, your callback function would be doing it. 
> Why name it "deprecated"?  How is this different than the "action"
> keyword argument that argparse already provides?

That sounds right. Maybe a better implementation would be  to implement
a custom action by inheriting from argparse.Action

https://docs.python.org/3/library/argparse.html#action

and do all the warning/deprecation job there.

I'll experiment with this idea on my side to see how it goes :)

Cheers
Tarek
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Argparse argument deprecation

[Tarek Ziadé]

Hey,

I don't think there's any helper to deprecate an argument in argparse

Let's say you have a --foo option in your CLI and want to deprecate it
in the next release before you completely remove it later.

My first though on how to do this by adding a new "deprecated" option to
https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.add_argument

"deprecated" would be a callable that is called after the argument has
been parsed by argparse, 
so the developer can decide if they want to issue a deprecation warning,
use the parsed value or override it etc.

Another interesting approach suggest by Doug Hellman, which I like as
much, is a set of higher level options that
provide a deprecation workflow for arguments, see

https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py#L441

What do you think?

Cheers
Tarek

-- 

Tarek Ziadé | coding: https://ziade.org | running: https://foule.es |
twitter: @tarek_ziade
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Adding full() to collections.deque

[Tarek Ziadé]

Hey,

When creating deque instances using a value for maxlen, it would be nice
to have a .full() method like what Queue provides, so one may do:

my_deque = deque(maxlen=300)

if my_deque.full():
   do_something()

instead of doing:

if len(my_deque) == my_deque.maxlen:
  do_something()


If people think it's a good idea, I can add a ticket in the tracker and
try to provide a patch for the collections module maintainer.
If this was already talked about, or is a bad idea, sorry! :)

Cheers
Tarek
-- 

Tarek Ziadé | coding: https://ziade.org | running: https://foule.es |
twitter: @tarek_ziade
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/