Re: Is npyscreen still alive?

[Tim Daneliuk via Python-list]

On 4/24/23 11:32, Grant Edwards wrote:

On 2023-04-24, Grant Edwards  wrote:


The other big advantage of an ncurses program is that since curses
support is in the std library, a curses app is simpler to
distribute.  Right now, the application is a single .py file you
just copy to the destination machine and run.  It supports
command-line use and a Tk GUI. I can add an ncurses "CUI" without
having to either adopt a more complex bundling mechanism that
requires it to be "installed" or require that users install
dependencies via pip/apt/yum/whatever.


However... I just realized that Python's curses support is missing two
huge chunks: both menu and form support are not there.  I guess that
explains why people feel the need to write high-level UI wrappers for
Python curses: the high level stuff that curses does support is
missing from the Python bindings.

Adding a curses UI for my app might not be feasible after all...

--
Grant




That's because the Gods Of Ancient Internets knew that all you
ever really need is VT220 escape sequences ... unless you
are one of the heathens that still program in HLLAPI on 327x
machinery    (I kid, I kid, NO one needs that pain ...)

--
https://mail.python.org/mailman/listinfo/python-list

Re: Is npyscreen still alive?

[Tim Daneliuk via Python-list]

On 4/24/23 09:14, Stefan Ram wrote:

Grant Edwards  writes:

The other big advantage of an ncurses program is that since curses
support is in the std library, a curses app is simpler to distribute.


   IIRC curses is not in the standard library /on Windows/. I miss
   a platform independent (well, at least for Linux, Mac, and
   Windows) package with curses features in the standard library.





That's correct (or was, last time I looked).  For this reason, I
resorted to using tkinter for the twander file browser.  While
it works, the code needs a complete rethink and to be written
to be Python3 compatible.  Perhaps when/if that happens, something
like Textual need serious consideration.

tkinter works, but is showing its age.  So a fresher look without
all the burden of X and or requiring a browser, while also giving
you that option is appealing.
--
https://mail.python.org/mailman/listinfo/python-list

Re: any author you find very good has written a book on Python?

[Tim Daneliuk via Python-list]

On 9/5/22 21:22, Meredith Montgomery wrote:

I never read a book on Python.  I'm looking for a good one now.  I just
searched the web for names such as Charles Petzold, but it looks like he
never wrote a book on Python.  I also searched for Peter Seibel, but he
also never did.  I also tried to search for Richard Heathfield.  (I took
a look at his ``C Unleashed'' once and I liked what I saw.)  This is how
I search for books --- I go through the authors first.  Charles Petzold,
for instance, anything he writes is worth reading it.  (Have you given
his Annotated Turing a shot?  It's a very nice read.)

So that's my request --- any author you find very good has written a
book on Python?

It could be for in a certain specific context.  For instance, I also
searched for Hadley Wickham in the hope that he could have written a
data-science-type of book using Python.  I like his writing a lot, but
he also only seems to have written only for the R language.

Thank you!


David Beazley

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python's carbon guilt

[Tim Daneliuk]

On 10/10/20 2:35 PM, Marco Sulla wrote:
> He should also calculate the carbon dioxide emitted by brains that
> works in C++ only. I omit other sources.
> 


yes, methane is an alleged greenhouse gas as well
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Tim Daneliuk]

On 8/19/20 3:29 PM, Ethan Furman wrote:
> On 8/19/20 12:40 PM, Tim Daneliuk wrote:
>> On 8/19/20 2:00 PM, Karen Shaeffer wrote:
> 
>>> Considering all your posts on this thread, it is reasonable to infer you 
>>> have some ideological motivations.
>>
>> My motivation was to demonstrate that if people of your ilk are free to
>> peddle their worldview,
> 
> Unless you know Karen personally, you don't know what her world view is.  
> Poking holes in arguments or observing what is being said does not require an 
> opposing world view.

I'd say this is at least a hint:

>>> this peer reviewed published research from Stanford University establishes 
>>> the fact that racism and bias are alive and well in the STEM fields:

I have rather lengthy counterpoint to this "research".  Ditto the claim that 
proper
ordinarily use of English is prima facie evidence of "White Supremacy".  Ditto
the claim that any reference to age is inherently bigoted and cannot be taken
in good spirit.  But ... I don't think this is the place you'd like to see
my disquisitions on the matter.*

My larger point holds:  You cannot allow one worldview in and then expect
other worldviews to keeps still  ... and that's true irrespective of WHICH
worldview first got in the door.


> As you yourself said:
> 
>> The sole relevant
>> demand should be civility.
> 
> Let's make sure we stay that course.


I have never been anything other than this in the entire thread.  It is not 
uncivil
to disagree.

> 
> -- 
> ~Ethan~
> Python List Moderator

* Is the use of the word "disquisition" a form of White Supermacy?  Asking for 
a friend.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Tim Daneliuk]

On 8/19/20 1:10 PM, J. Pic wrote:
> Tim, don't you also think that statements should be backed by
> evidence, even more if they are particularly accusatory ?
> 
> We'll be lucky if S's editor doesn't sue the PSF for slandering for
> publishing that S "upholds white supremacy".
> 

As a general matter, I agree: Claims should be supported by evidence.
But that's not the problem here.  The various cause crusaders are
attempting to insert themselves into every single discipline.  They
do this claiming their attacks as some kind of moral virtue and that
they are therefore above reproach or counterpoint.  It is natural
for people who disagree to punch back.

My real point in commenting at all was that we open this one-sided door
at our own peril. By allowing provocative commentary of the sort voice
in this godforsaken commit message, we absolutely invite others to the party.
Despite what the aforementioned cause crusaders think, they are not
unassailable, nor are they the only- or dominant voice in these matters.
Open this door and you get an absolute sewer of commentary
(from many sides of these issues).  Social media is full of many trenchant
such examples.  Best to leave it there.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Tim Daneliuk]

On 8/19/20 2:00 PM, Karen Shaeffer wrote:
> Where you conclude with: "Methinks there is an ideological skunk in the 
> parlor …”
> 
> Considering all your posts on this thread, it is reasonable to infer you have 
> some ideological motivations.

My motivation was to demonstrate that if people of your ilk are free to
peddle their worldview, it invites people of my worldview to join the party
and thereby wreak havoc.  The only way to solve this is: No politics, no 
culture,
no religion, or no sex. Period.  Ever.

Do take note that while I freely admit to having utter contempt for the
en courant culture and its various warriors, I also have been careful
to avoid flogging *my own* political/social view.  They don't belong
here.  Neither do yours.  Why don't we stick to discussing Python which
ought to not be infected by these subjects.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Tim Daneliuk]

On 8/18/20 12:18 PM, gia wrote:
>  That's why I picked Math, it is also universally accepted, it's very
> strict, and it leaves the reader to decide its color based on themselves
> (it's not white btw :)


Sorry, but when it comes to the demands of the woke, you are not
immune.  Reported widely earlier this month, seen here most recently:


https://thejewishvoice.com/2020/08/brooklyn-college-education-prof-claims-math-is-white-supremacist-patriarchy/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Tim Daneliuk]

On 8/19/20 8:35 AM, Alexandre Brault wrote:
> I've not seen anyone objecting to the idea of removing the reference to 
> Strunk and White in favour of the underlying message of "be understandable by 
> others who may read your comments" (there were at most a few philosophical 
> "what is understandable") . In fact, that is how the topic was initially 
> presented.
> 
> What people *are* complaining about is the use of a commit message to stand 
> on a soapbox and preach. The time to preach was when debating the change; 
> commit messages, in many people's opinions, is not the time to espouse 
> non-technical opinions

I would argue that these highly polarizing political discussions never have
a place at any point in the lifecycle of technology improvement.  Once you
open this door in any way, no end of mischief ensues.

You already see this in this very thread.  People are determined to flog
their particular political theory, virtue signal, and generally misappropriate
a technical forum to their own ends.

The right answer here is: No politics, no social commentary, no demands for
redress of perceived historical inequities ... none of that.  The sole relevant
demand should be civility.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Tim Daneliuk]

On 8/18/20 6:34 PM, rmli...@riseup.net wrote:
> I would kindly recommend that folks just educate themselves on what

Speaking of being educated ...  Could you please do an exposition
for all us ignorant types on the books that really animate
your worldview:


The_Origin of the Family, Private Property, and the State - Engels

Das Kapital - Marx

On Guerrilla Warfare - Mao

There are many noble causes for which to fight.  Yours isn't one of them.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Tim Daneliuk]

On 8/18/20 6:34 PM, rmli...@riseup.net wrote:
> I would kindly recommend that folks just educate themselves on what


I would also like to help you become educated.  Be sure to check
out these literary treasures - they are the foundation of the
worldview you are espousing:


The_Origin of the Family, Private Property, and the State - Engels

Das Kapital - Marx

On Guerrilla Warfare - Mao

Your claims to virtue are a fraud.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Tim Daneliuk]

On 8/18/20 12:28 PM, justin walters wrote:
> I apologize for being ageist earlier as well. That was out of line.

I am likely older than you and there is no reason to apologise.
Only the profoundly undeveloped psyche takes every opportunity to
find offense when  none is intended.  It is the sign of a puerile
mind, irrespective of actual chronological age.  Feel free to be
as "ageist" as you wish ... only make it funny or biting ...

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Final statement from Steering Council on politically-charged commit messages

[Tim Daneliuk]

On 8/17/20 1:26 PM, Chris Angelico wrote:
> For context, see this commit:
> 
> https://github.com/python/peps/commit/0c6427dcec1e98ca0bd46a876a7219ee4a9347f4
> 
> The commit message is highly politically charged and is now a
> permanent part of the Python commit history. The Python Steering
> Council has this to say:
> 
> https://github.com/python/steering-council/issues/34#issuecomment-675028005
> 
> "The SC discussed this and ... we do not deplore the message."
> 
> So now we know: go ahead and put all the political messages you like
> into the commit messages, just don't put anything inappropriate into
> the content. White supremacy has been mentioned; who wants to pick the
> next hot topic?
> 
> ChrisA
> 
Just a few thoughts here ...

- While languages evolve over time, _in any given moment_ there are better
  and worse ways to express ideas in a given language. "The Elements Of Style"
  remains relevant today because it provides guidance on improving
  written clarity.  It is not some blind defence of the
  perfect English.

- Precision of language and precision of thought go hand in hand.  Much
  of the grousing about languages standards (in this case, hiding in
  drag as social consciousness) is little more than intellectual laziness.
  In actual fact, our discipline has burned a lot of intellectual
  fuel in trying to find ways to be _more precise_ for things like
  specifications, formal semantics, and the like.

- It is my consistent experience when working with non-native English
  speakers, that they wish to _improve_ their use and precision of the
  language, not simplify it.

- Why is English the only target of these social pieties?  You never
  hear these demands to relax these linguistic standards for, say, French,
  German, or Spanish.  Similarly, where is the outcry to make
  Mandarin, Bantu, Swahili, or Arabic more approachable for
  Westerners?

Methinks there is an ideological skunk in the parlor ...


-- 
https://mail.python.org/mailman/listinfo/python-list

3.8.5 Failing To Install With pythonz

[Tim Daneliuk]

I have a weird problem I could use a bit of help with ...

I have successfully installed 3.8.5 using pew/pythonz on a BSD FreeBSD system.
But when I attempt to install it on a Linux system I get the traceback below.
In this case, pew/pythonz were installed locally in my own account using system
native python 3.6:

Installing CPython-3.8.5 into /home/tundra/.pythonz/pythons/CPython-3.8.5
Traceback (most recent call last):
  File 
"/home/tundra/.local/lib/python3.6/site-packages/pythonz/installer/pythoninstaller.py",
 line 204, in install
self.make()
  File 
"/home/tundra/.local/lib/python3.6/site-packages/pythonz/installer/pythoninstaller.py",
 line 350, in make
s.check_call(make)
  File "/home/tundra/.local/lib/python3.6/site-packages/pythonz/util.py", line 
294, in check_call
returncode = self.call(cmd)
  File "/home/tundra/.local/lib/python3.6/site-packages/pythonz/util.py", line 
286, in call
fp.write(line)
UnicodeEncodeError: 'ascii' codec can't encode character '\u2018' in position 
81: ordinal not in range(128)
ERROR: Failed to install CPython-3.8.5. Check 
/home/tundra/.pythonz/log/build.log to see why.
Traceback (most recent call last):
  File "/home/tundra/.local/bin/pew", line 11, in 
sys.exit(pew())
  File "/home/tundra/.local/lib/python3.6/site-packages/pew/pew.py", line 809, 
in pew
return command(sys.argv[2:])
  File "/home/tundra/.local/lib/python3.6/site-packages/pew/pew.py", line 704, 
in install_cmd
return actual_installer.install()
  File 
"/home/tundra/.local/lib/python3.6/site-packages/pythonz/installer/pythoninstaller.py",
 line 204, in install
self.make()
  File 
"/home/tundra/.local/lib/python3.6/site-packages/pythonz/installer/pythoninstaller.py",
 line 350, in make
s.check_call(make)
  File "/home/tundra/.local/lib/python3.6/site-packages/pythonz/util.py", line 
294, in check_call
returncode = self.call(cmd)
  File "/home/tundra/.local/lib/python3.6/site-packages/pythonz/util.py", line 
286, in call
fp.write(line)
UnicodeEncodeError: 'ascii' codec can't encode character '\u2018' in position 
81: ordinal not in range(128)


Ideas anyone?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Lists And Missing Commas

[Tim Daneliuk]

On 12/24/19 6:37 AM, Stefan Ram wrote:
>  And you all are aware that this kind of string concatenation
>   happens in C and C++, too, aren't you?
> 
>   main.c
> 
> #include 
> int main( void ){ puts( "a" "b" ); }
> 
>   transcript
> 
> ab

Noting that it has been a long time since I looked at the C specification ...

Is the above an artifact of how puts() is implemented or is it innate in the 
language spec?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Lists And Missing Commas

[Tim Daneliuk]

On 12/23/19 8:35 PM, Chris Angelico wrote:
> On Tue, Dec 24, 2019 at 12:56 PM DL Neil via Python-list
>  wrote:
>> However, your point involves the fact that whereas:
>>
>> 1 + 2   # 3 is *clearly* addition, and
>> "a" + "b"   # "ab" is *clearly* concatenation
>>
>> "a" "b" # also evaluates to "ab"
>>
>> and is thus, concatenation without any explicit infix operator! Just one
>> cotton-picking minute - what's going on here?
> 
> 1_2# Clearly concatenation. Right?
> 0_0# Clearly an emoticon
> 
> Just another cotton-picking minute..
> 
> ChrisA
> 

You are excused and required to do 30 days penance writing .NET.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Lists And Missing Commas

[Tim Daneliuk]

On 12/23/19 7:52 PM, DL Neil wrote:
> 
> WebRef: https://docs.python.org/3/reference/lexical_analysis.html


Yep, that explains it, but it still feels non-regular to me.  From a pointy 
headed academic
POV, I'd like to see behavior consistent across types. Again ... what do I know?

-- 
https://mail.python.org/mailman/listinfo/python-list

Lists And Missing Commas

[Tim Daneliuk]

If I do this:

foo = [ "bar", "baz" "slop", "crud" ]

Python silently accepts that and makes the middle term "bazslop".

BUT, if I do this:

foo = [ "bar", "baz" 1, "crud" ]

or this:

foo = [ "bar", 2 1, "crud" ]

The interpreter throws a syntax error.

This is more of an intellectual curiosity than anything else, but why do 
strings silently
concatenate like that, while all other case blow up with an error.  i.e., What 
is about
the language the promotes this behavior.  At first blush, it seems 
inconsistent, but what
do I know ...




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Randomizing Strings In A Microservices World

[Tim Daneliuk]

On 12/10/19 12:37 PM, Chris Angelico wrote:
> On Wed, Dec 11, 2019 at 5:01 AM Tim Daneliuk  wrote:
>>
>> On 12/10/19 10:36 AM, Peter Pearson wrote:
>>> Just to be sure: you *are* aware that the "Birthday Paradox" says
>>> that if you pick your 10-digit strings truly randomly, you'll probably
>>> get a collision by the time of your 10**5th string . . . right?
>>
>> I did not consider this, but the point is taken.
>>
>> Could you kindly point me to a source for calculating this given
>> n-digit numeric-only strings?
>>
> 
> The exact formula is pretty gnarly, but you can get remarkably close
> by assuming that you're likely to get a collision at the square root -
> which is half the exponent (so a 16-bit checksum will collide after
> about 2**8 examples, and a 128-bit UUID4 will collide after about
> 2**64 UUIDs are generated).
> 
> https://en.wikipedia.org/wiki/Birthday_problem
> 
> ChrisA
> 


Close enough for my purposes.  Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Randomizing Strings In A Microservices World

[Tim Daneliuk]

On 12/10/19 10:36 AM, Peter Pearson wrote:
> Just to be sure: you *are* aware that the "Birthday Paradox" says
> that if you pick your 10-digit strings truly randomly, you'll probably
> get a collision by the time of your 10**5th string . . . right?

I did not consider this, but the point is taken.

Could you kindly point me to a source for calculating this given
n-digit numeric-only strings?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Randomizing Strings In A Microservices World

[Tim Daneliuk]

On 12/9/19 8:54 PM, Dennis Lee Bieber wrote:
> On Mon, 9 Dec 2019 18:52:11 -0600, Tim Daneliuk 
> declaimed the following:
> 
>>
>> - Each of these services needs to produce a string of ten digits guaranteed 
>> to be unique
>>  on a per service instance basis AND to not collide for - oh, let's say - 
>> forever :)s
>>
>> Can anyone suggest a randomization method that might achieve this 
>> efficiently?
>>
>> My first thought was to something like nanonseconds since the epoch plus 
>> something
>> unique about the service instance - like it's IP?  (This is in a K8s 
>> cluster) - to
>> see the randomization and essentially eliminate the string being repeated.
>>
>> Ideas welcome ..
>>
> 
>   Well, 10 digits is rather short, but studying
> https://en.wikipedia.org/wiki/Universally_unique_identifier might provide
> some ideas.
> 
> 

For a variety of reasons the length of this string cannot exceed 10 digits.  It 
is believed
that - in this application - the consumption of values will be sparse over 
time.  All
I really need is a high entropy way to select from among the billion possible 
values to
minimize the possibility of collisions (which require retry and time we don't 
want to burn).

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Randomizing Strings In A Microservices World

[Tim Daneliuk]

On 12/9/19 8:50 PM, Paul Rubin wrote:
> Tim Daneliuk  writes:
>> - Imagine an environment in which there may be multiple instances of a given
>>   microservice written in Python.
> 
> Decide the maximum number of microservice instances, say 1000.  Chop up
> the 10 digit range into 1000 pieces, so 0..99, 100-199, etc.
> Give one range to each microservice instance.  Then have the
> microservices give out the numbers sequentially, but treating them as 10
> digit numbers and encrypting each one under a 10 digit pseudorandom
> permutation shared by all the instances.  Look up "format preserving
> encryption" for how to do this.
> 
> Obvious variants of the above are obvious, and maybe you need some way
> to hand around chunks of range if some instance gives out more than a
> million numbers.
> 


The problem here is that the services are ephemeral and the number of said
services is not fixed.

-- 
https://mail.python.org/mailman/listinfo/python-list

Randomizing Strings In A Microservices World

[Tim Daneliuk]

I ran across a kind of fun problem today that I wanted to run past you Gentle 
Geniuses (tm):

- Imagine an environment in which there may be multiple instances of a given
  microservice written in Python.

- Each of these services needs to produce a string of ten digits guaranteed to 
be unique
  on a per service instance basis AND to not collide for - oh, let's say - 
forever :)s

Can anyone suggest a randomization method that might achieve this efficiently?

My first thought was to something like nanonseconds since the epoch plus 
something
unique about the service instance - like it's IP?  (This is in a K8s cluster) - 
to
see the randomization and essentially eliminate the string being repeated.

Ideas welcome ..

P.S. I do not want to resort to a number generation service that each instance 
asks
 for a new number.  The network and service call overhead militates against 
this
 in my application.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 4:28 PM, Brian Oney wrote:
> Why not make a compromise? What would be a potential pitfall of the
> following spitbang?
> 
> #!python

Not sure this really changes the discussion.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/21/19 8:47 AM, Peter J. Holzer wrote:
> That's fine. Unlike Tim I don't claim that anybody who disagrees with me
> must be a newbie.

Peter, that's ad hominem and unfair.  I never said anything close to that.
What I said is that if someone were to spend an extended period of time
in devops and systems engineering at large scale, they'd quickly come to
hate hardwired paths.

The truth is that there is no single answer to this problem until you
hermetically seal an environment.  Docker comes close, a freestanding
VM does this most completely.   But short of that, the ability to decide
to use something other than system-provided tools absolutely IS a requirement
in systems of any scale.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 6:04 PM, Cameron Simpson wrote:
> If you require a specific outcoming, set a specific environment. It is under 
> your control. Control it.

Exactly right.  I have just had the REALLY irritating experience of trying to 
bootstrap a
location insensitive version of linuxbrew that mostly works, but is crippled by 
brain damage
that insists that the best version of perl will always be found in /usr/bin.

I have been a hardware engineer (analog and digital), software implementor, 
systems designer,
devops person, and large scale (physical data center) platform engineer and 
this kind of
bad thinking just kills site reliability.

I stipulate that there are corner cases where Chris A. is correct - it is 
certainly
possible to clobber a system with incorrect /usr/bin/env findings.   But I'd 
argue
that there is a simple work around - run another terminal session (terminator, 
tmux,
screen, ... whatever floats your boat) that has $PATH set up to find things in 
/usr/bin
first.  Voila! Problem solved and everyone can use env the way they want to.

In some respects, this problem does get a little simpler when you docker-ize 
your
software distributions until ... you need newer versions of tools than even the 
latest
docker OS implementations support.  Then you're back to the same old noise ...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 6:04 PM, Chris Angelico wrote:
> Are you aware of every systemwide command that happens to be
> implemented in Python, such that you won't execute any of them while
> you have the venv active?

No, but this has never been a problem because the newer versions of
python tend to be pretty good - within a major release tree - of being
backward compatible.  Certainly, if I were running, say, RedHat 4 with
a very new version of python in my path first, this could theoretically
be an issue.  I've just not run into it.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 5:47 PM, Tim Daneliuk wrote:
> On 7/20/19 5:14 PM, Chris Angelico wrote:
>> Using env for everything is a terrible idea and one that
>> will basically make virtual environments useless.
> 
> Not if you manage them properly.
> 
> Everyone's mileage is different, but when I enter a venv, I ensure everything 
> I do
> there is packaged to work together.  I pip install things in there and ensure
> that the tools I invoke will work in that environment.
> 
> In the example you cite, I would simply run the tools in question outside any
> environment where I know they don't work.
> 

I guess I should clarify something:

1) I pip install everything locally inside my own $HOME for that set of things 
I need
   generally and will not be using in a venv.

2) I pip install everything I need in each venv, even if it already exists 
under 1) above.

I effectively treat venvs as fully self-contained environments to the degree 
possible.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 5:14 PM, Chris Angelico wrote:
> Using env for everything is a terrible idea and one that
> will basically make virtual environments useless.

Not if you manage them properly.

Everyone's mileage is different, but when I enter a venv, I ensure everything I 
do
there is packaged to work together.  I pip install things in there and ensure
that the tools I invoke will work in that environment.

In the example you cite, I would simply run the tools in question outside any
environment where I know they don't work.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 2:56 PM, Peter J. Holzer wrote:
> On 2019-07-20 14:11:44 -0500, Tim Daneliuk wrote:
>> So, no, do NOT encode the hard location - ever.  Always use env to
>> discover the one that the user has specified.  The only exception is
>> /bin/sh which - for a variety of reasons - can reliably counted upon.
>>
>> We don't need to bikeshed this.  All we need is people who disagree
>> with this view to spend a year in software packaging, operations,
>> deployment and DevOps ... then get back to us...
>
> After 25 years in software packaging, operations, deployment and DevOps
> I disagree: A program should not behave differently because of a
> different path, #!/usr/bin/env is a total no-no.
>
> There is a nice way to achieve this: Just use the interpreter of the
> virtual environment in the shebang.
> (That requires rewriting the shebang during installation, but that's a
> minor inconvenience)
>
> hp
>



And what happens with programs that have no virtenv equivalent?
perl, go, ruby, awk, sed, grep, etc. have no simple way to
get installed virtual short of insisting the everything live in a
docker container or VM?

The fact is that most large compute environments are slow to upgrade
the OS.  That means core tools also lag considerably behind as well.
Being able to install newer versions along side the OS' own and then
use them by default is manifestly necessary.  That's why users have
the ability to modify $PATH to suit their own needs.  All /usr/bin/env
does is to tell the interpreter, "honor the intent of the spawning shell".
This shouldn't even be a question ... and it's why so much garbage continues
to live forever.  Having to constantly code around old systems versions of
tools with not other recourse is just crazy.

In actual fact, the very best way to write portable, reliable, and operable
systems of code is to divorce them entirely (or as a nearly as possible) for
the OS tools as you can.  That's why docker works so well.  That's why go
avoids dynamic linking.

In my case, that's why I compile my own version of
languages and daily use utilities to live outside the OS.  I get absolutely
predicable behavior irrespective of my distro and whatever backleveled cruft
it has laying around.   I _know_ every version of every important tool my code
will be using ... all by setting up $PATH properly and using /usr/bin/env in
my interpreters.

If you want really big fun, try going into an older CentOS or RedHat instances 
and, say,
upgrading system python to python3.  It's super fun.  Yes, in that case, you 
COULD
use a venv.  But there are tons of other tools for which this is not an option -
gcc, autoconf, perl, go awk, sed, bash, ad infinitum, ad nauseum are invariably
backleveled on production OS instances.  My way fixes that problem.  Your way
forces me to code around it ... which is a way worse solution.

You may have 25 years at this but I have 40 - does that make me nearly twice
as right?  Arguments from authority are silly.

P.S. https://www.tundraware.com/TechnicalNotes/Divorce-Your-Linux-Admin/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 1:20 PM, Chris Angelico wrote:
> On Sun, Jul 21, 2019 at 4:13 AM Michael Speer  wrote:
>>
>> You may want to use `#!/usr/bin/env python3` instead.
>>
>> There is a concept in python called the virtual environment. This used to
>> be done with a tool called virtualenv in python2, and is now done mainly
>> through a venv module in python3.
>>
>> A virtual environment goes into a directory of your choosing and will have
>> its own python3 executable, and pip3 executable, and when you add
>> dependencies, they are also placed into the directory structure under your
>> chosen directory.
>>
>> When you do a `. /bin/activate` the included source will places
>> the virtual environment's bin/ folder at the beginning of your PATH
>> environment variable, making it the default python3 when you type it
>> without a full path.
>>
>> This allows you to run scripts that need different, or even conflicting,
>> sets of dependencies without bothering with the underlying linux
>> distribution's python installation's modules.
>>
>> If you use `#!/usr/bin/python3`, it will always use exactly the system
>> version that is installed, and the system's installed modules.
>>
>> Your scripts will still default to the system installation if a virtual
>> environment is not activated. So you lose nothing by doing it this way, but
>> gain a little control from it.
>>
> 
> ONLY if you actually want this script to behave differently inside a
> venv. When you're setting a shebang on a script, you often do not want
> that. You potentially LOSE a lot of control.
> 
> ChrisA

Disagree strongly.  The environment should always define where you want to find 
key
binaries, whether in a venv or not.  There are many use cases where you want to
override system binaries.  For example, you may be running on an older OS 
release
and want to point to a newer one installed elsewhere.  Similarly, the DevOps 
workflow
may demand particular configurations for the pipelines to run properly.

I have spent way too much time in my career trying to undo this kind of 
imperious
programming that thinks the coder knows best, when in the actual runtime 
environment,
they actually don't.  Most recently, I have been working to produce a 
configuration
for linuxbrew that can be installed by an unprivileged user at any location 
they like.
It is maddening to spend hours getting things working only to find out that 
some genius
decided that the only version of, say, perl or autoconf, is to be found 
exclusively in
/usr/bin.

So, no, do NOT encode the hard location - ever.  Always use env to discover the 
one that
the user has specified.  The only exception is /bin/sh which - for a variety of 
reasons -
can reliably counted upon.

We don't need to bikeshed this.  All we need is people who disagree with this 
view to spend
a year in software packaging, operations, deployment and DevOps ... then get 
back to us...

Grr..

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: join and split with empty delimiter

[Tim Daneliuk]

On 7/17/19 4:24 PM, Chris Angelico wrote:
> Agreed. There are a number of other languages where splitting on an
> empty delimiter simply fractures the string into characters (I checked
> Pike, JavaScript, Tcl, and Ruby), and it's a practical and useful
> feature. +1.

Not only that, it makes the language more symmetric/consistent. Put
me down for +1 as well.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Lifetime of a local reference

[Tim Daneliuk]

On 2/26/19 3:54 PM, Marko Rauhamaa wrote:
> Consider this function:
> 
> def fun():
> f = open("lock")
> flock.flock(f, fcntl.LOCK_EX)
> do_stuff()
> sys.exit(0)
> 
> Question: can a compliant Python implementation close f (and,
> consequently, release the file lock) before/while do_stuff() is
> executed?
> 
> I couldn't find an immediate answer in the documentation.


Not quite sure what you are asking.  Are you asking if the file handle
can be closed before (or concurrently if do_stuff() is a thread) and
do_stuff() can continue to make use of the handle?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Accessing clipboard through software built on Python

[Tim Daneliuk]

On 10/28/2018 02:08 PM, Akkana Peck wrote:
> Tim Daneliuk writes:
>> However, the highlighted text must be copied explicitly:
>>
>> Highlight
>> Ctl-C
> [ ... ]
>> X actually has several clipboard buffers and it can be tricky to get this 
>> going.  I don't recall,
>> but either clipboard or pyperclip have a way to get to them all IIRC ...
> 
> To get the highlighted text in X without needing a copy, you want the
> PRIMARY X selection rather than CLIPBOARD.
> 
> I think pyperclip can get it (it uses an external program like xsel,
> and xsel can get any of the three X selections), or you can get the
> selection using a GUI library like GTK, Qt or Tk. Some examples:
> https://github.com/akkana/scripts/blob/master/pyclip
> 
> ...Akkana
> 


Yes, that sounds right in the distant recesses of my memory :)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Accessing clipboard through software built on Python

[Tim Daneliuk]

On 10/28/2018 02:08 PM, Akkana Peck wrote:
> Tim Daneliuk writes:
>> However, the highlighted text must be copied explicitly:
>>
>> Highlight
>> Ctl-C
> [ ... ]
>> X actually has several clipboard buffers and it can be tricky to get this 
>> going.  I don't recall,
>> but either clipboard or pyperclip have a way to get to them all IIRC ...
> 
> To get the highlighted text in X without needing a copy, you want the
> PRIMARY X selection rather than CLIPBOARD.
> 
> I think pyperclip can get it (it uses an external program like xsel,
> and xsel can get any of the three X selections), or you can get the
> selection using a GUI library like GTK, Qt or Tk. Some examples:
> https://github.com/akkana/scripts/blob/master/pyclip
> 
> ...Akkana
> 


Yes, that sounds right in the distant recesses of my memory :)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Accessing clipboard through software built on Python

[Tim Daneliuk]

On 10/27/2018 08:17 AM, Musatov wrote:
> I am wondering if Python could be used to write a program that allows:
> 
> 1. Highlight some text
> 2. Ctl+HOTKEY1 stores the string of text somewhere as COPIEDTEXT1
> 3. Highlight another string of text
> 4. Ctl+HOTKEY1 stores another string of text somewhere as COPIEDTEXT2
> 
> THEN
> 
> 5. Ctl+HOTKEY2 pastes COPIEDTEXT1
> 6. Ctl+HOTKEY2 pastes COPIEDTEXT2
> 
> I found "pyperclip" and "Tkinter" but I don't know where to start.
> 
> Thanks,
> 
> Musatov
> 


I am able to do this with clipboard (pip install clipboard).


However, the highlighted text must be copied explicitly:

Highlight
Ctl-C

In Python:

TEXT1 = clipboard.paste()

Highlight again

TEXT2 = clipboard.paste()

X actually has several clipboard buffers and it can be tricky to get this 
going.  I don't recall,
but either clipboard or pyperclip have a way to get to them all IIRC ...

HTH,
-Tim

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ESR "Waning of Python" post

[Tim Daneliuk]

On 10/12/2018 11:43 AM, Skip Montanaro wrote:
> I sort of skimmed ESR's post, and sort of skimmed this thread, so
> obviously I'm totally qualified to offer my observations on the post
> and follow ups. :-)

Skip -

In the 15-ish years I've been reading this group, this has NEVER been
an obstacle for posters :P

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ESR "Waning of Python" post

[Tim Daneliuk]

On 10/11/2018 12:15 AM, Gregory Ewing wrote:
> Paul Rubin wrote [concerning GIL removal]:
>> It's weird that Python's designers were willing to mess up the user
>> language in the 2-to-3 transition but felt that the C API had to be kept
>> sarcosanct.  Huge opportunities were blown at multiple levels.
> 
> You say that as though we had a solution for GIL removal all
> thought out and ready to go, and the only thing that stopped us
> is that it would have required changing the C API.
> 
> But it's not like that at all. As far as I know, all the
> attempts that have been made so far to remove the GIL have
> led to performance that was less than satisfactory. It's a
> hard problem that we haven't found a good solution to yet.
> 


Do you happen to have a reference that explains what the issues are
for GIL removal?


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Cult-like behaviour [was Re: Kindness]

[Tim Daneliuk]

On 07/14/2018 07:40 PM, Chris Angelico wrote:
> You'd better avoid most of JavaScript, C++, and most other languages,
> then. Every language feeps a little, and Python is definitely not as
> bad as some.

Point Of Order: C++ is one gigantic feep to be avoided at all costs... :)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Cult-like behaviour [was Re: Kindness]

[Tim Daneliuk]

On 07/14/2018 04:09 AM, Christian Gollwitzer wrote:
> I agree with this observation and it feels quite strange to me. I regularly 
> use three languages (C++, Python and Tcl), all three are under active 
> development, and IMHO all of them have flaws, there are is always something 
> which is elegantly solved in one system but needs more work in another.


Dusting off some of my musings written years ago but at least tangentially
related to all this:

   https://www.tundraware.com/TechnicalNotes/How-To-Pick-A-Programming-Language/
   https://www.tundraware.com/TechnicalNotes/Bullet/

I am not particularly enamored of the feeping creaturism that that infested
Python 3, but then again, I'm not required by law to use said feeping 
creatures...

I wish GvR well.  He's served this community magnificently and deserves far
better than he got, especially lately. := pedantry aside (and I am NOT a fan),
great things come from individual minds, not committees and I think it is
simply inarguable that GvR built a Very Great Thing.  So ... thanks ... and
go enjoy your life, sir, you've more than earned it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Guido van Rossum resigns as Python leader

[Tim Daneliuk]

On 07/14/2018 10:16 AM, Skip Montanaro wrote:
>> Is it irrational to wonder whether projects should be looking to migrate to
>> new languages? This kind of announcement makes me worry for the future.
> 
> Umm, yeah. The language is stable, widely used packages are stable.
> Guido actually has little involvement in the larger Python ecosystem.
> It's not like NumPy, Django, Pandas, Flask, PyPI, Conda, or other
> popular packages or subsystems built with/for Python are suddenly
> going to crumble because Guido is no longer BDFL.
> 
> But, by all means, if rewriting your applications in a different
> language floats your boat, go right ahead...
> 
> Skip
> 

+1
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP 526 - var annotations and the spirit of python

[Tim Daneliuk]

On 07/02/2018 06:22 PM, Gregory Ewing wrote:
> A
> truly good programmer will be able to learn about the language
> being used on the job.

Except that the current attempt is to use techniques like agile,
scrum, pair programming, and so forth to turn programming into
a factory activity.  High degrees of specialization are segmented
by architectural role (front end, back end, infrastructure,
DevOps ...), language, and even business unit.  In my view,
systems architecture, software design, and non functional
capabilities suffer thereby, but I am old and crabby :)

In particular, there is little interest in having programmers
learn on the job, only that they be as productive as possible
as fast they can.   Hiring specific languages skills - the theory
goes - means that the individual will be fluent in the entire
language ecosystem of libraries, tools, and so forth.  What gets
lost in this factory model is that fewer and fewer people are able
to stand back and ask, "Are we even using a good design, language,
toolkit, ..."

While it is true that a good programmer will pick up new things
out of personal curiosity, it is also true that this is not
as rewarded a behavior as we'd like to believe.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP 526 - var annotations and the spirit of python

[Tim Daneliuk]

On 07/01/2018 12:17 PM, MRAB wrote:
> On 2018-07-01 18:06, Abdur-Rahmaan Janhangeer wrote:
>> was viewing pep526, so, finally, python cannot do without hinting the type
>> as other languages?
>> will python finally move to
>> int x = 3 where int is a pre annotation?
>>
>> i am not arguing it's usefulness but rather, does it fit with python?
>>
> PEP 526 says that the annotation would be:
> 
> x: int = 3
> 
> It also says """It should also be emphasized that Python will remain a 
> dynamically typed language, and the authors have no desire to ever make type 
> hints mandatory, even by convention.""



This strikes me as syntactic noise.  Python is dynamically typed and will 
remain so.
Why clutter the language - even optionally - with stuff like this?


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thank you Python community!

[Tim Daneliuk]

On 03/19/2018 02:05 PM, bartc wrote:
> I've often wondered what the guys who invented C (around 1970) must have been 
> smoking to have come up with some of those ideas.

I dunno, but I do know that - if they were smoking something - it was
rolled in greenbar paper ...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python Templating Language

[Tim Daneliuk]

On 12/17/2017 12:41 AM, Abdur-Rahmaan Janhangeer wrote:
> Hi all,
> 
> Can somebody point out to me some py-based template languages interpreters
> resources?
> 
> Thank you !
> 

http://jinja.pocoo.org/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Old Man Yells At Cloud

[Tim Daneliuk]

On 09/16/2017 09:59 AM, Tim Daneliuk wrote:
> On 09/16/2017 12:38 AM, Steve D'Aprano wrote:
>> /rant on
>>
>> So apparently everyone who disagrees that Python should be more like 
>> Javascript
>> is an old greybeard fuddy-duddy yelling "Get off my lawn!" to the cool kids 
>> --
>> and is also too stupid to know how dumb they are.
>>
>> "Hi, I've been programming in Python for what seems like days now, and here's
>> all the things that you guys are doing wrong. I insist that you fix them
>> immediately, it doesn't matter how much code it will break, that's not
>> important. What is important is that Javascript programmers like me shouldn't
>> be expected to learn anything new or different when they program with 
>> Python."
>>
>> /rant off
>>
>> And no, for once it wasn't Ranting Rick.
>>
>>
>>
>>
> 
> Well, the whole integer floor division thing years ago was the beginning
> of the end - Python was doomed ...
> 


Ummm  I was KIDDING via hyperbole ... sheesh ...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Old Man Yells At Cloud

[Tim Daneliuk]

On 09/16/2017 12:38 AM, Steve D'Aprano wrote:
> /rant on
> 
> So apparently everyone who disagrees that Python should be more like 
> Javascript
> is an old greybeard fuddy-duddy yelling "Get off my lawn!" to the cool kids --
> and is also too stupid to know how dumb they are.
> 
> "Hi, I've been programming in Python for what seems like days now, and here's
> all the things that you guys are doing wrong. I insist that you fix them
> immediately, it doesn't matter how much code it will break, that's not
> important. What is important is that Javascript programmers like me shouldn't
> be expected to learn anything new or different when they program with Python."
> 
> /rant off
> 
> And no, for once it wasn't Ranting Rick.
> 
> 
> 
> 

Well, the whole integer floor division thing years ago was the beginning
of the end - Python was doomed ...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proposed new syntax

[Tim Daneliuk]

On 08/10/2017 09:28 AM, Steve D'Aprano wrote:
> Every few years, the following syntax comes up for discussion, with some 
> people
> saying it isn't obvious what it would do, and others disagreeing and saying
> that it is obvious. So I thought I'd do an informal survey.
> 
> What would you expect this syntax to return?
> 
> [x + 1 for x in (0, 1, 2, 999, 3, 4) while x < 5]

[1,2,3]

> 
> 
> For comparison, what would you expect this to return? (Without actually trying
> it, thank you.)
> 
> [x + 1 for x in (0, 1, 2, 999, 3, 4) if x < 5]

[1,2,3,4,5,]

> 
> 
> 
> How about these?
> 
> [x + y for x in (0, 1, 2, 999, 3, 4) while x < 5 for y in (100, 200)]

[100,200,101,201,102,202]

> 
> [x + y for x in (0, 1, 2, 999, 3, 4) if x < 5 for y in (100, 200)]


[100,200,101,201,102,202.103,203,104,204
]
> 
> 
> 
> Thanks for your comments!
> 
> 
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question About When Objects Are Destroyed (continued)

[Tim Daneliuk]

On 08/05/2017 05:36 PM, Ned Batchelder wrote:
> On 8/5/17 5:41 PM, Tim Daneliuk wrote:
>> On 08/05/2017 11:16 AM, Ned Batchelder wrote:
>>> It uses
>>> reference counting, so most objects are reclaimed immediately when their
>>> reference count goes to zero, such as at the end of local scopes. 
>> Given this code:
>>
>> class SomeObject:
>> .
>>
>>
>> for foo in somelist:
>>
>>a = SomeObject(foo)
>>b = SomeObject(foo)
>>c = SomeObject(foo)
>>
>># Do something or other
>>...
>>
>># Bottom of 'for' scope
>>
>>
>> Are you saying that each time a,b,c are reassigned to new instances of
>> SomeObject the old instance counts go to 0 and are immediately - as in
>> synchronously, right now, on the spot - removed from memory?  
> Yes, that is what I am saying.  In CPython, that is.  Other
> implementation can behave differently. Jython and IronPython use the
> garbage collectors from the JVM and .net, I don't know specifically how
> they behave.
>> My
>> understanding was (and I may well be wrong), that the reference count
>> does get decremented - in this case to 0 - but the *detection* of that
>> fact does not happen until the gc sweep looks through the heap for such
>> stale objects.
> That is how classic garbage collectors worked.  And Python has something
> like that, but it's only used to collect circular structures, where the
> reference counts will never go to zero, but nevertheless the entire
> structure can be unreferenced as a whole.
> 
> --Ned.
> 


Interesting.  I haz a confuzed.  Thanks for clearing that up.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question About When Objects Are Destroyed (continued)

[Tim Daneliuk]

On 08/05/2017 05:36 PM, Ned Batchelder wrote:
> On 8/5/17 5:41 PM, Tim Daneliuk wrote:
>> On 08/05/2017 11:16 AM, Ned Batchelder wrote:
>>> It uses
>>> reference counting, so most objects are reclaimed immediately when their
>>> reference count goes to zero, such as at the end of local scopes. 
>> Given this code:
>>
>> class SomeObject:
>> .
>>
>>
>> for foo in somelist:
>>
>>a = SomeObject(foo)
>>b = SomeObject(foo)
>>c = SomeObject(foo)
>>
>># Do something or other
>>...
>>
>># Bottom of 'for' scope
>>
>>
>> Are you saying that each time a,b,c are reassigned to new instances of
>> SomeObject the old instance counts go to 0 and are immediately - as in
>> synchronously, right now, on the spot - removed from memory?  
> Yes, that is what I am saying.  In CPython, that is.  Other
> implementation can behave differently. Jython and IronPython use the
> garbage collectors from the JVM and .net, I don't know specifically how
> they behave.
>> My
>> understanding was (and I may well be wrong), that the reference count
>> does get decremented - in this case to 0 - but the *detection* of that
>> fact does not happen until the gc sweep looks through the heap for such
>> stale objects.
> That is how classic garbage collectors worked.  And Python has something
> like that, but it's only used to collect circular structures, where the
> reference counts will never go to zero, but nevertheless the entire
> structure can be unreferenced as a whole.
> 
> --Ned.
> 


Interesting.  I haz a confuzed.  Thanks for clearing that up.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question About When Objects Are Destroyed (continued)

[Tim Daneliuk]

On 08/05/2017 05:58 PM, Chris Angelico wrote:
> On Sun, Aug 6, 2017 at 7:32 AM, Tim Daneliuk <i...@tundraware.com> wrote:
>> On 08/05/2017 03:21 PM, Chris Angelico wrote:
>>> After a 'with' block,
>>> the object *still exists*, but it has been "exited" in some way
>>> (usually by closing/releasing an underlying resource).
>>
>> The containing object exists, but the things that the closing
>> logic explicitly released do not.  In some sense, a context
>> acts like a deconstructor, just not on the object it's associated
>> with.
>>
>>> If there's a resource you need to clean up, you clean that up
>>> explicitly,
>>
>> Such "resources" *are* objects themselves notionally.  You are exactly
>> killing those objects to free the underlying resources they consume.
> 
> Utterly irrelevant. The original post was about the string in memory.
> An "open file" is no more an object than the part of a floating point
> number after the decimal is.
> 
>>> so the object's lifetime shouldn't matter to you.
>>
>> I disagree with this most strongly.  That's only true when the machine
>> resources being consumed by your Python object are small in size.  But
>> when you're dynamically cranking out millions of objects of relatively
>> short lifetime, you can easily bump into the real world limits of
>> practical machinery.  "Wait until the reference count sweep gets rid of
>> it" only works when you have plenty of room to squander.
>>
>> Also, waiting for the reference count/gc to do its thing is
>> nondeterministic in time.  It's going to happen sooner or later, but not
>> at the same or a predictable interval.  If you want to write large,
>> performant code, you don't want this kind of variability.  While I
>> realize that we're not typically writing embedded realtime drivers in
>> Python, the principle remains - where possible make things as
>> predictable and repeatable as you can.
>>
>> For reasons I am not free discuss here, I can say with some assurance
>> that there are real world applications where managing Python object
>> lifetimes is very much indicated.
> 
> Very VERY few. How often do you actually care about the lifetime of a
> specific Python object, and not (say) about the return of a block of
> memory to the OS? Memory in CPython is allocated in pages, and those
> pages are then suballocated into objects (or other uses). Sometimes
> you care about that block going back to the OS; other times, all you
> care about is that a subsequent allocation won't require more memory
> (which can be handled with free lists). But most of the time, you
> don't need to think about either, because the language *does the right
> thing*. The nondeterminism of the GC is irrelevant to most Python
> programs; in CPython, that GC sweep applies only to reference *cycles*
> (and to weak references, I think??), so unless you frequently create
> those, you shouldn't have to care.
> 
> I've written plenty of large programs in high level languages. Some of
> them in Python, some in Pike (which has the same refcount semantics),
> and some in REXX (which has very different technical semantics but
> comes to the same thing). I've had those programs running for months
> on end; in more than one instance, I've had a program running for over
> a year (over two years, even) without restarting the process or
> anything. Aside from taking care not to create cyclic references, I
> have not needed to care about when the garbage collector runs, with
> the sole exception of an instance where I built my own system on top
> of the base GC (using weak references and an autoloader to emulate a
> lookup table larger than memory). So yes, I maintain that most of the
> time, object lifetimes *should not matter* to a Python programmer.
> Python is not C, and you shouldn't treat it as C.
> 
> ChrisA
> 


OK, noted, and thanks for the clear explanation.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question About When Objects Are Destroyed (continued)

[Tim Daneliuk]

On 08/05/2017 05:58 PM, Chris Angelico wrote:
> On Sun, Aug 6, 2017 at 7:32 AM, Tim Daneliuk <i...@tundraware.com> wrote:
>> On 08/05/2017 03:21 PM, Chris Angelico wrote:
>>> After a 'with' block,
>>> the object *still exists*, but it has been "exited" in some way
>>> (usually by closing/releasing an underlying resource).
>>
>> The containing object exists, but the things that the closing
>> logic explicitly released do not.  In some sense, a context
>> acts like a deconstructor, just not on the object it's associated
>> with.
>>
>>> If there's a resource you need to clean up, you clean that up
>>> explicitly,
>>
>> Such "resources" *are* objects themselves notionally.  You are exactly
>> killing those objects to free the underlying resources they consume.
> 
> Utterly irrelevant. The original post was about the string in memory.
> An "open file" is no more an object than the part of a floating point
> number after the decimal is.
> 
>>> so the object's lifetime shouldn't matter to you.
>>
>> I disagree with this most strongly.  That's only true when the machine
>> resources being consumed by your Python object are small in size.  But
>> when you're dynamically cranking out millions of objects of relatively
>> short lifetime, you can easily bump into the real world limits of
>> practical machinery.  "Wait until the reference count sweep gets rid of
>> it" only works when you have plenty of room to squander.
>>
>> Also, waiting for the reference count/gc to do its thing is
>> nondeterministic in time.  It's going to happen sooner or later, but not
>> at the same or a predictable interval.  If you want to write large,
>> performant code, you don't want this kind of variability.  While I
>> realize that we're not typically writing embedded realtime drivers in
>> Python, the principle remains - where possible make things as
>> predictable and repeatable as you can.
>>
>> For reasons I am not free discuss here, I can say with some assurance
>> that there are real world applications where managing Python object
>> lifetimes is very much indicated.
> 
> Very VERY few. How often do you actually care about the lifetime of a
> specific Python object, and not (say) about the return of a block of
> memory to the OS? Memory in CPython is allocated in pages, and those
> pages are then suballocated into objects (or other uses). Sometimes
> you care about that block going back to the OS; other times, all you
> care about is that a subsequent allocation won't require more memory
> (which can be handled with free lists). But most of the time, you
> don't need to think about either, because the language *does the right
> thing*. The nondeterminism of the GC is irrelevant to most Python
> programs; in CPython, that GC sweep applies only to reference *cycles*
> (and to weak references, I think??), so unless you frequently create
> those, you shouldn't have to care.
> 
> I've written plenty of large programs in high level languages. Some of
> them in Python, some in Pike (which has the same refcount semantics),
> and some in REXX (which has very different technical semantics but
> comes to the same thing). I've had those programs running for months
> on end; in more than one instance, I've had a program running for over
> a year (over two years, even) without restarting the process or
> anything. Aside from taking care not to create cyclic references, I
> have not needed to care about when the garbage collector runs, with
> the sole exception of an instance where I built my own system on top
> of the base GC (using weak references and an autoloader to emulate a
> lookup table larger than memory). So yes, I maintain that most of the
> time, object lifetimes *should not matter* to a Python programmer.
> Python is not C, and you shouldn't treat it as C.
> 
> ChrisA
> 


OK, noted, and thanks for the clear explanation.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question About When Objects Are Destroyed (continued)

[Tim Daneliuk]

On 08/05/2017 11:16 AM, Ned Batchelder wrote:
> It uses
> reference counting, so most objects are reclaimed immediately when their
> reference count goes to zero, such as at the end of local scopes. 

Given this code:

class SomeObject:
.


for foo in somelist:

   a = SomeObject(foo)
   b = SomeObject(foo)
   c = SomeObject(foo)

   # Do something or other
   ...

   # Bottom of 'for' scope


Are you saying that each time a,b,c are reassigned to new instances of
SomeObject the old instance counts go to 0 and are immediately - as in
synchronously, right now, on the spot - removed from memory?  My
understanding was (and I may well be wrong), that the reference count
does get decremented - in this case to 0 - but the *detection* of that
fact does not happen until the gc sweep looks through the heap for such
stale objects.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question About When Objects Are Destroyed (continued)

[Tim Daneliuk]

On 08/05/2017 03:21 PM, Chris Angelico wrote:
> After a 'with' block,
> the object *still exists*, but it has been "exited" in some way
> (usually by closing/releasing an underlying resource).

The containing object exists, but the things that the closing
logic explicitly released do not.  In some sense, a context
acts like a deconstructor, just not on the object it's associated
with.


> If there's a resource you need to clean up, you clean that up
> explicitly,

Such "resources" *are* objects themselves notionally.  You are exactly
killing those objects to free the underlying resources they consume.

> so the object's lifetime shouldn't matter to you.

I disagree with this most strongly.  That's only true when the machine
resources being consumed by your Python object are small in size.  But
when you're dynamically cranking out millions of objects of relatively
short lifetime, you can easily bump into the real world limits of
practical machinery.  "Wait until the reference count sweep gets rid of
it" only works when you have plenty of room to squander.

Also, waiting for the reference count/gc to do its thing is
nondeterministic in time.  It's going to happen sooner or later, but not
at the same or a predictable interval.  If you want to write large,
performant code, you don't want this kind of variability.  While I
realize that we're not typically writing embedded realtime drivers in
Python, the principle remains - where possible make things as
predictable and repeatable as you can.

For reasons I am not free discuss here, I can say with some assurance
that there are real world applications where managing Python object
lifetimes is very much indicated.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question About When Objects Are Destroyed (continued)

[Tim Daneliuk]

On 08/04/2017 07:00 PM, Chris Angelico wrote:
> Again, don't stress about exactly when objects get
> disposed of; it doesn't matter.


Respectfully, I disagree strongly.  Objects get build on the heap and
persist even when they go out of scope until such time garbage
collection takes place.  This is unlike languages that build things in
stack frames which naturally disappear with an exit of scope.

For small or trivial programs, it does not matter.  But when there is a
lot of dynamic object construction - say, in very large programs, object
factories, etc. - it can be important to harvest the space of expired
objects sooner, rather than later.  This, after all, is one of the
rationale' for Python contexts - to ensure the release of resources no
matter how the logic ends - correctly or by exception.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to fast processing one million strings to remove quotes

[Tim Daneliuk]

On 08/04/2017 01:52 AM, Peter Otten wrote:


> It looks like Python is fairly competetive:
> 
> $ wc -l hugequote.txt 
> 100 hugequote.txt612250
> $ cat unquote.py 
> import csv
> 
> with open("hugequote.txt") as instream:
> for field, in csv.reader(instream):
> print(field)
> 
> $ time python3 unquote.py > /dev/null
> 
> real0m3.773s
> user0m3.665s
> sys 0m0.082s
> 
> $ time cat hugequote.txt | sed 's/"""/"/g;s/""/"/g' > /dev/null
> 
> real0m4.862s
> user0m4.721s
> sys 0m0.330s
> 
> Run on ancient AMD hardware ;)
> 

It's actually better than sed.  What you're seeing is - I believe -
load time dominating the overall time.  I reran this with a 20M line
file:

time cat superhuge.txt | sed 's/"""/"/g;s/""/"/g' >/dev/null

real0m53.091s
user0m52.861s
sys 0m0.820s


time python unquote.py >/dev/null

real0m22.377s
user0m22.021s
sys 0m0.352s

Note that this is with python2, not python3.  Also, I confirmed that the
cat and pipe into sed was not a factor in the performance.

My guess is that delimiter recognition logic in the csv module is far
more efficient than the general purpose regular expression/dfa
implementaton in sed.

Extra Credit Assignment:

Reimplement in python using:

- string substitution
- regular expressions


Tschüss...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to fast processing one million strings to remove quotes

[Tim Daneliuk]

On 08/02/2017 10:05 AM, Daiyue Weng wrote:
> Hi, I am trying to removing extra quotes from a large set of strings (a
> list of strings), so for each original string, it looks like,
> 
> """str_value1"",""str_value2"",""str_value3"",1,""str_value4"""
> 
> 
> I like to remove the start and end quotes and extra pairs of quotes on each
> string value, so the result will look like,
> 
> "str_value1","str_value2","str_value3",1,"str_value4"



This part can also be done fairly efficiently with sed:

time cat hugequote.txt | sed 's/"""/"/g;s/""/"/g' >/dev/null

real0m2.660s
user0m2.635s
sys 0m0.055s

hugequote.txt is a file with 1M copies of your test string above in it.

Run on a quad core i5 on FreeBSD 10.3-STABLE.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Development testing without reinstalling egg constantly?

[Tim Daneliuk]

On 06/29/2017 09:03 AM, Grant Edwards wrote:
> I've forked a copy of https://github.com/Roguelazer/muttdown and have
> been adding a few features and fixing a few bugs.  It's meant to be


When doing this sort of thing, I find 'pew' virtual environments immensely
helpful.  They allow you to control python configs and pip loads in userland
without having to fiddle with the system versions of things.

Highly recommended:

https://github.com/berdario/pew
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Development testing without reinstalling egg constantly?

[Tim Daneliuk]

On 06/29/2017 09:03 AM, Grant Edwards wrote:
> I've forked a copy of https://github.com/Roguelazer/muttdown and have
> been adding a few features and fixing a few bugs.  It's meant to be


When doing this sort of thing, I find 'pew' virtual environments immensely
helpful.  They allow you to control python configs and pip loads in userland
without having to fiddle with the system versions of things.

Highly recommended:

https://github.com/berdario/pew
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How coding in Python is bad for you

[Tim Daneliuk]

On 01/23/2017 02:19 PM, Chris Angelico wrote:
> On Tue, Jan 24, 2017 at 6:59 AM, Grant Edwards
>  wrote:
>> On 2017-01-23, breamore...@gmail.com  wrote:
>>
>>> The article is here http://lenkaspace.net/index.php/blog/show/111
>>
>> I don't really think any of his points are valid, but one way that
>> programming in Python is bad for you:
>>
>>  * It reduces your tolerance for progamming in PHP zero.  If you end
>>up assigned to a PHP project, you end up miserable and unpleasant
>>to work with or just plain unemployed.
> 
> I believe that's "bad for you" in the sense that chocolate is bad for you.
> 
> It isn't.
> 
> ChrisA
> 


It's bad for you in the same sense tha chocolate is bad for you - if you
melt it and pour it into your ears...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How coding in Python is bad for you

[Tim Daneliuk]

On 01/23/2017 11:24 AM, breamore...@gmail.com wrote:
> The article is here http://lenkaspace.net/index.php/blog/show/111
> 
> Kindest regards.
> 
> Mark Lawrence.
> 

Beyond silly.  Languages - like all tools - can be used properly or badly.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Grumpy: Python to Go compiler

[Tim Daneliuk]

On 01/08/2017 06:18 PM, Deborah Swanson wrote:
> (haha, unless
> you ask)

C'mon, go for it ... there hasn't been a good rant here in
4 or 5 minutes ...


-- 
https://mail.python.org/mailman/listinfo/python-list

[ANN] 'tsshbatch' Server Automation Tool Version 1.317 Released

[Tim Daneliuk]

'tsshbatch' Version 1.317 is now released and available for download at:

 http://www.tundraware.com/Software/tsshbatch

This is a major update with important bug fixes and improvements.  Existing
users will want to update sooner rather than later.

The last public release was 1.228.
-

What Is 'tsshbatch'?


'tsshbatch' is a server automation tool to enable you to issue commands
to many servers without having to log into each one separately.  When
writing scripts, this overcomes the 'ssh' limitation of not being able to
specify the password on the command line.

'tsshbatch' has a sophisticated inventory management system (host
files) as well as a mechanism for organizing libraries of standard jobs 
(command files).  Both local and global variables may be defined and/or
used in either.

'tsshbatch' also understands basic 'sudo' syntax and can be used
to access a server, 'sudo' a command, and then exit.

'tsshbatch' thus allows you to write complex, hands-off scripts that
issue commands to many servers without the tedium of manual login and
'sudo' promotion.  System administrators, especially, will find this
helpful when working in large server farms.

'tsshbatch' is written in Python and requires the 'paramiko library.
It has been tested on a number of Linux and FreeBSD variants.

See Also: https://en.wikipedia.org/wiki/Tsshbatch

Related: Ansible, Capistrano, ClusterSSH, Fabric, PSSH, Rundeck 


WHATSNEW For 'tsshbatch' 1.317(Sat Oct 15 16:12:34 CDT 2016)
--

  - It is now possible to define local variables with the
.local directive.  These have visibility and scope only
with the file where they are defined, but otherwise work
the same as globally defined variables.  Local variables
also support "execution variable" style definition.

  - Hostfile names must now be passed using the -i option.  The
argument can be the name of a single file or a quoted list of
files.  The option can appear on the command line more than once.
-H and -i can be used together to create custom host lists.

  - The -L option will list all (if any) host- and command files found
on their respective search paths.

  - The -W option will write out the inventory of hosts that would be
processed if you actually executed the program, and then
terminates.  This works only in test mode.  This allows you
to embed tsshbatch in external shell scripts like this:

   for server in $(tsshbatch.py -i devserverlist -uatserverlist -W)
   do
 ssh $server
   done

Why?  Because tsshbatch has lots of powerful ways to maintain
inventories of hosts and combine them through includes and
multiple command line arguments.  The -W option makes it
convenient for external programs to make use of those inventory
features.

  - The -F "string ..."  option will examine every file on the host-
or command paths, looking for matching strings within these files.
Matches will report the file name, the location within the file,
and the line containing any of the specified strings.

This is a simple, case-insensitive string literal match and does
not support regular expressions.

This is handy when you're looking for a host name or command
string, say like, "sudo", and you don't want to have to manually
go through all your support files.

  - The -V "string ..." option does the exact opposite of -F.  It
lists all the files that do NOT contain any of the specified
strings.

  - The -r option has been added to suppress reporting of start/stop
statistics.  This allows you to make statistics reporting the
default, say via the $TSSHBATCH environment variable, but
override it when you need to.

  - A new directive, .notify, tells tsshbatch to print an informative
message from within a command file.  This is a runtime activity
and is helpful, for example, when tracking progress of a long
command file. Notifications are disabled in silent mode.

[CHANGES]

  - The -H option can now appear on the command line multiple times
thereby creating an aggregate list of all hosts named therein.

  - Hostfiles must now be passed as an argument of -i.  This was
done to provide a consistent way of passing multiple host
files on the commandline.

  - .include targets (file name specifications) may now reference
previously defined variables.


[BUG FIXES]

  - File transfers now properly honor the -s (silent output) flag.


--------
Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/


-- 
https://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/

ANN: 'tsshbatch' Server Automation Tool Version 1.228 Released

[Tim Daneliuk]

'tsshbatch' Version 1.228 is now released and available for download at:

 http://www.tundraware.com/Software/tsshbatch

This is a major update with important bug fixes and improvements.  Existing
users will want to update sooner rather than later.

The last public release was 1.212.
-

What Is 'tsshbatch'?


'tsshbatch' is a server automation tool to enable you to issue commands
to many servers without having to log into each one separately.  When
writing scripts, this overcomes the 'ssh' limitation of not being able to
specify the password on the command line.

'tsshbatch' also understands basic 'sudo' syntax and can be used
to access a server, 'sudo' a command, and then exit.

'tsshbatch' thus allows you to write complex, hands-off scripts that
issue commands to many servers without the tedium of manual login and
'sudo' promotion.  System administrators, especially, will find this
helpful when working in large server farms.

'tsshbatch' is written in Python and requires the 'paramiko library.
It has been tested on various Linux and FreeBSD variants.

See Also: https://en.wikipedia.org/wiki/Tsshbatch

Related: Ansible, Capistrano, ClusterSSH, Fabric, PSSH, Rundeck 


WHATSNEW For 'tsshbatch' 1.228(Mon Jan 18 17:45:19 CST 2016)
--

- There is now limited support for ssh configuration files. Only the
  HostName and IdentityFile directives are currently supported.

  By default, tsshbatch will look in ~/.ssh/config for this
  configuration file. However, the location of the file can be
  overridden with the -C option.

- The -b option has been added to continue after a sudo failure.
  Previous releases of the program stopped all further processing
  on any sudo failure.  With -b, it's now possible to go on to
  the remaining hosts even if one of them failed to do proper
  sudo promotion.

- The -B option has been added to print an informative "banner"
  at the beginning and end of each program run.


[CHANGES]

- On screen help now displays default settings for all options where
  appropriate.

[BUG FIXES]

- Fixed bug that caused program to exit after a failed
  file transfer even when -a was specified.

- Fixed bug that failed to present user name during key-based
  auth.  This prevented connection when the desired name
  was different than the initating user - say when using
  process IDs instead of "real" users.


--------
Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/


-- 
https://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/

Re: More tkinter Madness

[Tim Daneliuk]

On 11/13/2015 12:32 AM, Christian Gollwitzer wrote:
> Apfelkiste:Sources chris$

Well, I get window and when I do this:

pack [button .b -text Hello -command exit]

Nothing appears.

tkinter appears borked

I have reinstalled once already, will try again


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: More tkinter Madness

[Tim Daneliuk]

On 11/13/2015 01:58 PM, Michael Torrie wrote:
> On 11/13/2015 12:14 PM, Tim Daneliuk wrote:
>> On 11/13/2015 12:32 AM, Christian Gollwitzer wrote:
>>> Apfelkiste:Sources chris$
>>
>> Well, I get window and when I do this:
>>
>> pack [button .b -text Hello -command exit]
>>
>> Nothing appears.
>>
>> tkinter appears borked
>>
>> I have reinstalled once already, will try again
> 
> Tkinter is the name of the Python package for using Tk, but Tk itself is
> usually called Tk (package on Fedora and RHEL is tk).
> 
> On my CentOS installs, in /usr/share/tk8.5/demos, there are a number of
> Tcl/Tk demos you can run.  If Tcl and Tk are working, then I would move
> to reinstalling tkinter.
> 

Yep, tcl/tk is borked.  That would explain why pure X apps like xterm work, but 
not tkinter apps.
Reinstalling tcl/tk has not helped at all.  Now I am really curious what the 
interaction is between VPS and tk... Sigh. 


(Thanks)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: More tkinter Madness

[Tim Daneliuk]

On 11/13/2015 03:30 PM, Christian Gollwitzer wrote:
> Am 13.11.15 um 20:14 schrieb Tim Daneliuk:
>> On 11/13/2015 12:32 AM, Christian Gollwitzer wrote:
>>> Apfelkiste:Sources chris$
>>
>> Well, I get window and when I do this:
>>
>> pack [button .b -text Hello -command exit]
>>
>> Nothing appears.
> 
> No error, nothing? Just to be sure, you haven't closed the empty window, that 
> appeared when you typed "wish"? and copied the command into the wish prompt?
> 
>> tkinter appears borked
>> I have reinstalled once already, will try again
> 
> This is using pure Tcl/Tk. If it is not working, reinstall the corresponding 
> packages in your distro. tkinter is merely the Python interface to those.
> 
> Christian

Yep tcl/tk is the culprit, but reinstalling has not helped.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: More tkinter Madness

[Tim Daneliuk]

On 11/13/2015 01:56 PM, Laura Creighton wrote:
> In a message of Fri, 13 Nov 2015 13:14:08 -0600, Tim Daneliuk writes:
>> On 11/13/2015 12:32 AM, Christian Gollwitzer wrote:
>>> Apfelkiste:Sources chris$
>>
>> Well, I get window and when I do this:
>>
>> pack [button .b -text Hello -command exit]
>>
>> Nothing appears.
>>
>> tkinter appears borked
>>
>> I have reinstalled once already, will try again
> 
> Next idea.
> 
> Try to run a basic Tk app and a basic Tcl one.  See if they work or not.
> 
> Laura
> 

Looks like tcl/tk is hosed for some reason.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What is '@' for

[Tim Daneliuk]

On 11/13/2015 05:14 PM, Chris Angelico wrote:
> On Sat, Nov 14, 2015 at 10:04 AM, fl  wrote:
>> I read the following code snippet. A question is here about '@'.
>> I don't find the answer online yet.
>>
>> What function is it here?
>>
>>
>> @pymc.deterministic
>> def theta(a=alpha, b=beta):
>> """theta = logit^{-1}(a+b)"""
>> return pymc.invlogit(a+b*x)
> 
> That's called a "function decorator". And now that you know the name,
> you'll be able to find what it is online; as well as the Python docs,
> there are a number of blog posts and other articles about it.
> 
> ChrisA
> 


One small point of order ... if you want to be precise, "@pymc.deterministic" 
is a *function decoration*.  The "decorator" is the function 
pymc.deterministic().

I know this is a fussy point, but this distinction is helpful when first 
learning the concept.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: More tkinter Madness

[Tim Daneliuk]

On 11/11/2015 08:12 PM, Paul Rubin wrote:
> Tim Daneliuk <tundrabo...@tundraware.com> writes:
>> Some months ago, I put it on a couple of VPS servers (FreeBSD
>> and Linux) and BOOM, it doesn't run.  I asked around here and got some
>> suggestions and then did some homework.
> 
> I'd expect a VPS server to have no display--is it an X client forward to
> your workstation?  Are all the proper X libraries installed?  Note that
> X port forwarding over the internet is generally unusable even if your
> internet connection is pretty fast.  It's kind of ok over a LAN.
> 

twander is a very, very ligthweight file browser with minimal GUI load.
It's always worked well even with X forwarded over an ssh session over the
open internet.

I will further investigate the X libs matter.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: More tkinter Madness

[Tim Daneliuk]

On 11/11/2015 08:25 PM, Chris Angelico wrote:
> On Thu, Nov 12, 2015 at 12:52 PM, Tim Daneliuk
> <tundrabo...@tundraware.com> wrote:
>> I am the author of twander (https://www.tundraware.com/Software/twander).
>> This code has run flawlessly for years on FreeBSD, Linux, MacOS and
>> Windows.  Some months ago, I put it on a couple of VPS servers (FreeBSD
>> and Linux) and BOOM, it doesn't run.  I asked around here and got some
>> suggestions and then did some homework.
>>
>> Traceback (most recent call last):
>>   File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
> 
> It's running here under Python 2.6. Sorry if this is a question you've
> already been asked, but were you successfully running under 2.6
> elsewhere? Might be a versioning issue.
> 
> ChrisA
> 

It always ran OK on every previous version of Python from about 2.4 forward
on VMs, physicals, *nix, OSX, Windows, etc.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: More tkinter Madness

[Tim Daneliuk]

On 11/12/2015 10:46 PM, Michael Torrie wrote:
> On 11/12/2015 05:25 PM, Tim Daneliuk wrote:
>> On 11/11/2015 08:25 PM, Chris Angelico wrote:
>>> On Thu, Nov 12, 2015 at 12:52 PM, Tim Daneliuk
>>> <tundrabo...@tundraware.com> wrote:
>>>> I am the author of twander (https://www.tundraware.com/Software/twander).
>>>> This code has run flawlessly for years on FreeBSD, Linux, MacOS and
>>>> Windows.  Some months ago, I put it on a couple of VPS servers (FreeBSD
>>>> and Linux) and BOOM, it doesn't run.  I asked around here and got some
>>>> suggestions and then did some homework.
>>>>
>>>> Traceback (most recent call last):
>>>>   File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
>>>
>>> It's running here under Python 2.6. Sorry if this is a question you've
>>> already been asked, but were you successfully running under 2.6
>>> elsewhere? Might be a versioning issue.
>>>
>>> ChrisA
>>>
>>
>> It always ran OK on every previous version of Python from about 2.4 forward
>> on VMs, physicals, *nix, OSX, Windows, etc.
> 
> And other X11 apps are working fine on the test VPS where your app
> crashes? Other Tk apps?  Other Python Tk apps?
> 
> Your app runs fine on my VPS (Linode), running CentOS 7, Python 2.7.
> Also runs fine on my CentOS 6 VM (also Linode), Python 2.6.6.
> 
> I also verified that if X11 forwarding is not working, Python reports an
> exception "_tkinter.TclError: no display name and no $DISPLAY
> environment variable" immediately, so that's definitely not your problem.
> 
> 
> 

Curioser and curioser.  Yes, xterm works fine.  Dunno about any other tk apps.
H..
-- 
https://mail.python.org/mailman/listinfo/python-list

More tkinter Madness

[Tim Daneliuk]

I am the author of twander (https://www.tundraware.com/Software/twander).
This code has run flawlessly for years on FreeBSD, Linux, MacOS and
Windows.  Some months ago, I put it on a couple of VPS servers (FreeBSD
and Linux) and BOOM, it doesn't run.  I asked around here and got some
suggestions and then did some homework.

I see the error being thrown by using the trace module, but it's not 
terribly meaningful to me.  Any ideas of what this means - again,
I emphasize this is only happening on VPS hosts:

 --- modulename: Tkinter, funcname: _cnfmerge
Tkinter.py(76): if type(cnfs) is DictionaryType:
Tkinter.py(77): return cnfs
Tkinter.py(1046): res = ()
Tkinter.py(1047): for k, v in cnf.items():
Tkinter.py(1048): if v is not None:
Tkinter.py(1049): if k[-1] == '_': k = k[:-1]
Tkinter.py(1050): if callable(v):
Tkinter.py(1052): elif isinstance(v, (tuple, list)):
Tkinter.py(1064): res = res + ('-'+k, v)
Tkinter.py(1047): for k, v in cnf.items():
Tkinter.py(1048): if v is not None:
Tkinter.py(1049): if k[-1] == '_': k = k[:-1]
Tkinter.py(1050): if callable(v):
Tkinter.py(1052): elif isinstance(v, (tuple, list)):
Tkinter.py(1064): res = res + ('-'+k, v)
Tkinter.py(1047): for k, v in cnf.items():
Tkinter.py(1048): if v is not None:
Tkinter.py(1049): if k[-1] == '_': k = k[:-1]
Tkinter.py(1050): if callable(v):
Tkinter.py(1052): elif isinstance(v, (tuple, list)):
Tkinter.py(1064): res = res + ('-'+k, v)
Tkinter.py(1047): for k, v in cnf.items():
Tkinter.py(1065): return res
Traceback (most recent call last):
  File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
"__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
exec code in run_globals
  File "/usr/lib64/python2.6/trace.py", line 823, in 
main()
  File "/usr/lib64/python2.6/trace.py", line 811, in main
t.runctx(code, globs, globs)
  File "/usr/lib64/python2.6/trace.py", line 512, in runctx
exec cmd in globals, locals
  File "/local/TundraWare/bin/twander.py", line 5464, in 
UI = twanderUI(UIroot)
  File "/local/TundraWare/bin/twander.py", line 2152, in __init__
self.CmdBtn = Menubutton(self.mBar, text=COMMANDMENU, underline=0, 
state=DISABLED)
  File "/usr/lib64/python2.6/lib-tk/Tkinter.py", line 2710, in __init__
Widget.__init__(self, master, 'menubutton', cnf, kw)
  File "/usr/lib64/python2.6/lib-tk/Tkinter.py", line 1932, in __init__
(widgetName, self._w) + extra + self._options(cnf))
_tkinter.TclError
-- 
https://mail.python.org/mailman/listinfo/python-list

Are There Known Problems With tkinter And VPS Servers?

[Tim Daneliuk]

I am the author of https://www.tundraware.com/Software/twander, a cross 
platform, macro-
programmable file manager written in python/tkinter.

Of late, I am seeing core dumps of this program (which has been stable/mature 
for some
years) but only on VPS servers, both FreeBSD 10 and CentOS 6/7.

Is there are know problem here and/or can anyone suggest and approach to 
isolate the 
fault?

Thanks
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Are There Known Problems With tkinter And VPS Servers?

[Tim Daneliuk]

On 09/27/2015 04:20 PM, Tim Daneliuk wrote:
> I am the author of https://www.tundraware.com/Software/twander, a cross 
> platform, macro-
> programmable file manager written in python/tkinter.
> 
> Of late, I am seeing core dumps of this program (which has been stable/mature 
> for some
> years) but only on VPS servers, both FreeBSD 10 and CentOS 6/7.
> 
> Is there are know problem here and/or can anyone suggest and approach to 
> isolate the 
> fault?
> 
> Thanks
> 


Correction:  It dumps core on FreeBSD 10.2 but on CentOS 6 I get:

Traceback (most recent call last):
  File "/usr/local/TundraWare/bin/twander.py", line 5464, in 
UI = twanderUI(UIroot)
  File "/usr/local/TundraWare/bin/twander.py", line 2152, in __init__
self.CmdBtn = Menubutton(self.mBar, text=COMMANDMENU, underline=0, 
state=DISABLED)
  File "/usr/lib64/python2.6/lib-tk/Tkinter.py", line 2710, in __init__
Widget.__init__(self, master, 'menubutton', cnf, kw)
  File "/usr/lib64/python2.6/lib-tk/Tkinter.py", line 1932, in __init__
(widgetName, self._w) + extra + self._options(cnf))
_tkinter.TclError

This code runs fine on physical FreeBSD/CentOS/Debian servers, Rasperberry PIs 
running
Raspbian, OSX, VirtualBox and VMware Workstation Linux servers, Windows  so 
I suspect
this is somehow VPS related but not sure where to start.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Are There Known Problems With tkinter And VPS Servers?

[Tim Daneliuk]

On 09/27/2015 05:29 PM, Paul Rubin wrote:
> Tim Daneliuk <tun...@bogus-city.tundraware.com> writes:
>> this is somehow VPS related but not sure where to start.
> 
> How are you expecting tkinter to work on a vps, when there is no window
> system?  It wouldn't surprise me if tk is crashing.
> 

You may have heard about this thing called X Windows and this other thing called
ssh that easily permit VPS instances to run GUI code while displaying things
on a remote X server.


P.S. X applications like xterm work flawlessly on the hosts in question.

P.P.S.  You might want to dial down the snark, particularly since you seem
to be unfamiliar with the basics here.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Are There Known Problems With tkinter And VPS Servers?

[Tim Daneliuk]

On 09/27/2015 06:32 PM, Terry Reedy wrote:
> On 9/27/2015 5:31 PM, Tim Daneliuk wrote:
> 
>>> Of late, I am seeing core dumps of this program (which has been 
>>> stable/mature for some
>>> years) but only on VPS servers, both FreeBSD 10 and CentOS 6/7.
> 
>> Correction:  It dumps core on FreeBSD 10.2
> 
> Dumping core when there is no terminal server, instead of exiting gracefully, 
> might be considered a bug.
> 
>> but on CentOS 6 I get:
>>
>> Traceback (most recent call last):
>>File "/usr/local/TundraWare/bin/twander.py", line 5464, in 
>>  UI = twanderUI(UIroot)
>>File "/usr/local/TundraWare/bin/twander.py", line 2152, in __init__
>>  self.CmdBtn = Menubutton(self.mBar, text=COMMANDMENU, underline=0, 
>> state=DISABLED)
>>File "/usr/lib64/python2.6/lib-tk/Tkinter.py", line 2710, in __init__
>>  Widget.__init__(self, master, 'menubutton', cnf, kw)
>>File "/usr/lib64/python2.6/lib-tk/Tkinter.py", line 1932, in __init__
>>  (widgetName, self._w) + extra + self._options(cnf))
>> _tkinter.TclError
> 
> Beside Paul's comment, I suggest running with 2.7.latest and tk 8.6.latest if 
> at all possible to get tkinter and tk bug fixed.
> 
> Also, Menubutton is considered obsolete for Window level menus, so it may not 
> be maintained as well.  Just use Menu if at all possible.
> 

Unfortunately, CentOS tracks the corresponding RHEL component versions pretty 
much exactly (that's sort of the
point).  Moreover, this exact OS release level works flawlessly in other kinds 
of VMs and physical instances.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python, convert an integer into an index?

[Tim Daneliuk]

On 09/22/2015 04:43 PM, Chris Roberts wrote:
> 
> (How do I make it into an index? )
> Preferably something fairly easy to understand as I am new at this.
> 
> results = 134523  #(Integer) 
> 
> Desired: 
> results = [1, 2, 3, 4, 5, 2, 3]   #(INDEX)
> 
> Somehow I see ways to convert index to list to int, but not back again.
> 
> Thanks, 
> crzzy1
> 



results = [x for x in str(results)]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: RPI.GPIO Help

[Tim Daneliuk]

On 08/16/2015 02:40 PM, John McKenzie wrote:
> 
>  Hello, all. I am hoping some people here are familiar with the RPi.GPIO 
> python module for the Raspberry Pi.


I am not familiar with the module, but I am quite familiar with dealing
with hardware interfacing, mostly in assembler.

One thing you need to ensure is that either the module or your
code implents software debounce.  When a button is pressed or switched moved,
there is mechanical bounce for a some period of time immediately thereafter.
In the time right after the switch/button is pressed, the mechanical closure
"bounces" between "on" and "off" for a little while, until it settles down
to the position you've selected.

The idea is to use the initial change of state to initiate the button handling 
logic but
wait until the switch/button settles down to it's final (and therefore, 
reliable) state. 

One algorithm to do this is to wait a fixed amount of time after the initial 
event - say,
100ms - before reading the switch.  The other way to do is to read the 
button/switch state
ever few milliseconds and only accept the input if it has not changed in, say,
5 measurements.

HTH,

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ah Python, you have spoiled me for all other languages

[Tim Daneliuk]

On 05/22/2015 11:11 PM, amber wrote:
 «»
 
 On 22/05/2015 21:40, Tim Daneliuk wrote:
https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
 Quoting that article
 «And no, you couldn't get a C based OS to do what TPF does even if you
 did have a couple hundred million dollars to redo it, »
 
 Why couldn't a C based OS do what TPF does?
 
 My understanding is that if the programming language is Turing-Complete,
 it can do anything that any other Turing-Complete language can do.  And
 that assembler is a Turing-Complete language.
 
 jonathon
 
 
 
 


C could - in principle - do it.  As you point out, both are Turing Complete.
But, in principle is quite different than in actual fact.  At the time
that article was written almost 15 years ago, the TPF systems in use
were running applications written in hand optimized assembly language on the
largest mainframes money could buy.  Everything about that environment was
optimized for speed - from partially populating disk surfaces to running the
thinnest possible operating system executive.  (In those days, TPF itself 
didn't 
even have memory protection, or at least, it was just being added as an OS
feature.)

The point is that it would have been nearly impossible technically to
rewrite all that in C and expect the same level of minimal path length/
max throughput.  Give the sheer transaction load on those machines, this
would have been a showstopper.

But that was then and this is now.  These days, much of that capability 
HAS been written in higher level languages because of scale-out architectures
and messaging/queuing middleware.  Instead of one very fast centralized 
cluster of mainframes, we see thousands of servers spreading the workload
and achieving much the same results.  This has it's own challenges.  Managing
a few things is harder than managing thousands of things. 

-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ah Python, you have spoiled me for all other languages

[Tim Daneliuk]

On 05/22/2015 08:54 PM, Terry Reedy wrote:
 On 5/22/2015 5:40 PM, Tim Daneliuk wrote:
 
 Lo these many years ago, I argued that Python is a whole lot more than
 a programming language:

 https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
 
 Perhaps something at tundraware needs updating.
 '''
 This Connection is Untrusted
 
 You have asked Firefox to connect securely to www.tundraware.com, but we 
 can't confirm that your connection is secure.
 
 Normally, when you try to connect securely, sites will present trusted 
 identification to prove that you are going to the right place. However, this 
 site's identity can't be verified.
 '''
 

It's self signed - something done quite routinely on the net.

-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ah Python, you have spoiled me for all other languages

[Tim Daneliuk]

On 05/22/2015 11:49 PM, Chris Angelico wrote:
 When the information you're sharing is completely public,
 there's no point taking the overhead of encryption.


I disagree.  With two different ways to access data, the metadata about
when you do- and do not use an encrypted channel can be useful to
a snoopy third party.  For example, repressive governments might use
the fact of your connecting via https as a prima facie evidence you're
doing something subversive.  The argument for https everywhere is that this
sort of distinction becomes impossible to make and one less piece of metadata
is left around to misuse.


-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ah Python, you have spoiled me for all other languages

[Tim Daneliuk]

On 05/23/2015 01:55 AM, Johannes Bauer wrote:
 On 23.05.2015 05:31, Michael Torrie wrote:
 
 Sigh. I blame this as much on the browser.  There's no inherent reason
 why a connection to a site secured with a self-signed certificate is
 insecure.
 
 The problem is *not* that the certificate is self-signed.
 
 It's that it's unknown previously to being encountered within the TLS
 handshake. And that *does* make it inherently insecure.
 
 Not algorithmically, obviously.  I can still do a DH-handshake with the
 remote peer that will generate a shared secret no eavesdropper will
 know. The browser just can't be sure that whoever it negotiated the DH
 with is really the endpoint (i.e. the webserver). That is the problem.
 
 I dislike CAs as much as the next guy. But the problem of distributing
 trust is just not easy to solve, a TTP is a way out. Do you have an
 alternative that does not at the same time to providing a solution also
 opens up obvious attack surface?
 
 Cheers,
 Johannes
 

Trust has context.  You're going to that site to read an article.  This
is rather different than, say, going somewhere to transact commerce or
move money.

I have been doing an experiment with tundraware.com to try out https
everywhere to see just what breaks and who squawks.  I've seen a number
of concerns like the ones on this thread.  Most interestingly, this
seems to be breaking the FreeBSD ports build mechanism for the ports
I'd previously contributed to the project.

This is a tough tradeoff.  If you don't run https, then your every interaction
with the website can be trivially monitored by a third party.   Even just
the metadata of when you do use https and when you do not can be useful
to an eavesdropper.  So, there is increasing thought that we should all just
run https everywhere all the time.  But then we run into the signing problem.
I am hoping that we will soon see free or inexpensive CAs to make that
problem go away.  See:

  https://www.eff.org/deeplinks/2014/11/certificate-authority-encrypt-entire-web



-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ah Python, you have spoiled me for all other languages

[Tim Daneliuk]

On 05/22/2015 10:29 AM, Chris Angelico wrote:
 On Sat, May 23, 2015 at 12:58 AM, Steven D'Aprano st...@pearwood.info wrote:
 I think Python is a prettier
 language visually than either Lua or Ruby, but they're in the ball-park.
 Both languages have their warts and quirks, but if Python were declared
 illegal overnight[1] I'd probably have no trouble adapting to Ruby or Lua.
 Python would still be my first love, but these two languages make a
 reasonable rebound language.
 
 A good start. Toy programs don't always tell the whole story, though.
 How good are the three languages at making your code reliable in the
 face of user action? My hobby-horse, Unicode, is a notable flaw in
 many languages - if you ask the user for information (in the most
 obvious way for whatever environment you're in, be that via a web
 browser request, or a GUI widget, or text entered at the console), can
 it cope equally with all the world's languages? What if you want to
 manipulate that text - is it represented as a sequence of codepoints
 (Python 3), UTF-16 code units (JavaScript), UTF-8 bytes (quite a few),
 or bytes in whatever codepage your system was set to (anything that
 hasn't cared)?
 
 ChrisA
 


Lo these many years ago, I argued that Python is a whole lot more than 
a programming language:

   https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/

-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ah Python, you have spoiled me for all other languages

[Tim Daneliuk]

On 05/22/2015 11:31 AM, Tony the Tiger wrote:
 On Sat, 23 May 2015 00:58:17 +1000, Steven D'Aprano wrote:
 
 I get after a couple of hours is that Javascript tries really hard to do
 everything it can for you except what you actually want.
 
 You just described a certain operating system, which shall remain 
 nameless.
 
 
  /Grrr
 

CP/M ?

-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What killed Smalltalk could kill Python

[Tim Daneliuk]

On 01/23/2015 04:57 PM, Chris Angelico wrote:
 On Sat, Jan 24, 2015 at 9:51 AM, Tim Daneliuk tun...@tundraware.com wrote:
 On 01/21/2015 05:55 PM, Chris Angelico wrote:
 On Thu, Jan 22, 2015 at 10:37 AM, Tim Daneliuk tun...@tundraware.com 
 wrote:
 I find these kinds of discussions sort of silly.  Once there is a critical
 mass of installed base, no language EVER dies.

 Not sure about that. Back in the 1990s, I wrote most of my code in
 REXX, either command-line or using a GUI toolkit like VX-REXX. Where's
 REXX today? Well, let's see. It's still the native-ish language of
 OS/2. Where's OS/2 today? Left behind. REXX has no Unicode support (it
 does, however, support DBCS - useful, no?), no inbuilt networking
 support (there are third-party TCP/IP socket libraries for OS/2 REXX,
 but I don't know that other REXX implementations have socket services;
 and that's just basic BSD sockets, no higher-level protocol handling
 at all), etc, etc. Sure, it's not technically dead... but is anyone
 developing the language further? I don't think so. Is new REXX code
 being written? Not a lot. Yet when OS/2 was more popular, REXX
 definitely had its installed base. It was the one obvious scripting
 language for any OS/2 program. Languages can definitely die, or at
 least be so left behind that they may as well be dead.

 ChrisA


 Rexx is still well used on mainframes.
 
 Oh, is it? Cool! Maybe some day I'll be interviewing for a job, and
 they'll ask if I know REXX.. and I'll actually be able to say yes. :)
 
 ChrisA
 

Let me first note for the record that I make my living on *NIX type
systems.  So, my comments below are not bound to my personal technical
religion ...


I work in and among some of the larger corporations on the planet and I
have a little secret for you:  Mainframes are NOT going away wholesale.
As the population of mainframe-savvy staff dwindles into retirement,
one of the hottest tickets you're going to see in the next 20 years
is people who can do that work.  It's so bad, that IBM themselves are
working to train the next generation of MVS systems programmers, and
the like.

And no, you can't just replace mainframes with open systems equivalents.
There are likely hundreds of millions of lines of COBOL and assembler
code that cannot just be ported over to a new platform and language.
Why?  Because there is an ecosystem of support in the mainframe world
that is utterly absent in the *NIX/Windoze world.   This includes things
especially like CICS, IMS (ask Caterpillar if they'll stop using it ;),
DB2 and so forth.

So, yes, if you're mainframe savvy, you may have a nice consulting career
in your old age :)

-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What killed Smalltalk could kill Python

[Tim Daneliuk]

On 01/21/2015 05:55 PM, Chris Angelico wrote:
 On Thu, Jan 22, 2015 at 10:37 AM, Tim Daneliuk tun...@tundraware.com wrote:
 I find these kinds of discussions sort of silly.  Once there is a critical
 mass of installed base, no language EVER dies.
 
 Not sure about that. Back in the 1990s, I wrote most of my code in
 REXX, either command-line or using a GUI toolkit like VX-REXX. Where's
 REXX today? Well, let's see. It's still the native-ish language of
 OS/2. Where's OS/2 today? Left behind. REXX has no Unicode support (it
 does, however, support DBCS - useful, no?), no inbuilt networking
 support (there are third-party TCP/IP socket libraries for OS/2 REXX,
 but I don't know that other REXX implementations have socket services;
 and that's just basic BSD sockets, no higher-level protocol handling
 at all), etc, etc. Sure, it's not technically dead... but is anyone
 developing the language further? I don't think so. Is new REXX code
 being written? Not a lot. Yet when OS/2 was more popular, REXX
 definitely had its installed base. It was the one obvious scripting
 language for any OS/2 program. Languages can definitely die, or at
 least be so left behind that they may as well be dead.
 
 ChrisA
 

Rexx is still well used on mainframes.
-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What killed Smalltalk could kill Python

[Tim Daneliuk]

On 01/21/2015 10:34 AM, Steven D'Aprano wrote:
 In 2009, Robert Martin gave a talk at RailsConf titled What Killed
 Smalltalk Could Kill Ruby. (No cheering, that sort of attitude is one of
 the things that killed Smalltalk.) Although Martin discusses Ruby, the
 lessons could also apply to Python.


I find these kinds of discussions sort of silly.  Once there is a critical
mass of installed base, no language EVER dies.

I suspect the real reason Smalltalk sort of got kicked to the curb is because
a) It clung to a kind of OO purity that simply is at odds with the practice
of programming at large scale  and   b) It thus never built the aforementioned
critical mass.

Language adoption at the scale needed to make a real dent doesn't happen
because of technical superiority (witness PHP as just one example).  It happens
because lots of people solve real problems faster than they used to.
In fact - outside the language cognoscenti and uber nerd community - I'd
argue that  Python adoption has little to do with functional programming,
lambda, OO, generators, or whatever happens to float your boat.  Python
got adopted because it made code production faster, and therefore cheaper.
Economics matters way more than technology here, I think.

I wrote some rambling disquisition on these matters some years ago ...

  http://www.tundraware.com/TechnicalNotes/Python-Is-Middleware

  http://www.tundraware.com/TechnicalNotes/How-To-Pick-A-Programming-Language
-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

-- 
https://mail.python.org/mailman/listinfo/python-list

ANN: 'tsshbatch' Server Automation Tool Version 1.204 Released

[Tim Daneliuk]

'tsshbatch' Version 1.204 is now released and available for download at:

 http://www.tundraware.com/Software/tsshbatch

This is a major update with many bug fixes and improvements.  Existing
users will want to update sooner rather than later.

The last public release was 1.177.
-

What Is 'tsshbatch'?


'tsshbatch' is a server automation tool to enable you to issue commands
to many servers without having to log into each one separately.  When
writing scripts, this overcomes the 'ssh' limitation of not being able to
specify the password on the command line.

'tsshbatch' also understands basic 'sudo' syntax and can be used
to access a server, 'sudo' a command, and then exit.

'tsshbatch' thus allows you to write complex, hands-off scripts that
issue commands to many servers without the tedium of manual login and
'sudo' promotion.  System administrators, especially, will find this
helpful when working in large server farms.

'tsshbatch' is written in Python and requires the 'paramiko library.
It has been tested on various Linux and FreeBSD variants.



WHATSNEW For 'tsshbatch' 1.204(Thu Dec  4 17:49:30 CST 2014)
--

[NEW FEATURES]

- Added the following builtin variables:

  __DATE__   # Date in MMDD format
  __DATETIME__   # Date and time in MMDDHHMMSS format
  __HOSTNAME__   # Full name of current host as passed to program
  __HOSTNUM__# Count of host being processed, starting at 1
  __HOSTSHORT__  # Leftmost component of hostname as passed to program
  __TIME__   # Time in HHMMSS format

- Added an execution variable.  This runs a command of your
  choosing (on the local machine) and assigns the results to
  a user-defined variable.

- Added -E to redirect all stderr output to stdout instead,

- Added -T timeout option (default is 15 sec).

- Added -a to allow program to continue after file transfer error.

- Added -l logging option. Defaults to /dev/null.  This fixes the error
  that was previously being reported:

No handlers could be found for logger paramiko.transport

- Added -q for quieter output.


[CHANGES]

- File transfers now properly preserve the file's permissions.

- Changed hostname separator from ':' to '-' when using the -G command.

- The HOSTNAME and HOSTSHORT builtins have been replaced
  with the new builtins described above.

- Error messages now more consistent and clear.

- Test mode now expands variable references to their values for
  all variables except the builtins above (which are only
  evaluated at runtime).ANN: 'tsshbatch' Server Automation Tool Version 1.204 
Released


- Documentation has been rewritten and improved considerably.


[BUG FIXES]

- Fixed bug that prevented the proper dereferencing of
  __HOSTNAME|SHORT__ (formerly HOSTNAME|SHORT) in file transfer
  specifications.

- Fixed bug that prevented variable substitution in hostnames.

- Fixed bug that prevented '.define' variables from being substituted
  in file transfer specifications.

- Fixed bug that only recognized sudo invocations if they were the
  first statement on a command line.  All instances of the string
  sudo  will now force sudo password prompting and processing.
  That string is ignored if it appears inside single- or double quotes.

- Fixed a bug that intermittently occurred during password-based auth
  sessions because ssh-agent and key searching were still being used.

- Fixed error reporting blowout when key-exchange auth fails.


Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/


--
https://mail.python.org/mailman/listinfo/python-announce-list

   Support the Python Software Foundation:
   http://www.python.org/psf/donations/

ANN: 'tsshbatch' Server Automation Tool Version 1.204 Released

[Tim Daneliuk]

'tsshbatch' Version 1.204 is now released and available for download at:

 http://www.tundraware.com/Software/tsshbatch

This is a major update with many bug fixes and improvements.  Existing
users will want to update sooner rather than later.

The last public release was 1.177.
-

What Is 'tsshbatch'?


'tsshbatch' is a server automation tool to enable you to issue commands
to many servers without having to log into each one separately.  When
writing scripts, this overcomes the 'ssh' limitation of not being able to
specify the password on the command line.

'tsshbatch' also understands basic 'sudo' syntax and can be used
to access a server, 'sudo' a command, and then exit.

'tsshbatch' thus allows you to write complex, hands-off scripts that
issue commands to many servers without the tedium of manual login and
'sudo' promotion.  System administrators, especially, will find this
helpful when working in large server farms.

'tsshbatch' is written in Python and requires the 'paramiko library.
It has been tested on various Linux and FreeBSD variants.



WHATSNEW For 'tsshbatch' 1.204(Thu Dec  4 17:49:30 CST 2014)
--

[NEW FEATURES]

- Added the following builtin variables:

  __DATE__   # Date in MMDD format
  __DATETIME__   # Date and time in MMDDHHMMSS format
  __HOSTNAME__   # Full name of current host as passed to program
  __HOSTNUM__# Count of host being processed, starting at 1
  __HOSTSHORT__  # Leftmost component of hostname as passed to program
  __TIME__   # Time in HHMMSS format

- Added an execution variable.  This runs a command of your
  choosing (on the local machine) and assigns the results to
  a user-defined variable.

- Added -E to redirect all stderr output to stdout instead,

- Added -T timeout option (default is 15 sec).

- Added -a to allow program to continue after file transfer error.

- Added -l logging option. Defaults to /dev/null.  This fixes the error
  that was previously being reported:

No handlers could be found for logger paramiko.transport

- Added -q for quieter output.


[CHANGES]

- File transfers now properly preserve the file's permissions.

- Changed hostname separator from ':' to '-' when using the -G command.

- The HOSTNAME and HOSTSHORT builtins have been replaced
  with the new builtins described above.

- Error messages now more consistent and clear.

- Test mode now expands variable references to their values for
  all variables except the builtins above (which are only
  evaluated at runtime).ANN: 'tsshbatch' Server Automation Tool Version 1.204 
Released


- Documentation has been rewritten and improved considerably.


[BUG FIXES]

- Fixed bug that prevented the proper dereferencing of
  __HOSTNAME|SHORT__ (formerly HOSTNAME|SHORT) in file transfer
  specifications.

- Fixed bug that prevented variable substitution in hostnames.

- Fixed bug that prevented '.define' variables from being substituted
  in file transfer specifications.

- Fixed bug that only recognized sudo invocations if they were the
  first statement on a command line.  All instances of the string
  sudo  will now force sudo password prompting and processing.
  That string is ignored if it appears inside single- or double quotes.

- Fixed a bug that intermittently occurred during password-based auth
  sessions because ssh-agent and key searching were still being used.

- Fixed error reporting blowout when key-exchange auth fails.


Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/



--

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-announce-list

   Support the Python Software Foundation:
   http://www.python.org/psf/donations/

ANN: 'tsshbatch' Server Automation Tool Version 1.204 Released

[Tim Daneliuk]

'tsshbatch' Version 1.204 is now released and available for download at:

 http://www.tundraware.com/Software/tsshbatch

This is a major update with many bug fixes and improvements.  Existing
users will want to update sooner rather than later.

The last public release was 1.177.
-

What Is 'tsshbatch'?


'tsshbatch' is a server automation tool to enable you to issue commands
to many servers without having to log into each one separately.  When
writing scripts, this overcomes the 'ssh' limitation of not being able to
specify the password on the command line.

'tsshbatch' also understands basic 'sudo' syntax and can be used
to access a server, 'sudo' a command, and then exit.

'tsshbatch' thus allows you to write complex, hands-off scripts that
issue commands to many servers without the tedium of manual login and
'sudo' promotion.  System administrators, especially, will find this
helpful when working in large server farms.

'tsshbatch' is written in Python and requires the 'paramiko library.
It has been tested on various Linux and FreeBSD variants.



WHATSNEW For 'tsshbatch' 1.204(Thu Dec  4 17:49:30 CST 2014)
--

[NEW FEATURES]

- Added the following builtin variables:

  __DATE__   # Date in MMDD format
  __DATETIME__   # Date and time in MMDDHHMMSS format
  __HOSTNAME__   # Full name of current host as passed to program
  __HOSTNUM__# Count of host being processed, starting at 1
  __HOSTSHORT__  # Leftmost component of hostname as passed to program
  __TIME__   # Time in HHMMSS format

- Added an execution variable.  This runs a command of your
  choosing (on the local machine) and assigns the results to
  a user-defined variable.

- Added -E to redirect all stderr output to stdout instead,

- Added -T timeout option (default is 15 sec).

- Added -a to allow program to continue after file transfer error.

- Added -l logging option. Defaults to /dev/null.  This fixes the error
  that was previously being reported:

No handlers could be found for logger paramiko.transport

- Added -q for quieter output.


[CHANGES]

- File transfers now properly preserve the file's permissions.

- Changed hostname separator from ':' to '-' when using the -G command.

- The HOSTNAME and HOSTSHORT builtins have been replaced
  with the new builtins described above.

- Error messages now more consistent and clear.

- Test mode now expands variable references to their values for
  all variables except the builtins above (which are only
  evaluated at runtime).

- Documentation has been rewritten and improved considerably.


[BUG FIXES]

- Fixed bug that prevented the proper dereferencing of
  __HOSTNAME|SHORT__ (formerly HOSTNAME|SHORT) in file transfer
  specifications.

- Fixed bug that prevented variable substitution in hostnames.

- Fixed bug that prevented '.define' variables from being substituted
  in file transfer specifications.

- Fixed bug that only recognized sudo invocations if they were the
  first statement on a command line.  All instances of the string
  sudo  will now force sudo password prompting and processing.
  That string is ignored if it appears inside single- or double quotes.

- Fixed a bug that intermittently occurred during password-based auth
  sessions because ssh-agent and key searching were still being used.

- Fixed error reporting blowout when key-exchange auth fails.


Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/



--

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-announce-list

   Support the Python Software Foundation:
   http://www.python.org/psf/donations/

ANN: 'tsshbatch' Server Automation Tool Version 1.204 Released

[Tim Daneliuk]

'tsshbatch' Version 1.204 is now released and available for download at:

 http://www.tundraware.com/Software/tsshbatch

This is a major update with many bug fixes and improvements.  Existing
users will want to update sooner rather than later.

The last public release was 1.177.
-

What Is 'tsshbatch'?


'tsshbatch' is a server automation tool to enable you to issue commands
to many servers without having to log into each one separately.  When
writing scripts, this overcomes the 'ssh' limitation of not being able to
specify the password on the command line.

'tsshbatch' also understands basic 'sudo' syntax and can be used
to access a server, 'sudo' a command, and then exit.

'tsshbatch' thus allows you to write complex, hands-off scripts that
issue commands to many servers without the tedium of manual login and
'sudo' promotion.  System administrators, especially, will find this
helpful when working in large server farms.

'tsshbatch' is written in Python and requires the 'paramiko library.
It has been tested on various Linux and FreeBSD variants.



WHATSNEW For 'tsshbatch' 1.204(Thu Dec  4 17:49:30 CST 2014)
--

[NEW FEATURES]

- Added the following builtin variables:

  __DATE__   # Date in MMDD format
  __DATETIME__   # Date and time in MMDDHHMMSS format
  __HOSTNAME__   # Full name of current host as passed to program
  __HOSTNUM__# Count of host being processed, starting at 1
  __HOSTSHORT__  # Leftmost component of hostname as passed to program
  __TIME__   # Time in HHMMSS format

- Added an execution variable.  This runs a command of your
  choosing (on the local machine) and assigns the results to
  a user-defined variable.

- Added -E to redirect all stderr output to stdout instead,

- Added -T timeout option (default is 15 sec).

- Added -a to allow program to continue after file transfer error.

- Added -l logging option. Defaults to /dev/null.  This fixes the error
  that was previously being reported:

No handlers could be found for logger paramiko.transport

- Added -q for quieter output.


[CHANGES]

- File transfers now properly preserve the file's permissions.

- Changed hostname separator from ':' to '-' when using the -G command.

- The HOSTNAME and HOSTSHORT builtins have been replaced
  with the new builtins described above.

- Error messages now more consistent and clear.

- Test mode now expands variable references to their values for
  all variables except the builtins above (which are only
  evaluated at runtime).

- Documentation has been rewritten and improved considerably.


[BUG FIXES]

- Fixed bug that prevented the proper dereferencing of
  __HOSTNAME|SHORT__ (formerly HOSTNAME|SHORT) in file transfer
  specifications.

- Fixed bug that prevented variable substitution in hostnames.

- Fixed bug that prevented '.define' variables from being substituted
  in file transfer specifications.

- Fixed bug that only recognized sudo invocations if they were the
  first statement on a command line.  All instances of the string
  sudo  will now force sudo password prompting and processing.
  That string is ignored if it appears inside single- or double quotes.

- Fixed a bug that intermittently occurred during password-based auth
  sessions because ssh-agent and key searching were still being used.

- Fixed error reporting blowout when key-exchange auth fails.


Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/



--

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/


--

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-list

Re: Quotation Ugliness

[Tim Daneliuk]

On 11/26/2014 06:56 AM, Tim Chase wrote:

On 2014-11-26 00:04, Tim Daneliuk wrote:

someprog.py uname  sudo cat /etc/sudoers

vs.

someprog.py uname  echo sudo cat /etc/suoders


In the first instance, I need the sudo passoword, in the second I
don't.


This doesn't jibe with the pairs of quotes you sent and your request
for nesting.  In most popular shells, the majority of your quote
characters don't actually quote anything:

   bash$ echo // hello
   hello
   bash$ echo /* hello */
   [returns all the items in my root directory, the word hello,
   along with all the sub-directories in the current directory]
   bash$ echo this#and#that
   this#and#that
   bash$ echo this # and #that
   this

and has problems with things like

   someprog.py uname  sudo cat /etc/sudoers
   someprog.py uname  sudo cat /etc/sudoers

which my shell will parse valid execution of sudo.


SNIP


I am not writing in bash,  but in python.

The specific program in question I am modifying is
one that takes a shell command and executes it remotely on many machines.
The problem I am trying to solve is to determine whether the user needs to
provide a sudo password or not.  Right now, the program just naively checks
the arguments to see if .startswith(sudo').  As the example given illustrates,
that clearly fails if the sudo is given later in a longer pipeline.

Scanning the whole argument string for 'sudo' is better but will yield
false positives if the string is inside a quote of some sort.  Since I have
to solve the problem for ' and  delimiters, I thought I'd generalize the 
solution
for other strings possibly being quoted by other delimiters.

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-list

Re: Quotation Ugliness

[Tim Daneliuk]

On 11/26/2014 01:10 AM, Chris Angelico wrote:

Why not set up sudo to not require a password



Because I do not control the machines to which this program is talking and
the security policy in question requires passwords.

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-list

Re: Quotation Ugliness

[Tim Daneliuk]

On 11/26/2014 08:12 AM, random...@fastmail.us wrote:

On Wed, Nov 26, 2014, at 01:04, Tim Daneliuk wrote:

In this case, I am not trying to write a fullblown language or recover
from syntax errors.   Here's a usecase - I want to know whether I need
to use a sudo password when the user passes a command on the command line
of a program:

someprog.py uname  sudo cat /etc/sudoers

vs.

someprog.py uname  echo sudo cat /etc/suoders


In the first instance, I need the sudo passoword, in the second I don't.


I think first you need to understand how the command line works. Much of
this parsing - including both  and quotes - is handled by the shell
before your program ever sees it.



I am not writing in shell nor am I escaping to the shell locally for execution

someprog.py is a paramiko-based wrapper that executes the given command
on many machines remotely.  I (over) simplified the example and in my
haste introduced and incorrect idea.  What this should say is:

  someprog.py uname  sudo cat /etc/sudoers

 vs.

  someprog.py 'uname  echo sudo cat /etc/suoders'



--

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-list

Re: Quotation Ugliness

[Tim Daneliuk]

On 11/26/2014 09:09 AM, Chris Angelico wrote:

On Thu, Nov 27, 2014 at 2:07 AM, Chris Angelico ros...@gmail.com wrote:

I was searching the ol' memory banks, trying to figure out if there
was some way to tell the internal 'echo' command to use slash instead
of dash (maybe for DOS/Windows people??), in which case that would be
parsed as echo -- hello and would indeed simply echo hello.


Speaking of which, it's entirely possible to have a -sudo parameter
to a command (which would be -s -u -d -o to many programs), so
that's another way to get a false positive.

ChrisA



Yeah, there is no foolproof way to do this short of implementing a parser
that acts exactly as the shell itself would when parsing the command line.

The more I think about this, the more I think I am just going to look for the
string 'sudo' anywhere in the argument.  This merely will force the user to
enter their sudo password if detected.  If it turns out to be a false positive,
no harm will be done and the password will just go unused.


Thanks for the feedback.

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-list

Re: Quotation Ugliness

[Tim Daneliuk]

On 11/26/2014 09:48 AM, Chris Angelico wrote:

On Thu, Nov 27, 2014 at 2:36 AM, Tim Daneliuk tun...@tundraware.com wrote:

The more I think about this, the more I think I am just going to look for
the
string 'sudo' anywhere in the argument.  This merely will force the user to
enter their sudo password if detected.  If it turns out to be a false
positive,
no harm will be done and the password will just go unused.


That sounds reasonable; imperfect, but reasonable. But what happens if
the password goes unused? Will it be provided on stdin to the
program? That could be VERY bad in two ways (revealing the password,
and breaking the program's expectations).

ChrisA



Nope.  Password only exist in memory locally.

If you want to see the whole program in action as it currently exists:

  http://www.tundraware.com/Software/tsshbatch

I do a lot of work in very large data centers where I need to execute a
bunch of commands across, say, a thousand servers.  tsshbatch has solved
the problem neatly for some years.  I'm just trying to knock some of
the rough edges off it :)

P.S. There are other such programs like 'capistrano' that are spiritually quite
 similar.  I wrote tsshbatch before I found out about the others, which
 are all written in other lanugages (like Ruby).  So, not only did
 this serve to solve my problem, it also taught me paramiko :)
--

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-list

Re: Quotation Ugliness

[Tim Daneliuk]

On 11/26/2014 10:00 AM, random...@fastmail.us wrote:

On Wed, Nov 26, 2014, at 10:55, Tim Daneliuk wrote:

Nope.  Password only exist in memory locally.


How does it send it to the remote sudo?



Over paramiko transport (ssh) and then only if it sees a custom
string coming back from sudo asking for the pw.

--

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-list

Re: Quotation Ugliness

[Tim Daneliuk]

On 11/26/2014 10:16 AM, random...@fastmail.us wrote:

On Wed, Nov 26, 2014, at 11:02, Tim Daneliuk wrote:

On 11/26/2014 10:00 AM, random...@fastmail.us wrote:

On Wed, Nov 26, 2014, at 10:55, Tim Daneliuk wrote:

Nope.  Password only exist in memory locally.


How does it send it to the remote sudo?



Over paramiko transport (ssh) and then only if it sees a custom
string coming back from sudo asking for the pw.


So why not ask for the password only after seeing that string?




Because this is a batch program. Once launched against hundreds or even
thousands of machines, no further user interaction is involved.

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-list

Re: Quotation Ugliness

[Tim Daneliuk]

On 11/26/2014 10:45 AM, alister wrote:

On Wed, 26 Nov 2014 10:02:57 -0600, Tim Daneliuk wrote:


On 11/26/2014 10:00 AM, random...@fastmail.us wrote:

On Wed, Nov 26, 2014, at 10:55, Tim Daneliuk wrote:

Nope.  Password only exist in memory locally.


How does it send it to the remote sudo?



Over paramiko transport (ssh) and then only if it sees a custom string
coming back from sudo asking for the pw.


is it possible for you to post to either the news groout or the mailing
list but not both?
all of your posts are currently doubled



Noted, fixed, sorry.
--

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

--
https://mail.python.org/mailman/listinfo/python-list