Re: [Tutor] which of these is more efficient?

[Mats Wichmann]

On 8/18/19 5:55 PM, nathan tech wrote:
> Hi there,
> 
> So I am running over some coding ideas in my head for creating a map for 
> a game.
> 
> This map would expand based on how far the user explores.
> 
> I figure there are two ways to do this:
> 
> 1: the list method:
> 
> map=[]
> for x in range(3):
>   temp=[]
>   for y in range(3):
>    temp.append(default_grid_format)
>   map.append(temp)
> 
> then when ever the user explores a square not on the current map, it 
> would do this:
> 
> for x in range(len(map)):
>   map[x].append(default_grid_format)
> temp=[]
> for x in range(len(map[0])):
>   temp.append(default_grid_format)
> map.append(temp)
> 
> Obviously, though, this creates a lot of data for squares that are still 
> ultimately unexplored.
> 
> So here was my other idea:
> 
> 2. the dictionary method:
> 
> map={}
> for x in range(3):
>   for y in range(3):
>    key=str(x)+":"+str(y)
>    map[key]=default_grid_format
> 
> Then when user explores new square do:
> 
> key=str(player_x)+":"+str(player_y)
> map[key]=default_grid_format
> 
> 
> Is this an efficient method compared to 1?
> 
> Is it, code wise, sound logic?
> 
> 
> 
> I guess I'm just looking for a second opinion from experienced peoples.

A few notes...

There's a nice little tension in design and programming of a project:
don't optimize too early, you may not need to, and/or your early guesses
may be completely wrong VS. if you pick a bad design up front, it may be
"too late" to optimize away the problems that brings later.  Then again
they _also_ say "build one to throw away"  :)

You haven't defined what needs to happen for a new square.  If the "map"
(please don't use that as a variable name, by the way, as it's already a
Python builtin function) is a rectangular grid, each square has eight
neighbors.  If you've ever played with minesweeper, you see this in
action... "exploring" from one point means figuring out which of the
neighbors has a mine.  So what should happen for a "new" square? Do you
want to generate the information for all of those neighbors? Your
conceptual code seems to just add a row-and-column, which seems a little
odd... what if the exploration went in the other direction, i.e. "left"
or "up"?

Python has some support for "don't calculate it until it's needed",
called generators. Depending on your design you may or may not be able
to make use of such.  Here's a silly little snippet to illustrate a
generator which produces the neighbors of a square (in this case, just
the coordinates):

def neighbors(point):
x, y = point
yield x + 1, y
yield x - 1, y
yield x, y + 1
yield x, y - 1
yield x + 1, y + 1
yield x + 1, y - 1
yield x - 1, y + 1
yield x - 1, y - 1

nlist = neighbors(point)
print(f"neighbors returns {nlist}")
print(f"neighbors of {point}: {list(nlist)}")

When run:

neighbors returns 
neighbors of (7, 6): [(8, 6), (6, 6), (7, 7), (7, 5), (8, 7), (8, 5),
(6, 7), (6, 5)]

... the data (just the coordinates of a neighbor square) is not
generated until you consume it, which the string in the print call
forces by calling list() on it. The more normal case is you'd loop over
it...

It this was too complicated for where you are in your Python studies,
don't worry about it, but you'll get there eventually :)


Micro-comment: a piece of code I'd hope to never see again:

for x in range(len(map)):
   map[x].append(default_grid_format)

this lazily produces a counter from the size of an iterable object
(similar in concept to a generator), and then uses the counter to index
into said object; that's not needed since you can just iterate the
object directly.  Instead write it like this:

for m in map:
  m.append(default_grid_format)


Micro-comment: you can use anything (well, anything "hashable") as the
key for a dict.  So for:

   key=str(x)+":"+str(y)

you can just do:

   key = (x, y)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Package which can extract data from pdf

[Mats Wichmann]

On 8/14/19 10:10 AM, Nupur Jha wrote:
> Hi All,
> 
> I have many pdf invoices with different formats. I want to extract the line
> items from these pdf files using python coding.
> 
> I would request you all to guide me how can i achieve this.
> 

There are many packages that attempt to extract text from pdf.  They
have varying degrees of success on various different documents: you need
to be aware that PDF wasn't intended to be used that way, it was written
to *display* consistently.  Sometimes the pdf is full of instructions
for rendering that are hard for a reader to figure out, and need to be
pieced together in possibly unexpected ways.  My experience is that if
you can select the interesting text in a pdf reader, and paste it into
an editor, and it doesn't come out looking particularly mangled, then
reading it programmatically has a pretty good chance of working. If not,
you may be in trouble. That said...

pypdf2, textract, and tika all have their supporters. You can search for
all of these on pypi, which will give you links to the projects' home pages.

(if it matters, tika is an interface to a bunch of Java code, so you're
not using Python to read it, but you are using Python to control the
process)

There's a product called pdftables which specifically tries to be good
at spreadsheet-like data, which your invoices *might* be.  That is not a
free product, however. For that one there's a Python interface that
sends your data off to a web service and you get answers back.

There are probably dozens more... this seems to be an area with a lot of
reinvention going on.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Union

[Mats Wichmann]

On 8/12/19 2:50 PM, Jim wrote:
> I was reading the docs for PySimpbleGUI here:
> https://pysimplegui.readthedocs.io/en/latest/#building-custom-windows
> 
> In the table of parameters for the Window() function for example the
> icon parameter the meaning is  Union[str, str] Can be either a filename
> or Base64 value.
> 
> What is the usage of "Union". I don't recall seeing anything like it
> before.

it's type annotation. Search here:

https://docs.python.org/3/library/typing.html


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] HELP PLEASE

[Mats Wichmann]

On 8/12/19 10:54 AM, Marissa Russo wrote:
> Hello,
> 
> I am trying to figure out what is going on and why my output is saying 
> “” instead of giving me a number. Please let me know if 
> you see the error in my code!!

to quickly illustrate the specific question you asked - you got comments
on other stuff already:


>>> def foo():
... return "string from foo()"
...
>>> print(foo)

>>> print(foo())
string from foo()
>>>


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] instantiate and name a class from / with a string

[Mats Wichmann]

On 8/9/19 11:55 AM, ingo wrote:
> With the available classes Root, Channel and SSE I build the following
> (in CherryPy):
> 
> root = Root()
> root.channel = Channel()
> 
> root.channel.weather = SSE('weather')
> root.channel.energy = SSE('energy')
> root.channel.mail = SSE('mail')
> root.channel.rss = SSE('rss')
> ...
> 
> http://example.com/channel/weather/ will then become the emitter of the
> weather event stream.
> 
> I'd like create the instances of SSE programmatically by pulling the
> string 'weather', 'energy' etc. from a database.

To channel Bob:  what are you actually asking for here? We can't figure
it out. Is there some database that has these strings? If so, what is
it?  Or do you mean "some data structure in your program that maps those
strings to something to do with them" (if so, where do you get that
information from?)

"With the available classes... in CherryPy"

Available from where? Your code? Some external piece of code you're
using? Does CherryPy actually have anything to do with the question
you're not quite asking?

Please be more precise.  We're not trying to be argumentative; you have
to help us know enough before we can help you.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Error terminal

[Mats Wichmann]

On 8/7/19 5:27 PM, Richard Rizk wrote:
> Thank you Cameron for your message.
> 
> Please find below the error message i am receiving.
> 
> I think the command line i'm running is trying to connect with the
> python3.7 that i have on my computer which is a requirement for the command
> line to work but it keeps connecting to the default python2.7 that is on
> Macbook.
> 
> I'd love to jump on a 15min call if  it's possible for you.
> 
> Best regards,
> Richard
> 
> Error:
> -
> 
> DEPRECATION: Python 2.7 will reach the end of its life on January 1st,
> 2020. Please upgrade your Python as Python 2.7 won't be maintained after
> that date. A future version of pip will drop support for Python 2.7. More
> details about Python 2 support in pip, can be found at
> https://pip.pypa.io/en/latest/development/release-process/#python-2-support

Whichever way you get the Python you want to work for you, use the same
way to do installs.  Thus, if it works like this:

$ python3
Python 3.7.2 (default, Dec 27 2018, 07:35:45)
[Clang 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>


Then request your installations like this:

$ python3 -m pip install bunch-o-stuff


Your instructions probably said do "pip install bunch-o-stuff", don't do
it that way.


You can also check what you're getting this way (this is a snip from my
system, where the Python 3 came from homebrew, which is how I happen to
install it):

$ which python
/usr/bin/python# system version, you don't want this one
$ which python3
/usr/local/bin/python
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Name for this type of class?

[Mats Wichmann]

On 8/3/19 10:56 AM, Malcolm Greene wrote:
> Thanks for everyone's feedback. Some interesting thoughts including Alan's 
> "classes should never be named for their data but for their function" 
> feedback. I'm going to have to noodle on that one. Good stuff!

And more:

if the class is really just a data container, then no need to make it a
class. Use a namedtuple (or as Cameron suggested,
types.SimpleNamespace).  Of course we still don't know if that's the
case...

General comment on naming: you don't need to qualify things if the
context makes it clear, all it does is make you type more stuff (or, I
guess, exercise autocomplete in your IDE).  A favorite one is to call
your custom exception MyProgramSpecificException.  Since this is going
to appear in a line with raise or except in it, the word Exception is
superfluous.  This comes up in regards to an idea of using something
like TelemetryData or TelemetryStatistics.  It also comes up here:

>> file_count
>> line_count
>> byte_count
>> row_count
>> batch_count
>> job_count
>> error_count
>> warning_count

why not "files", "lines", "bytes"... the plural form already tells you
it's a counter.

Opinions, we all have 'em  :)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Difference between decorator and inheritance

[Mats Wichmann]

On 7/31/19 11:57 AM, Gursimran Maken wrote:
> Hi,
> 
> Anyone could please let me know the difference between decorators and
> inheritance in python.
> 
> Both are required to add additional functionality to a method then why are
> we having 2 separate things in python for doing same kind of work.

I started to write something here several times and never felt like I
got it right.  Let me try once more without fussing too much.

Python's decorators feel like a "local" thing - I have some code, and I
want to make some change to it.  Most often this comes up in predefined
scenarios - I enable one of my functions to be line-profiled by
decorating with @profile.  I turn an attribute into a getter by using
@property. I decorate to time a function. I decorate to add caching
(memoizing) capability (@functools.lru_cache).

If doesn't _have_ to be local; if I wanted to wrap a function that is
not in "my" code (e.g. from Python standard library, or some module that
I've obtained from the internet) I can, although the convenience form
using @decorator doesn't really apply here since I don't want to modify
"foreign" code; I have to use the underlying form of writing a function
that generates and returns a function which augments the original
function (that sounds so messy when you try to write it in English!).

Inheritance is a more systematic building of relationships between
classes - I can accomplish some of that with decorators, but not all.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Which is better in principle: to store (in file) calculated data or to re-calculate it upon restarting program?

[Mats Wichmann]

On 7/30/19 5:58 PM, Alan Gauld via Tutor wrote:
> On 30/07/2019 17:21, boB Stepp wrote:
> 
>> musings I am wondering about -- in general -- whether it is best to
>> store calculated data values in a file and reload these values, or
>> whether to recalculate such data upon each new run of a program.  
> 
> It depends on the use case.
> 
> For example a long running server process may not care about startup
> delays because it only starts once (or at least very rarely) so either
> approach would do but saving diskspace may be helpful so calculate the
> values.
> 
> On the other hand a data batch processor running once as part of a
> chain working with high data volumes probably needs to start quickly.
> In which case do the calculations take longer than reading the
> extra data? Probably, so store in a file.
> 
> There are other options too such as calculating the value every
> time it is used - only useful if the data might change
> dynamically during the program execution.
> 
> It all depends on how much data?, how often it is used?,
> how often would it be calculated? How long does the process
> run for? etc.


Hey, boB - I bet you *knew* the answer was going to be "it depends" :)

There are two very common classes of application that have to make this
very decision - real databases, and their toy cousins, spreadsheets.

In the relational database world - characterized by very long-running
processes (like: unless it crashes, runs until reboot. and maybe even
beyond that - if you have a multi-mode replicated or distributed DB it
may survive failure of one point) - if a field is calculated it's not
stored. Because - what Alan said: in an RDBMS, data are _expected_ to
change during runtime. And then for performance reasons, there may be
some cases where it's precomputed and stored to avoid huge delays when
the computation is expensive. That world even has a term for that: a
materialized view (in contrast to a regular view).  It can get pretty
tricky, you need something that causes the materialized view to update
when data has changed; for databases that don't natively support the
behavior you then have to fiddle with triggers and hopefully it works
out.  More enlightened now?

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Inserting long URL's into comments & docstrings?

[Mats Wichmann]

On 7/29/19 2:36 PM, James Hartley wrote:
> This should be a slow ball pitch.  Unfortunately, I haven't stumbled across
> a reasonable answer yet.
> 
> On occasion, I put long URL's into comments/docstrings simply to document
> where I found specific information.  However, to be a good disciple of
> PEP8, anything which can't fit within 72 characters needs to be split
> across multiple lines.  Since a number of you seem to be prolific Python
> coders, what opinion do you have about splitting URL's in
> comments/docstrings?

One person's opinion:

Don't.

It's not worth being so dogmatic that you split if something like a URL
by itself on a line goes a little long... even if it goes 105 chars, say
(there's not a magic number).

And if the URL is really long and full of crud, rather than simple and
readable, consider feeding it to a shortener - its visual clarity is
already compromised, after all.



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] REQUIRED SUPPORT FOR CODE

[Mats Wichmann]

On 7/25/19 9:58 AM, NITESH KUMAR wrote:
> Dear Sir/Madam
> 
> I want to make Autocomplete searchbox using database .Please suggest me the
> code for this.

Your question leaves many more questions...

A searchbox implies you're using a graphical toolkit. Which one do you
intend to use (there are many available for Python)?  Or is your gui a
web browser maybe?

What do you mean "search... using database"?  Do you want the expanded
text to be used to search a database?  Or do you want to feed unexpanded
entered text to a database to get it to give you back completion
possibilities? Where would that data be coming from?  As you just trying
to replicate / replace Elasticsearch?

and so on  it's likely you can find some existing code for this on
the web if you search a bit.  It's a pretty specialized problem, don't
know if anyone here has any specific places to point you to.  Which
isn't really what this list is for anyway.

The normal model here is you try something and ask for help if you can't
get it to work. Something like this:

 -  Send a code example that illustrates your problem - If possible,
make this a minimal example rather than an entire application.
 -  Details on how you attempted to solve the problem on your own.
 -  Full version information — for example, "Python 3.6.4 with
discord.py 1.0.0a". Mention the operating system (Linux, Mac, Windows)
as well.
 -  The full traceback if your code raises an exception - Do not curate
the traceback as you may inadvertently exclude information crucial to
solving your issue.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] __weakref__ question

[Mats Wichmann]

On 7/24/19 1:57 PM, Sarah Hembree wrote:
> Can anyone provide some simple example/s or two or three of using weakref?
> I'm baffled and not seeing any documentation that is meaningful. My
> interest is to minimize memory usage (generally speaking, overall) and am
> wondering if this might help.

The answer is "maybe" and as a guess "probably not" going to give
signifgant memory savings - but it depends on circumstances.

It would help if you could describe a bit more of the scenario that
causes you to worry about memory usage.

I looked for documentation on __weakref__ since you said there didn't
seem to be anything meaningful and agree there's not much to go on - it
mostly seems to be something you have to fiddle with if you're using
__slots__.  On the other hand, the weakref module, which is what you're
more likely to directly use, seems reasonably well documented.

There are sure to be people with more expertise here who can explain
this better... but the two prominent uses of weakref seem to be in
caching, and in avoiding reference loops (or rather avoiding problems
related to them).  In the former case, if the things you put in your
cache are weak references, they can be garbage-collected by the Python
interpreter as needed, but before that happens you can refer to them.
Where it perhaps makes sense is where you have an object that is
expensive that you're doing some processing on, and you also add it,
with a weak reference to a list of recent objects.  Once you're done...
well, I find that what I'm saying here is actually said better by the
documentation so I'll stop, just read there:

https://docs.python.org/3/library/weakref.html

The latter case, reference cycles, where things refer to each other in a
way that potentially nothing gets released, ought to eventually get
caught by the interpreter which has a garbage collection algorithm for
this, but you can speed things along by being explicit. Something like
this perhaps:

import ctypes

# Use ctypes to access unreachable object by memory address.
class PyObject(ctypes.Structure):
_fields_ = [("refcnt", ctypes.c_long)]

d1 = {}
d2 = {}
d1['obj2'] = d2
d2['obj1'] = d1

daddr = id(d1)
print("d1 ref count:", PyObject.from_address(daddr).refcnt)
## output: d1 ref count: 2

del d1, d2
print("d1 ref count:", PyObject.from_address(daddr).refcnt)
## output: d1 ref count: 2

## There's a non-zero refcount even though both dictionaries
## have been removed: d2 couldn't go away completely because
## it was still being referenced in d2, and d2 couldn't go away
## completely because it was still being referenced in d1,
## which couldn't go away because...

## We can change this by marking the values as weak references:

n1 = weakref.WeakValueDictionary()
n2 = weakref.WeakValueDictionary()
n1['obj2'] = n2
n2['obj1'] = n1

naddr = id(n1)
print("n1 ref count:", PyObject.from_address(naddr).refcnt)
## output: n1 ref count: 1

del n1, n2
print("n1 ref count:", PyObject.from_address(naddr).refcnt)
## output: n1 ref count: 0
## :: weakrefs don't contribute to the refcount.


Does this help at all?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python Generator expressions

[Mats Wichmann]

On 7/23/19 11:06 AM, Animesh Bhadra wrote:
> Hi All,
> 
> Need one help in understanding generator expression/comprehensions.
> 
> This is my sample code.
> 
> # This code creates a generator and not a tuple comprehensions.
> my_square =(num *num fornum inrange(11))
> print(my_square) #  at 0x7f3c838c0ca8>
> # We can iterate over the square generator like this.
> try:
> whileTrue:
> print(next(my_square)) # Prints the value 0,1,4
> exceptStopIterationasSI:
> print("Stop Iteration")

is this code you were actually running? because it won't work... an
except needs to be matched with a try, it can't match with a while.

you *can* comsume your the values your generator expression generates by
doing a bunch of next's, but why would you?  Instead, just iterate over
it (every generator is also an iterator, although not vice versa):

for s in my_square:
print(s)

you don't have to manually catch the StopIteration here, because that's
just handled for you by the loop.


> # Another iteration
> forx inmy_square:
> print(x) # This prints nothing.
> 
> 
> Does the generator exhausts its values when we run the iterator once?
> Lastly any specific reason for not having a tuple comprehensions?
> 
> Have checked this link, but could not understood the reason?
> 
>  *
> https://stackoverflow.com/questions/16940293/why-is-there-no-tuple-comprehension-in-python
> 
> 
> Regards,
> Animesh
> 
> 
> 
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] multiprocessing: getting data back from the processes

[Mats Wichmann]

Time for me to ask something...

I've got a scenario where I need to pass data in both directions of a
multiprocessing program.  Specifically what I'm working with is a test
runner from a homegrown testing harness (aside having nothing to do with
the question: if it was me, I would build this starting with pytest, but
the codebase predates pytest and I'm only willing to do so much surgery
at the moment!). The workflow is: the main program either uses it
arguments to figure out what tests to run or scans for them, building up
a list of files which is then modified into a list of Test class instances.

This is passed to a set of workers through a multiprocessing.Queue.
Each worker pulls items off the queue and runs them, which causes the
Test instance to be updated with results. This is easy enough and looks
just like the threaded version of the runner; since everything is
gathered and enqueued before the workers are fired up, finding when
things are done is easy enough. For example, can use the sentinel trick
of writing a None for each worker at the end so each one in turn will
pick up one of those and quit. That's not even necessary, it seems.

The mp version (which I'm doing just as a personal exercise) has a
difference: the main program wants to go through the results and produce
a summary report, which means it needs access to the modified Test
instances. Unlike the threaded version, the list of instances is not
shared between the processes. I figured I'd send the modified instances
back through another queue which the main process reads.  So now after
all this vebiage, the question: how does the main process, acting in a
consumer role on the resultq, read this queue effectively in the
presence of multiple producers?

If it does q.get in a loop, it will eventually block when the queue is
empty.  If it does q.get_nowait, it _might_ see an empty queue when some
tests are pending that haven't been written to the queue yet (especially
since the pickling that takes place on the queue apparently introduces a
little delay)

A somewhat simplified view of what I have now is:

Queue, JoinableQueue imported from multiprocessing

queue = JoinableQueue()
resultq = Queue()
procs = [RunTest(queue=queue, resultq=resultq) for _ in range(jobs)]
total_num_tests = len(tests)
for t in tests:
queue.put(t)
for p in procs:
p.daemon = True
p.start()
queue.join()

# collect back the modified test instances
tests = []
for t in iter(resultq.get, None):
tests.append(t)
# because q.get() blocks, we need a way to break out
# q.empty() is claimed not to be reliable in mp.
if len(tests) >= total_num_tests:
break

This actually works fine based on knowing how many tests there are and
assuming when we've collected the same number, everything is done.  But
it doesn't sound entirely "clean" to me. What if a test is "lost" due to
an error, so its modified Test instance is never written to the result
queue?  Could use a sentinel, but who's going to write the sentinel -
this seems to suffer from the same problem as the non-blocking get, in
that a worker could write a sentinel when *it* is done, but other
workers might still be running their last test and we'll conclude the
queue has been drained a bit too early.  I did play some with trying to
set up a barrier across the workers, so they're all finished before each
writes a sentinel, but that still blocked in the q.get (mea culpa: I'm
violating the guidelines and not posting code for that - I didn't save
that particular hack).

I've read a little bit about multiprocessing.Manager and about
multiprocessing.Pool to look for other approaches, both seemed to have
limitations that made them awkward for this case, probably based on me
failing to understand something.


So... what else should I be trying to make this a little cleaner?

thanks,

-- mats
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Stylometry Help

[Mats Wichmann]

On 7/19/19 4:53 AM, ali...@mailbox.org wrote:
> Hello together,
> 
> I try to write a Python tool but after hours of trying , reading and looking 
> for help nothing an no one would help me. So this is my most desperate call 
> for help. I attached the project and my required files. If there is someone 
> who would help me not loosing my scholarship and so beeing excluded from 
> university you'll be my hero.
> 
> 
> Best regards
> 
> Hanna

Hanna, this ia bit over the top.

If you send code - with descriptions of specific problems and what
you've tried, not just a huge blob of code with a non-specific "it
doesn't work" - pepople here will happily try to help.  There are other
places you can look, like Stack Overflow, again it will require asking
targeted questions.

But you really can't expect us, as a band of spare-time volunteers, to
bail you out of a real or imagined situation where if you can't solve
the problem you'll be sent down. If it's that dire, there is usually
paid tutoring available at most institutions or hanging around them, and
potential loss of a scholarship ought to make that investment worthwhile.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Unit testing: Just the API or internal use only methods, too?

[Mats Wichmann]

On 7/16/19 4:41 PM, boB Stepp wrote:
> Peter Otten, while responding to one of my questions in the past,
> mentioned something in passing that apparently has been mulling around
> in the back of my head.  I don't recall his exact words, but he
> essentially said that I should be testing the public interface to my
> classes, but not the methods only used internally by the class and not
> meant to be publicly accessible.  Is this generally how I should be
> viewing testing?  Would someone be willing to expand at some length on
> this topic?

Test everything (within reason).

If you practice TDD, say, where your tests are the contract for how an
interface shall behave, that makes a lot more sense for public APIs,
which shall not change or be flagged immediately by your unit tests,
than for internal functions which should be able to change within reason
to suit implementation needs.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] raising exceptions in constructor code?

[Mats Wichmann]

On 7/16/19 3:29 PM, James Hartley wrote:
> I ask this having more C++ knowledge than sense.
> 
> There is an adage in the halls of everything Stroustrup that one needs to
> think about how resource allocation will be unwound if an exception is
> thrown.  This gets watered down to the mantra "Don't throw exceptions from
> within constructors."  Does this carry over to Python?  I'm trying to
> develop a Pythonistic mindset as opposed to carrying over old baggage...

If you mean __init__, that's not a constructor, so you should set your
mind at rest :)   It's more properly an "initializer", the instance has
already been constructed when it's called.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Object references in Python

[Mats Wichmann]

On 7/16/19 2:33 PM, Steven D'Aprano wrote:

> x = Parrot()
> 
> Now x is a reference to a Parrot instance. y remains a reference to the 
> list.
> 
> x.colour is a reference to the string "blue" (by default).
> 
> x.speak is a reference to the "speak" method of Parrot objects.
> 
> 
> 
> Does this help?
> 
> 
> 

Let's add one more little cute one for good measure:

>>> def foo():
... print("This function does Foo")
...
>>> foo()
This function does Foo
>>> # we created a function object, and foo is a reference to it
...
>>> x = foo
>>> # x should be a reference to the same object
...
>>> x()
This function does Foo
>>> x is foo
True
>>> def foo():
... print("This function no longer Foos")
...
>>> # we created a new function object, and foo is now a reference to it
...
>>> foo()
This function no longer Foos
>>> x()
This function does Foo
>>> # x is still a reference to the original function object
...
>>> x is foo
False
>>>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Lengthy copyright notices?

[Mats Wichmann]

On 7/15/19 3:25 PM, James Hartley wrote:
> help(module_name) will place any text in the *first* module-level docstring
> into the description section of the help page in Python 3.4.5.  Subsequent
> docstrings found at module level are ignored.
> 
> I have been using this factoid for placement of a copyright & licensing
> notice.  By placing a rather lengthy copyright & license in the code in a
> the second module-level docstring, it is prominent within the code, but not
> cluttering up help() output.
> 
> Two questions.  Is there a more standardized way of including long license
> descriptions in code, & is it documented that any other module-level
> docstring will be ignored in help() output?

Rule #1: it's all opinion in the end...

The common practice is that licence/copyright text is included as a
comment in the code, not in a docstring.

It's only a docstring if it's the first thing in its block, and that is
assigned to the object's __doc__ attribute, and that's what help fishes
out, so yes, that behavior is documented.

Your second dosctring isn't technically a docstring, it's just a string,
which isn't assigned to anything, so it just ends up getting lost as a
runtime thing (no references). Is that what you want?

You're on the right track in your second paragraph - "not cluttering up
help output".  There is _some_ support for this opinion, namely PEP 257

https://www.python.org/dev/peps/pep-0257/#multi-line-docstrings

The 4th, 5th and 6th paragraphs of that section all suggest what should
be in the docstrings, none of them mentions copyright/license, all are
oriented to making the help text usable by users.

I'm sure someone else will have an opinion :)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Reading .csv data vs. reading an array

[Mats Wichmann]

On 7/15/19 1:59 PM, Chip Wachob wrote:
> Mats,
> 
> Thank you!
> 
> So I included the QUOTE_NONNUMERIC to my csv.reader() call and it almost
> worked.
> 
> Now, how wonderful that the scope's csv file simply wrote an s for
> seconds and didn't include quotes.  Now Python tells me it can't create
> a float of s.  Of course I can't edit a 4G file in any editor that I
> have installed, so I have to work with the fact that there is a bit of
> text in there that isn't quoted.

yeah, the chips don't always fall right...

not sure what you're running, there are "stream editors" that can just
work on data as it passes through, even on Windows you can install
environments (cygwin, mingw) where the old "sed" command exists. Of
course Python can do that too, by working line-at-a-time, explicitly by
calling readlines() or implicitly by looping over the file handle. The
latter looks something like this;

with open("/path/to/datafile", "r") as f:
for line in f:
if REDFLAGTEXT in line:  # skip these
continue
do-something-with line


I don't want to be leading you down further ratholes, just trying to
answer the questions as they come up


> 
> Which leads me to another question related to working with these csv
> files. 
> 
> Is there a way for me to tell the reader to skip the first 'n' rows? 
> Or, for that matter, skip rows in the middle of the file? 

it's usually easier to skip based on a pattern, if you can identify a
pattern, but you can certainly also add a counter used to skip.  If 'n'
is always the same!

> 
> A this point, I think it may be less painful for me to just skip those
> few lines that have text.  I don't believe there will be any loss of
> accuracy.
> 
> But, since row is not really an index, how does one conditionally skip a
> given set of row entries?

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Reading .csv data vs. reading an array

[Mats Wichmann]

On 7/15/19 12:35 PM, Chip Wachob wrote:
> Oscar and Mats,
> 
> Thank you for your comments and taking time to look at the snips.
> 
> Yes, I think I had commented that the avg+trigger was = triggervolts in
> my original post.
> 
> I did find that there was an intermediary process which I had forgotten
> to comment out that was adversely affecting the data in one instance and
> not the other.  So it WAS a case of becoming code blind.  But I didn't
> give y'all all of the code so you would not have known that.  My apologies.
> 
> Mats, I'd like to get a better handle on your suggestions about
> improving the code.  Turns out, I've got another couple of 4GByte files
> to sift through, and they are less 'friendly' when it comes to
> determining the start and stop points.  So, I have to basically redo
> about half of my code and I'd like to improve on my Python coding skills.
> 
> Unfortunately, I have gaps in my coding time, and I end up forgetting
> the details of a particular language, especially a new language to me,
> Python.
> 
> I'll admit that my 'C' background keeps me thinking as these data sets
> as arrays.. in fact they are lists, eg:
> 
> [
> [t0, v0],
> [t1, v1],
> [t2, v2],
> .
> .
> .
> [tn, vn]
> ]
> 
> Time and volts are floats and need to be converted from the csv file
> entries.
> 
> I'm not sure that follow the "unpack" assignment in your example of:
> 
> for row in TrigWind:
>     time, voltage = row  # unpack
> 
> I think I 'see' what is happening, but when I read up on unpacking, I
> see that referring to using the * and ** when passing arguments to a
> function...

That's a different aspect of unpacking.  This one is sequnce unpacking,
sometimes called tuple (or seqeucence) assignment.  In the official
Python docs it is described in the latter part of this section:

https://docs.python.org/3/tutorial/datastructures.html#tuples-and-sequences


> I tried it anyhow, with this being an example of my source data:
> 
> "Record Length",202,"Points",-0.005640001706,1.6363
> "Sample Interval",5e-09,s,-0.005639996706,1.65291
> "Trigger Point",1128000,"Samples",-0.005639991706,1.65291
> "Trigger Time",0.341197,s,-0.005639986706,1.60309
> ,,,-0.005639981706,1.60309
> "Horizontal Offset",-0.00564,s,-0.005639976706,1.6363
> ,,,-0.005639971706,1.65291
> ,,,-0.005639966706,1.65291
> ,,,-0.005639961706,1.6363
> .
> .
> .
> 
> Note that I want the items in the third and fourth column of the csv
> file for my time and voltage.
> 
> When I tried to use the unpack, they all came over as strings.  I can't
> seem to convert them selectively..

That's what the csv module does, unless you tell it not to. Maybe this
will help:

https://docs.python.org/3/library/csv.html#csv.reader

There's an option to convert unquoted values to floats, and leave quoted
values alone as strings, which would seem to match your data above quite
well.

> Desc1, Val1, Desc2, TimeVal, VoltVal = row
> 
> TimeVal and VoltVal return type of str, which makes sense.
> 
> Must I go through yet another iteration of scanning TimeVal and VoltVal
> and converting them using float() by saving them to another array?
> 
> 
> Thanks for your patience.
> 
> Chip
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Sat, Jul 13, 2019 at 9:36 AM Mats Wichmann  <mailto:m...@wichmann.us>> wrote:
> 
> On 7/11/19 8:15 AM, Chip Wachob wrote:
> 
> kinda restating what Oscar said, he came to the same conclusions, I'm
> just being a lot more wordy:
> 
> 
> > So, here's where it gets interesting.  And, I'm presuming that
> someone out
> > there knows exactly what is going on and can help me get past this
> hurdle.
> 
> Well, each snippet has some "magic" variables (from our point of view,
> since we don't see where they are set up):
> 
> 1: if(voltage > (avg + triglevel)
> 
> 2: if((voltage > triggervolts)
> 
> since the value you're comparing voltage to gates when you decide
> there's a transition, and thus what gets added to the transition list
> you're building, and the list size comes out different, and you claim
> the data are the same, then guess where a process of elimination
> suggests the difference is coming from?
> 
> ===
> 
> Stylistic comment, I know this wasn't your question.
> 
> >         for row in range (len(TrigWind)):
> 
> Don't do this.  It's not a coding error giving you wrong results, but
> it's not efficient and makes for harder to read code.  You already have
> an iterable in TrigWind.  You the

Re: [Tutor] Reading .csv data vs. reading an array

[Mats Wichmann]

On 7/11/19 8:15 AM, Chip Wachob wrote:

kinda restating what Oscar said, he came to the same conclusions, I'm
just being a lot more wordy:


> So, here's where it gets interesting.  And, I'm presuming that someone out
> there knows exactly what is going on and can help me get past this hurdle.

Well, each snippet has some "magic" variables (from our point of view,
since we don't see where they are set up):

1: if(voltage > (avg + triglevel)

2: if((voltage > triggervolts)

since the value you're comparing voltage to gates when you decide
there's a transition, and thus what gets added to the transition list
you're building, and the list size comes out different, and you claim
the data are the same, then guess where a process of elimination
suggests the difference is coming from?

===

Stylistic comment, I know this wasn't your question.

> for row in range (len(TrigWind)):

Don't do this.  It's not a coding error giving you wrong results, but
it's not efficient and makes for harder to read code.  You already have
an iterable in TrigWind.  You then find the size of the iterable and use
that size to generate a range object, which you then iterate over,
producing index values which you use to index into the original
iterable.  Why not skip all that?  Just do

for row in TrigWind:

now row is actually a row, as the variable name suggests, rather than an
index you use to go retrieve the row.

Further, the "row" entries in TrigWind are lists (or tuples, or some
other indexable iterable, we can't tell), which means you end up
indexing into two things - into the "array" to get the row, then into
the row to get the individual values. It's nicer if you unpack the rows
into variables so they can have meaningful names - indeed you already do
that with one of them. Lets you avoid code snips like  "x[7][1]"

Conceptually then, you can take this:

for row in range(len(Trigwind)):
voltage = float(TrigWind[row][1])
...
edgearray.append([float(TrigWind[row][0]), float(TrigWind[row][1])])
...

and change to this:

for row in TrigWind:
time, voltage = row  # unpack

edgearray.append([float)time, float(voltage)])

or even more compactly you can unpack directly at the top:

for time, voltage in TrigWind:
...
edgearray.append([float)time, float(voltage)])
...

Now I left an issue to resolve with conversion - voltage is not
converted before its use in the not-shown comparisons. Does it need to
be? every usage of the values from the individual rows here uses them
immediately after converting them to float.  It's usually better not to
convert all over the place, and since the creation of TrigWind is under
your own control, you should do that at the point the data enters the
program - that is as TrigWind is created; then you just consume data
from it in its intended form.  But if not, just convert voltage before
using, as your original code does. You don't then need to convert
voltage a second time in the list append statements.

for time, voltage in TrigWind:
voltage = float(voltage)
...
edgearray.append([float)time, voltage])
...


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Multiprocessing with many input input parameters

[Mats Wichmann]

On 7/12/19 5:53 AM, Shall, Sydney via Tutor wrote:
> Thanks Mike,
> 
> But I am still not clear.
> 
> do I write:
> 
> def f([x,y,z]) ?
> How exactly do one write the function and how does one ensure that each 
> positional argument is accounted for.


The concept of packing will be useful, you can use the * operator to
pack and unpack.  A trivial example to get you started:

>>> a = [1, 2, 3, 4]
>>> print(a)
[1, 2, 3, 4]
>>> print(*a)
1 2 3 4

In the first print we print the list, in the second we print the result
of unpacking the list - you see it's now four numbers rather than one
list of four numbers.

In a function definition you can pack with the * operator:

>>> def f(*args):
... print(type(args))
... print(len(args))
... print(args)
...
>>>
>>>
>>> f(1, 2, 3, 4)

4
(1, 2, 3, 4)


Here we called the function with four arguments, but it received those
packed into the one argument args, which is a tuple of length 4.

Python folk conventionally name the argument which packs the positional
args that way - *args - but the name "args" has no magic, its
familiarity just aids in recognition.  By packing your positional args
you don't error out if you're not called with the exact number you
expect (or if you want to accept differing numbers of args), and then
you can do what you need to with what you get.

The same packing concept works for dictionaries as well, here the
operator is **.

>>> def fun(a, b, c):
... print(a, b, c)
...
>>> d = {'a':2, 'b':4, 'c':10}
>>> fun(**d)
2 4 10

What happened here is in unpacking, the keys in the dict are matched up
with the names of the function parameters, and the values for those keys
are passed as the parameters.  If your dict doesn't match, it fails:

>>> d = {'a':2, 'b':4, 'd':10}
>>> fun(**d)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: fun() got an unexpected keyword argument 'd'


Dictionary unpacking looks like:

>>> def fun(**kwargs):
... print(f"{kwargs}")
...
>>>
>>> fun(a=1, b=2, c=3)
{'a': 1, 'b': 2, 'c': 3}

again the name 'kwargs' is just convention.

There are rules for how to mix regular positional args, unpacked
positional args (or varargs), and keyword ares but don't want to go on
forever here...
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Output reason

[Mats Wichmann]

On 7/12/19 11:39 AM, Alan Gauld via Tutor wrote:
> On 12/07/2019 15:24, Gursimran Maken wrote:
> 
>> Can someone please explain me the reason for below output.
> 
> You've been bitten by one of the most common gotchas in Python :-)
> 
>> def fun(n,li = []):
>> a = list(range(5))
>> li.append(a)
>> print(li)
>>
>> fun(4)
>> fun(5,[7,8,9])
>> fun(4,[7,8,9])
>> fun(5) # reason for output (why am I getting to values in this output.)
> 
> When you define a default value in Python it creates the default value
> at the time you define the function. It then uses that value each time a
> default is needed. in the case of a list that means Python creates an
> empty list and stores it for use as the default.

It may help in seeing why this happens to be aware that a def: statement
is an executable statement like any other, which is executed at the time
it is reached in the file.  Running it generates a function object, a
reference to it being attached to the name of the function. Conceptually,

def foo():

is like

foo = FunctionConstructor()

foo ends up referring to an object that is marked as callable so you can
then later call it as foo().  So it makes some sense that some things
have to happen at object construction time, like setting up the things
that are to be used as defaults.


> When you first call the function with the default Python adds values to
> the defaiult list.
> 
> Second time you call the function using the default Python adds (more)
> values to (the same) default list.

FWIW, Python does the same no matter the type of the default argument,
but it only causes the trap we poor programmers fall into if the type is
one that can be modified ("mutable").  If you have fun(n, a=5) or fun(n,
s="stringystuff") those are unchangeable and we don't get this little
surprise.


By the way, checker tools (and IDEs/editors with embedded checking
capabilities) will warn about this, which is a hint on using good tools.
 pylint would tell you this:

W0102: Dangerous default value [] as argument (dangerous-default-value)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Make a linked list subscriptable?

[Mats Wichmann]

On 7/10/19 6:30 PM, Sarah Hembree wrote:
> How might I best make a linked list subscriptable? Below is skeleton code
> for a linked list (my
> actual is much more). I've done __iter__ and __next__ but I would like to
> be able to do start:stop:stride I just can't figure out how. Suggestions or
> just hints please?

As a learning exercise this can be interesting, but as to practical
applications, one would like to ask "why"?  If index into the list is
important, then choose a regular list; the "interesting" part of a
linked list, which is "next node" is then available as index + 1.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] pointers or references to variables or sub-sets of variables query.

[Mats Wichmann]

On 7/6/19 8:39 PM, mhysnm1...@gmail.com wrote:
> Hi all.
> 
> In C, you can use pointers to reference variables, arrays, ETC. In python, I
> do not recall anything specifically that refers to such a capability. What I
> want to do is:
> 
> I want to create different data structures such as dictionaries which
> contain specific  list elements based upon specific keys. The original data
> structure could look like:
> 
> Data = [
>   ['2019-01-19','Fred Flintstone',23],
>   ['2019-02-01','Scooby doo', 99]
> ]
> 
> The above structure does have 100's of elements. I want to reference
> specific lists within the above structure. Using the only method I know how:
> 
> Category = {'under-50':[data[0]], 'over-50':[data[1]]}
> 
> If I understand things correctly with Python. The above will copy the value
> into the list within the key. Not the memory address of the nested list I am
> referencing. I am using a list within the key to permit multiple references
> to different nested lists from the original data structure. The end result
> of the structure for the dict could look like this (using example, not real
> output)
> 
> Category['under-50'] = [ List1 pointer, List22 pointer, List52 pointer]
> 
> I hope the above makes sense. How can this be done? 

It's easy enough to convince yourself that what Alan said is true. You
can, for example, use the id function to show this:

https://docs.python.org/3/library/functions.html#id

# identity of 0'th element of Data:
print(id(Data[0]))
# identity of the list that is the value of 'under-50' key:
print(id(Category['under-50']))
# identity of 0'th element of that list:
print(id(Category['under-50'][0])

the first and third should be the same, showing you that's the same
object referred to by those two places.  Again, like Python's
"variables", these are just references to objects. As in:

item = Data[0]
print(id(item), id(Data[0]))

(note: id() is handy for explorations, especially interactive ones, but
isn't terribly useful for production code.  don't attach any meaning to
the value returned by id() other than "unique" - different Pythons
famously generate different id values, something that's been known to
confuse people doing experiments in, for example, PyPy)

Since you turned up here you sometimes also get free unasked-for advice:
I know this was a toy fragment just to explain the concept you're
getting at, but you'll normally want to build your design in a way that
minimizes "magic".  Using numbered indices into an array-like structure
is one of those bits of magic that raises flags. To refer to Fred's age,
you could end up with  Data[0][2].  That would be pretty ugly, and
worse, hard to remember what it meant.  Try to seek ways you can give
meaningful names to things.  We can make suggestions if that's of
interest (I don't want to belabor the point).
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] enumerate over Dictionaries

[Mats Wichmann]

On 7/4/19 3:53 PM, Alan Gauld via Tutor wrote:

>> Does this means that the Dict is ordered? or it is implementation dependent?
> 
> Neither, it means the items in a list always have indexes
> starting at zero.
> 
> By pure coincidence dictionaries in recent Python versions (since 3.6
> or 3.7???) retain their insertion order. But that was not always the
> case, but the result would have been the same so far as the 0,1,2 bit goes.
> 

To be a little more precise, in 3.6 CPython insertion order was
preserved as an artefact of the new implementation of dicts, but not
promised to be that way. Since 3.7 it is guaranteed (it is actually in
the language specification, so other Pythons have to do this too now).

It's still not the same as a collections.OrderedDict, which has some
useful additional features in case you care a lot about ordering.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Timsort on short lists

[Mats Wichmann]

On 7/2/19 1:48 AM, Jordan Baltes wrote:
> I've been researching python's sorting algorithm, Timsort, and know that
> it's a hybrid between insertion sort (best case time complexity O(n)) and
> merge sort (O(n log(n))), and has an overall time complexity of O(n
> log(n)). What I'm trying to figure out is, when the list is very short,
> let's say 2 or 3 units in length, does the time complexity of Timsort
> become O(n)? I was thinking that the insertion-sort portion of Timsort
> would be able to sort a list of length=3 (such as [8,7,9] or [9,8,7]) in
> O(3) time, therefore O(n). Thoughts?

Try timing it?


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] path

[Mats Wichmann]

On 6/30/19 12:01 AM, ingo wrote:
> 
> 
> On 29-6-2019 15:42, Mats Wichmann wrote:
>>
>> Most people don't use pathlib, and that's kind of sad, since it tries to
>> mitigate the kinds of questions you just asked. Kudos for trying.
> 
> In the end, it works,

Sounds good.  One suggestion - a sort of general programming suggestion
really - whenever you take user input, do some kind of validation on it
before using it.  The context may not be one where anything malicious
could happen, but still, just to catch errors.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] path

[Mats Wichmann]

Sigh... something dropped my raw string, so that was a really bad sample :(

inp = r"c:\test\drive\this"


On Sat, Jun 29, 2019, at 07:44, Mats Wichmann wrote:
> 
> For your example, when you define inp as a string, it needs to be a raw
> string because otherwise Python will interpret the backslash sequences.
> \t means tab, which is why the the results look mangled.
> 
> inp = "c:\test\drive\this"
> 
> 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] path

[Mats Wichmann]

On 6/29/19 6:46 AM, ingo wrote:
> A user has to type a path in the commandline on Win 10, so just a
> string.
> A short excerpt:
> 
> Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64
> bit (AMD64)] on win32
> Type "copyright", "credits" or "license()" for more information.
 inp = "c:\test\drive\this"
 import pathlib
 p=pathlib.PurePath(inp)
 p
> PureWindowsPath('c:\test/drive\this')
 print(pathlib.PurePosixPath(p))
> c:/ est/drive his
 inp
> 'c:\test\\drive\this'
 import os
 print(os.path.normpath(inp))
> c:  est\drive his
 print(pathlib.Path(inp))
> c:  est\drive his

> 
> how to go from a string to a path, how to append to a path (os.path.join
> or / with Path), how to turn it into 'posix'

Most people don't use pathlib, and that's kind of sad, since it tries to
mitigate the kinds of questions you just asked. Kudos for trying.

For your example, when you define inp as a string, it needs to be a raw
string because otherwise Python will interpret the backslash sequences.
\t means tab, which is why the the results look mangled.

inp = "c:\test\drive\this"

If you're going to use pathlib, then may as well use the / operator for
joining,

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] double nodes being enter into tree structure

[Mats Wichmann]

On 6/28/19 12:10 AM, mhysnm1...@gmail.com wrote:

> class Node(object):
> 
> # I am not 100% sure the purpose of (object) does.

as to this bit:  having your class inherit from object means it's a
"new-style class".  That's only significant if you're writing code that
is expected to run under Python 2; in Python 3 all classes are new-style
and you don't need to inherit from object to get that - and if you run a
code checker on your program in a Python 3 context, it will likely tell
you so.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Environment variables and Flask

[Mats Wichmann]

On 6/27/19 11:24 PM, Mayo Adams wrote:
> I have for some time been flummoxed as to the significance of setting
> environment variables, for example in order to run a Flask application.
> What are these environment variables, exactly, and why is it necessary to
> set them? "Googling" here simply leads me into more arcana, and doesn't
> really help.

As others have noted, it's a way to pass information from one process to
another at startup time.   Since this is a Python list, I thought it
might be instructive to show how it works. In Python, you access these
environment variables through a dictionary in the os module, called
environ, which "survives" across the call-another-process boundary,
unlike any normal variables you might set in your program. Here's a
trivial Python app that is able to recognize those environment variables
that begin with MYENV_.  That points up one issue with environment
variables right away: it's a namespace you share with everybody, and
there's a chance someone accidentally is using a variable you think is
important - because it's important to them in their context, not yours.
So tricks like special naming conventions may be useful.

In this snip, we build a dictionary from os.environ, using only the keys
that seem to be "for us":


=== child.py ===
import os

myenv = { k: v for k, v in os.environ.items() if "MYENV_" in k }

print("child found these settings:", myenv)
==

Now write another script which sets a value, then calls the child
script; then sets a different value, then calls the child script.

=== parent.py ===
import os
import subprocess

print("Calling with MYENV_foo set")
os.environ['MYENV_foo'] = "Yes"
subprocess.run(["python", "child.py"])

print("Calling with MYENV_bar set")
os.environ['MYENV_bar'] = "1"
subprocess.run(["python", "child.py"])
==

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python and DB

[Mats Wichmann]

On 6/27/19 3:20 PM, Brave Heart via Tutor wrote:
> I have a little RSS program , I current the out are basically outputted on my 
> screen, but I would like python to write to DB so  I can from DB write on a 
> webpage with PHP...
> 
> Kinda like I want to run a news website ..

There doesn't seem to be a question here.

Does this help in any way?

https://docs.python.org/3.7/library/sqlite3.html


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Range command

[Mats Wichmann]

On 6/26/19 5:07 PM, Brick Howse via Tutor wrote:
> Hello all, 
> 
> New to programming and I noticed the range command did not function like it 
> does in the tutorial. 
> For example,
> I type 
> range(5, 10)
> And the output is 
> range(5, 10)
> 
> In section 4.3
> It shows it should display 
> range(5, 10)
> 5, 6, 7, 8, 9
> 
> I'm using windows 10 
> Python 3.7.3
> 
> Any suggestions is greatly appreciated.

Things change.  You have a Python2-oriented tutorial.  Neither version
is wrong.  In Python2, range returns a list.  In Python3, range returns
a range object, which you can iterate over.  So for a typical use, which
is to iterate over the range, the results are identical:

>>> for i in range(5,10):
... print(i)
...
5
6
7
8
9



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] data structures general query

[Mats Wichmann]

On 6/26/19 11:59 AM, William Ray Wing via Tutor wrote:

> One of the most useful (to me) structures is the double-ended queue ("from 
> collections import deque”).  It creates a queue that can quickly remove an 
> item from one end and add an item to the other.  Particularly useful for 
> displaying a sliding window into time series data, or a moving track of the 
> most recent n observations of a physical measurement.
> 
> Bill

Indeed.  deques are often implemented using double-linked lists, the
Python version is (though you can't get at the details easily).


I forgot to add the snide-comment part of "what are these good for":

(a) binary search trees are excellent for Computer Science professors
who want to introduce recursion into their classes.

(b) all the classic data structures are candidates for being job
interview questions ("implement a linked list in Python on the whiteboard")

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] data structures general query

[Mats Wichmann]

On 6/26/19 4:40 AM, mhysnm1...@gmail.com wrote:
> All,
> 
>  
> 
> General computer science question for data structures.
> 
> When would you use the below structures and why? If you can provide a real
> life example on when they would be used in a program  This would be great. I
> am not after code, just explanation.
> 
>  
> 
> Link lists
> 
> Double link-lists

This becomes a question of what kind of data you have and what you need
to do with it.

Python has "lists", which are what most people think of as arrays - you
reference them by index. Lists have fast access: if you know you want
the 7th element, you just ask for that.  Then if you want to insert a
new element following that, the whole array past that point has to move
to make room - the grunt work is normally abstracted away by languages
that have array datatypes, but it is still happening, same for deleting,
so lots of random inserts/deletes (except at the end) are "expensive".
In linked lists, insert is cheap. You just take the existing element's
"next" field and stick that into the new element's "next" field, and
then stick the new element into the existing element's "next" field.
However, deleting an still isn't cheap: unless your traversal mechansm
saves it on the way to finding a certain element, you have to start from
the beginning and walk through to find the previous element, because
it's this one which has to have its "next" field changed.  A doubly
linked list however makes removal trivial - you already have a reference
to that element via the "previous" field.  For both kinds of linked
lists, searching is expensive as you have to walk the chain.  The
fastest data structure for searching is when the lookup is hashed - this
is what the Python dict type allows for, as the keys are hashed as
they're stored (leading to the sometimes surprising rule for Python
dicts that "key must be hashable"). Other popular data structures are
queues and stacks.  Note there are also circular lists, which only means
the tail hooks back into the head.

So your choices depend on what the usage pattern of your data is. The
fact that Python doesn't have link lists, neither natively or in the
standard library, suggests the designers didn't think they represented a
common enough usage pattern for users of the language.


> Binary trees
> 
> What other trees are their other than hierarchical tree (file systems)

Can let this go without mentioning the Merkle tree, darling of anyone
who's ever dabbled in blockchain :)

> Link lists I would guess be useful in an index for a database? 

I believe the "classic" use case is memory management - keeping track of
chunks of memory.

> Binary trees useful for searches?

it's useful for searches if the "is A less than B" comparison is a
useful property of the data you are representing. If not, it isn't
useful at all as you have no way to construct the tree :)




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Basic Question about Visualization for enduser

[Mats Wichmann]

On 6/25/19 6:39 AM, Sinardy Xing wrote:
> Hi,
> 
> I am a newbie with python and the data visualization.
> I have completed pandas.DataFrame stuff and also the matplotlib.
> 
> All of the example that I learned from internet currently are using the
> Anaconda Jupyter Notebook.
> I know there are API where we can output the result of the graph to png,
> will that be possible all these done automatically and dynamically via an
> apps ?

You can generate graphs with mathplotlib.  If you're specifically
looking for a png, it looks a bit like this (assuming you want to follow
the usual conventions for shortening the name, which certainly isn't
required):

from matplotlib import pyplot as plt

plt.savefig('foo.png')

savefig looks at the file extension in determining what to output.

Without a specific question, I'm not sure what else we can say...


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] replacing a loop

[Mats Wichmann]

On 6/24/19 4:24 PM, johnf wrote:
> Thank you - it worked!  I'm glad you are no longer sleepy!
> 
> Actually I wanted the naming to remain consistent with the other loops
> 
> So the name of the function/method (it is in a class) caused the use of
> the underscore
> 
> locChoices == location choices
> 
> def locChoices(self) cause me to use loc_Choices in the code.
> 
> I am torn about the readability of the code - at least in my mind I can
> read the loop a lot easier than I can read the comprehension.

That's actually fine.  There are plenty of people who don't think
comprehensions are more readable, at least unless they're really simple.
It's an odd self-fulfilling prophecy: if you use them a lot, they
usually start to feel more familiar. If you don't like the look of them,
you don't tend to use them, and they never reach that point :) But it's
also easy to abuse newer syntax - "newer must mean better" rather than
"oh, here's an addition that maybe is cleaner in some cases".

Do what works for you if you're the only person likely to look at the
code later.  If others may need to support it later, you may have to
think a little harder about how to write - readability is not a binary
choice.

> Of course I considered the use of a function where I passed the required
> parameters.  But the issue is the access to the data - that getDataSet()
> was a performance hit when I used a passed parameter.  I could have
> opened/accessed/retrieve the data during the opening but thought it best
> to use a lazy data access as needed.
> 
> Today was not really about performance but more about learning to use
> the comprehension.  You see I did attempt to do create the
> comprehension.  But after seeing your code I realized that I had the '['
> and the '{' confused.  I believed I had to use '{'.  I just reviewed a
> tutorial off the net and that was what they were using.

With curly braces it would be a dictionary comprehension.


I just remembered I once saw a tutorial that I thought did a pretty nice
job on comprehensions, and I had saved the link, again it's down to
personal taste but take a gander at this to see if it makes sense:

https://treyhunner.com/2015/12/python-list-comprehensions-now-in-color/

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python type annotation question

[Mats Wichmann]

On 6/24/19 12:48 PM, Arup Rakshit wrote:
> I am little experimenting with Python type annotation today. I wrote a simple 
> class as below:
> 
 from datetime import date
 class Person:
> ... dob: date
> ... def __init__(self, dob):
> ... self.dob = dob
> ... 
 Person(11)
> <__main__.Person object at 0x10e078128>
 p = Person(11)
 p.dob
> 11
> 
> Although I marked dob as date type, why am I able to assign it an int? So my 
> question how does type annotation helps, when should we use them and what 
> advantages it brings?

it's a mechansim to support static checking in various forms - tools,
IDES, etc.  It also serves to document your intent without having to
write as much in the docstring.

Python remains a "consenting adults" language, that is if you really
want to do something, you can, so the type hints are not enforced at
runtime.  Since it's not always the case that doing something possible
is a good idea, the type hints are there to make it easier to detect issues.


Your annotation looks funky, by the way, I'd expect to see it written as:

> ... def __init__(self, dob: date):
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] replacing a loop

[Mats Wichmann]

On 6/24/19 10:15 AM, johnf wrote:
.
> 
> Since I use a lot of similar loops to populate many dropdown controls I
> started investigating the use of list comprehensions.  But I can't
> figure out how to use them in this loop and wonder if it will improve
> the performance. 

To amplify a tiny bit on what Peter said: comprehensions are a more
concise way to express a loop-to-build-a-collection, but I don't believe
were intended specifically as a performance improvement.

They may be a little faster on the principle of "avoiding dots" - that
is, the lookup every time through your loop from listname.append does
take a little bit of time.  And probably more significantly, append is a
function that has to be called, and the setup stack frame/call/return
sequence also takes some time. So depending on circumstances you may see
insignificant or significant performance differences.

But, again -

Rule number one: only optimize when there is a proven speed bottleneck

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] difference between array([1,0,1]) and array([[1,0,1]])

[Mats Wichmann]

On 6/20/19 5:39 PM, Markos wrote:
> Hi,
> 
> I'm studying Numpy and I don't understand the difference between
> 
 vector_1 = np.array( [ 1,0,1 ] )
> 
> with 1 bracket and
> 
 vector_2 = np.array( [ [ 1,0,1 ] ] )
> 
> with 2 brackets

the first is one-dimensional, the second two-dimensional.  If we expand
how we write the second a bit does it make it more clear?

np.array([
   [1, 0, 1],
   # no other elements
])

the double brackets look magical, but as soon as you have more than one
row it makes sense.

> 
> The shape of vector_1 is:
> 
 vector_1.shape
> (3,)
> 
> But the shape of vector_2 is:
> 
 vector_2.shape
> (1, 3)
> 
> The transpose on vector_1 don't work:
> 
 vector_1.T
> array([1, 0, 1])
> 
> But the transpose method in vector_2 works fine:
> 
 vector_2.T
> array([[1],
>    [0],
>    [1]])
> 
> 
> I thought that both vectors would be treated as an matrix of 1 row and 3
> columns.
> 
> Why this difference?
> 
> Any tip?
> 
> Thank you,
> Markos
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Installing Python v3 on a laptop Windows 10

[Mats Wichmann]

On 6/15/19 3:23 PM, Ken Green wrote:\


You've already gotten some good answers, don't consider this as
contradictory.

> I understood there is a preferable method
> of installing Python into Windows. I pray
> tell on how about to do it, gentlemen.

There isn't, there are actually many ways, and to some extent it depends
on what you want to do.  For example, in addition to what you've heard,
these days there are a ton of people doing data analysis, Big Data, etc.
and they often prefer to install Python through the Anaconda
distribution, which has optimised for getting the particularly relevant
packages installed easily alongside and in sync with Python, and then
keeping those up to date.

In the near future, but maybe not quite there yet, Windows 10 will also
have a Python "app" preinstalled, which, when you launch it, installs
the current version of Python through the Microsoft Store.  I think you
can already install via the Microsoft Store, but it's not something that
magically appears on your system even before you think to look for it...
see here:

https://www.microsoft.com/en-us/p/python-37/9nj46sx7x90p?activetab=pivot:overviewtab

That looks like it wants later than the 1809 version, but should be
ready for the 1903 version of Windows 10?


and you wanted a simple answer

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Follow-up on my removing elements from lists question.

[Mats Wichmann]

On 6/15/19 3:35 AM, mhysnm1...@gmail.com wrote:

Data structure:

  ['123123',[2019-2-18', 'transaction text', 'amount'],

I presume the second opening brace is a typo and was supposed to be a
quote mark?

> The 2nd column where the transaction text I am modifying the content and
> using the end result of the built-up words as the string match as you will
> see in the code. This is all working fine. The last loop in the code I am
> trying to delete the elements in reverse order. This doesn't work. The
> length of the list reduces by 1. When it should have reduced by 42. Is the
> logic wrong? This is in Python 3.6 under windows 10.

There's a lot that looks odd in this code, let me poke at a few:

You _appear_ to be be trying to avoid modifying 'unknown_transactions'
while looping over it, which is admirable if that was the intent:

for e, v in enumerate (unknown_transactions):
if word in unknown_transactions[e][2]:
if not word  in transaction:
transaction[word] = unknown_transactions
else:
transaction[word].append(unknown_transactions)
delete_transactions.append (e)
for del_element in reversed(delete_transactions):
unknown_transactions.pop(del_element)

but that whole sequence is inside another loop over
'unknown_transactions'.  Is that intentional?  It seem odd to loop over
something inside a loop over that thing.

As a hint, if you want to modify a list while looping you can loop over
a copy of it, like

  for s in somelist[:]:

or if you don't like the slice notation,

  for s in list(somelist):

you're looping over 'unknown_transactions' here, but you're not using
the loop value 'v' at all in the body. Perhaps that's the thing you
wanted to add to 'transactions'? adding 'unknown_transactions' seems
strange.

usually if you're using multiple levels of indexing you're not
necessarily wrong, but leaving yourself (or others) hard to read code.
In the past I've written such things and a year later I come back to
look and wonder about the magic in something like [e][2]:

   if word in unknown_transactions[e][2]:

You can unpack each transaction list instead to avoid indexing it, and
if you are deleting on the spot, you don't need the enumerate value
either. So a possible rewrite could be:

  for u in unknown_transactions[:]:
  id, date, text, amount = u
  if word in text:
  if not word in transaction:
  transaction[word] = u
  else:
  transaction[word].append(u)
  unknown_transactions.remove(u)

the inner 'if' statement could become a try block as well if you wanted:

  try:
  transaction[word].append(u)
  except KeyError:
  transaction[word] = u

And you can save a level of indentation by inverting your early
if-statement, from

if line != '':
whole bunch of code

to

if not line:
continue
whole bunch of code

Just to make things a little more readable.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Download audios & videos using web scraping from news website or facebook

[Mats Wichmann]

On 6/14/19 12:35 AM, Sijin John wrote:
> Hello Sir/Mam, 
> I am trying to Download audios & videos using web scraping from news website 
> (eg: https://www.bbc.com/news/video_and_audio/headlines) or Facebook & I 
> could't. So in real scenario is it really possible to download audios/videos 
> using python code ? 

as others have pointed out, remove the "using python code" from this
question to be more accurate.

Modern media-serving websites often try quite hard to not have you be
able to grab the media objects except on their terms.  This often means
they are delivered in a streaming manner through a (non-open) player
app, and there never is a file at all that you're allowed to directly
access.  And there's often some wrapping which ends up delivering
advertising, because that's probably how the site monetizes their
content.  If you're expected to access it, there's usually an API for
that, which you would use rather than scraping.

Obviously people figure out ways, as the youtube downloader shows.



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Running Lib/test/test_shutil.py

[Mats Wichmann]

On 6/13/19 12:19 PM, Peter Otten wrote:
> Tom Hale wrote:
> 
>> Hi all,
>>
>> I hope this is the best place to ask (please let me know if there is a
>> more appropriate list):
>>
>> Checking out CPython v3.8.0b1, I'm trying to run:
>>
>>% python Lib/test/test_shutil.py
> 
> Are you sure 
> 
> % python
> 
> invokes the 3.8 interpreter? 
> 
>> I'm getting:
>>
>> Traceback (most recent call last):
>>File "Lib/test/test_shutil.py", line 19, in 
>>  from shutil import (make_archive,
>> ImportError: cannot import name '_GiveupOnFastCopy' from 'shutil'
>> (/usr/lib/python3.7/shutil.py)
>>
> 
> The traceback suggests that it may be 3.7.

and indeed _GiveupOnFastCopy is a 3.8-only thing (an exception, it turns
out), so this must be the deal.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Where to store test-code?

[Mats Wichmann]

On 6/13/19 2:22 AM, Thomas Güttler wrote:
> Up to now I use this structure:
> 
> src/myapp/setup.py
> src/myapp/myapp/real_code.py
> 
> Now I want to write a test for a method which is implemented real_code.py.
> 
> Where should I write store the file which contains the unittest?
> 
> Is there a guideline for the directory structure of tests?
> 
> I know that there are several ways. I know that all these ways work.
> Nevertheless
> it would be great to have a sane default. If there is a guideline IDEs
> could assist
> to create new tests at a common location.

This question gets asked all the time, and it's hard to answer.

Other than the general requirement that the placement of the test should
not make it hard to find the code to be tested, *your* answer will be
determined by the scope of the project and by preferences.

You can put tests inline in the docstring, if you want to use doctest.
People dismiss this as a viable alternative, but the Python standard
library does this in some places - you don't get to write exhaustive
unit tests this way, but you can have something that serves as an
example and a quick test at the same time.  For some examples, try this:

pydoc difflib

(hopefully you're not on Windows where that's probably not in the search
path)

If using pytest or unittest, you can put your tests in the same
directory as the code to be tested, naming the test file for foo.py as
test_foo.py, that way they're picked up automatically. For your example,
in src/myapp/myapp.

If you want separation of your app/module code and the tests , you can
put them a level up, thus src/myapp.

If you will have a lot of "source" files, you'll probably want a
parallel directory (probably simplifies deploy, if you intend to deploy
the code without the tests), thus src/myapp/tests (which is what it
sounds like what you're angling for us to tell you :) ).  If you do that
though, there's a complication - tests will now have a harder time
finding the module code which is to be imported in running the tests,
and you'll have to take a few extra steps to make that work right.

If this were a TV courtroom drama someone would have risen to say
"objection, calls for speculation" - these things don't have an absolute
answer.

Anyway, the pytest project has some commentary on this that will likely
be more well thought out than my ramblings:

https://docs.pytest.org/en/latest/goodpractices.html

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python printing parentheses and quotes

[Mats Wichmann]

On 6/10/19 10:50 AM, Sai Allu wrote:
> Hello!
> 
> I was just wondering if anybody encountered an issue where the Python 
> interpreter was changing how it interprets print statements. So I'm using 
> default Python on Mac OSX (2.7.10 I'm pretty sure) and running with the 
> "python script.py" command.
> 
> Basically what happened was that I had a few lines in the script like this
> ip = "10.41.17.237"
> print(" Welcome to Squid Monitoring for ", ip)
> print("")
> 
> and the output was like this
> 
> ("   Welcome to Squid Monitoring for 10.41.17.237")
> 
> ("")
> 
> So it was printing parentheses and quotes. The above result might not be 
> exactly accurate because I didn't save the output, but it was something 
> generally like that.

In Python 2, print is a statement. In Python 3 it's a function and
behaves like you're expecting.

However, the behavior you're seeing is odd (printing parentheses is a
surprise unless there's more going on than you've listed)

If you want them consistent across both versions, add a statement at the
very top:

from __future__ import print_function



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Interactive editing of variables.

[Mats Wichmann]

On 6/4/19 2:49 AM, Steven D'Aprano wrote:

> The bad news is that this only works on Linux and other Unix systems 
> with readline installed (which nearly all of them do). It won't work 
> under Windows.
> 
> But if you install the third-party library pyreadline, you *may* be able 
> to use that instead:
> 
> import pyreadline as readline
> 
> and the rest might work. (I don't have Windows and can't try it.)

you don't need to change code, the installation of pyreadline also adds
a stub so that "import readline" works.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Interactive editing of variables.

[Mats Wichmann]

>> The issue I have with a lot of GUI programs built for Python they generally
>> fail in the accessibility department for a screen reader. 
> 
> I can't help there I have nearly zero experience of using accessibility
> tools. But I'd expect any GUI toolkit to work with the standard
> OS tools. After all they are ultimately all built using the
> underlying primitive GUI API

On the gui front,

tk developers make no bones about tk not having been built with
accessibility considerations  tk (and thus tkinter which is just the
Python binding to tk),  is not going to work with a screen reader.

wxPython is probably the best choice, it explicitly has support
(although sadly only for Windows):

https://docs.wxpython.org/wx.Accessible.html

possibly some of the other toolkits do - I wouldn't rule out Qt either,
but haven't any experience.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] is this doable

[Mats Wichmann]

On 5/31/19 1:41 PM, nathan tech wrote:
> Hi there,
> 
> So for a future project of mine, I was wondering something.
> 
> Is it possible, in python, to store a running task id in the registry?
> 
> I might be using the complete wrong terms here, because I'm only used to 
> doing this with a specific language, but here's what I want to do:
> 
> 
> python mytest.py:
> 
> if(registry.taskid==valid_task):
> 
>   print 'already open'
> 
>   send to open program to make a ding noise.
> 
> 
> I understand that the second part, the "send to program" requires the 
> program to handle being sent a "wake up!" event, which is fine, it's the 
> "is it already running" which I am not sure on.

there's a lot your question leaves unasked...  do you want to just code
your own apps and have one be able to poke another? that's one problem,
you can define the interface yourself.  Or do you want to be able to
poke arbitrary running tasks?  that ends up more complicated.  many
systems have notification APIs that you can make use of, some of those
are more oriented to that model (the mobile systems Android and Tizen),
some a little less but still support it (Windows - it's a more prevalent
thing in the UWP model).

the psutil module can let you find things out about processes, might be
useful in your "is the task running" query.

if it's okay to start processes together and it's not arbitrary, the
multiprocessing module may be of some help.




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Query about python recipies for practices

[Mats Wichmann]

On 5/27/19 11:43 AM, boB Stepp wrote:
> On Sun, May 19, 2019 at 12:55 PM bijaya dalei <2212bij...@gmail.com> wrote:
>>
>> Hii, Good morning. I am a new user of python programming language. I have a
>> small query on "where to get python recepies for practices".plz
>> suggest.Thanks.
> 
> It is not very clear to me what you are asking for, which may be why
> you have not gotten any responses so far.
> 
> ActiveState used to have a Python "recipe" section.  Apparently it has
> been moved to a GitHub repository at
> https://github.com/ActiveState/code

and, if you meant problems for practicing... there are a lot of those
around.  Most courses have problems for you to solve, naturally, and a
brief hunt around turned up some sites like these that are not courseware:

http://www.practicepython.org/
https://github.com/zhiwehu/Python-programming-exercises/blob/master/100%2B%20Python%20challenging%20programming%20exercises.txt
https://w3resource.com/python-exercises/

No personal experience of the quality of any of these

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: Re: Setting Command Line Arguments in IDLE

[Mats Wichmann]

> On 26/05/2019 02:55, Richard Damon wrote:
>> I am working on a python script that will be provided arguments when run
>> from the system command line. Is there any place in IDLE to provide
>> equivalent arguments for testing while developing in IDLE?
>>
> 
> There used to be a dialog for that but in the latest version
> of IDLE I can't find it. I wonder when it disappeared and why?
> 
> The best place to ask is probably on the IDLE-dev list.
> 
> It can be found here:
> 
> https://mail.python.org/mailman/listinfo/idle-dev

I've seen this question come up on stack overflow, can't recall I've
seen a completely satisfactory answer.

I would suggest, however, that doing the testing you're considering
should be written as unit tests.  You can invoke unit tests from inside
the program by adding something like this (I'm a pytest fan, but it
could be unittest as well of course):

import pytest

# your code here

if __name__ == "__main__":
pytest.main(["--capture=sys", "name-of-unittest-script.py"])

You can write your own argument array by fiddling with sys.argv; pytest
also provides a mechansim for injecting arguments (I think it's called
pytest_addoption).

The somewhat hacky way for a script to find out that it's running inside
IDLE (note: every time someone asks how to do this, a crowd of people
pop up and say "you don't want to be doing that".  But enabling a
testing scenario might actually be a time you want to?):

import sys

if "idlelib" in sys.modules:
print("We're running in IDLE")



These aren't really an answer to what you're asking for, but maybe some
tools you might use to think further about the problem?

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] regular expressions query

[Mats Wichmann]

On 5/23/19 6:15 PM, mhysnm1...@gmail.com wrote:
> All,
> 
>  
> 
> Below I am just providing the example of what I want to achieve, not the
> original strings that I will be using the regular expression against.
> 
>  
> 
> The original strings could have:
> 
>  
> 
> "Hello world"
> 
> "hello  World everyone"
> 
> "hello everyone"
> 
> "hello   world and friends"
> 
>  
> 
> I have a string which is "hello world" which I want to identify by using
> regular expression how many times:
> 
> * "hello" occurs on its own.
> * "Hello world" occurs in the list of strings regardless of the number
> of white spaces.

I don't know if you've moved on from this problem, but here's one way
one might tackle finding the hello world's in this relatively simple
scenario:

1. join all the strings into a single string, on the assumption that you
care about substrings that span a line break.
2. use the findall method to hit all instances
3. specify the ingore case flag to the re method
4. specify one-or-more bits of whitespace between words of the substring
in your regular expression pattern.

most of that is assumption since as Alan said, you didn't describe the
problem precisely enough for a programmer, even if it sounds precise
enough in English (e.g. hello occurs on its own - does that mean all
instances of hello, or all instances of hello not followed by world?, etc.)

strings = [ all your stuff ]
hits = re.findall(r'hello\s+world', ' '.join(strings), flags=re.IGNORECASE)

Running this on your sample data shows there are three hits (you can do
len(hits) for that)

===

That's the kind of thing regular expressions are good for, but always
keep in mind that they're not always that simple to wrestle with, which
has led to the infamous quote (credited to Jamie Zawinski, although he
repurposed it from an earlier quote on something different):

Some people, when confronted with a problem, think "I know, I'll use
regular expressions." Now they have two problems.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How arguments to the super() function works?

[Mats Wichmann]

On 5/19/19 12:28 AM, Arup Rakshit wrote:
> 
>> On 19-May-2019, at 4:46 AM, Mark Lawrence  wrote:
>>
>> On 18/05/2019 17:21, Arup Rakshit wrote:
>>> I am writing an Flask app following a book, where a piece of python concept 
>>> I am not getting how it works. Code is:
>>> class Role(db.Model):
>>> __tablename__ = 'roles'
>>> id = db.Column(db.Integer, primary_key=True)
>>> name = db.Column(db.String(64), unique=True)
>>> default = db.Column(db.Boolean, default=False, index=True)
>>> permissions = db.Column(db.Integer)
>>> users = db.relationship('User', backref='role', lazy='dynamic')
>>> def __init__(self, **kwargs):
>>> super(Role, self).__init__(**kwargs)
>>> if self.permissions is None:
>>> self.permissions = 0
>>> Here, why super(Role, self).__init__(**kwargs) is used instead of 
>>> super().__init__(**kwargs) ? What that Role and self argument is 
>>> instructing the super() ?
>>> Thanks,
>>> Arup Rakshit
>>> a...@zeit.io
>>
>> Please check this https://www.youtube.com/watch?v=EiOglTERPEo out.  If that 
>> doesn't answer your question please ask again.
>>
> 
> 
> Hello Mark,
> 
> Thanks for sharing the link. Just finished the talk. Little better now about 
> how super uses the MRO.
> 
> class Dad:
> def can_i_take_your_car(self):
> print("No...")
> 
> class Mom(Dad):
> def can_i_take_your_car(self):
> print("Asking your dad...")
> 
> class Victor(Mom, Dad):
> def can_i_take_your_car(self):
> print("Asking mom...")
> 
> class Pinki(Mom, Dad):
> def can_i_take_your_car(self):
> print("I need it today..")
> 
> class Debu(Pinki, Victor):
> def can_i_take_your_car(self):
> super(Victor, self).can_i_take_your_car()
> 
> In this piece of code:
> 
> print(Debu().can_i_take_your_car())
> 
> Why the above call prints "Asking your dad…” but not " print("Asking mom…”)” 
> although can_i_take_your_car is defined inside the class Victor ?

Because that's what you asked for?

in Debu you asked to call the method through the object obtained by
calling super on Victor, and that resolves to Mom per the MRO.  help()
gets that for you, but you can also check it programmatically:

print(Victor.__mro__)

(, , , )


you can also print the method it would resolve to, change class Debu to:

class Debu(Pinki, Victor):
def can_i_take_your_car(self):
print(super(Victor, self).can_i_take_your_car)  # note this is
not a call
super(Victor, self).can_i_take_your_car()

and you can see how it resolves:

>
Asking your dad...


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Case Insensitive Globing

[Mats Wichmann]

On 5/17/19 8:14 PM, Richard Damon wrote:
> I am working on a program to process some files created by an old
> windows program that created it files with varying case with a python
> program.
> 
> Using glob.glob on Windows seems to ignore the case, and find all the
> matching files.
> 
> The same directory, running the same program under Mac OS X, which also
> is a case insensitive file system, is only files that match the case of
> the glob, and is missing many of the files that were found under windows.
> 
> Is there an easy was to make glob match files as a case insensitive manner?
> 
> Or a simple way to do this with something else.
> 
> I am trying to do something like:
> 
>   for file in glob.glob(pattern): processfile(file)
> 

here's a little snippet I've used in the past - uses fnmatch to help
translate the pattern into a regular expression:

import os, re, fnmatch

def findfiles(pattern, path='.'):
rule = re.compile(fnmatch.translate(pattern), re.IGNORECASE)
return [name for name in os.listdir(path) if rule.match(name)]

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] simple question about scope

[Mats Wichmann]

On 5/18/19 2:20 AM, marcus lütolf wrote:
> Dear experts
> 
> in learning the principles of Python I came across scope in the
> control structure's section.
> There I read the notion that variables createted inside a 
> control structute can't be seen or accessed from outside that
> structure, Python would raise a Name Error.
> 
> However in for loops  - also control structures - like in this
> simple example:
> 
> for i in range(1,11):
> sum = 0
> sum += i
> print(sum)
> 
> the print function "returns" the last number of range as 
> value of the variable sum created inside this for loop.
> It should have raised a Name Error instead.
> Could somebody please explain this discrepancy?

in addition to Alan's comment, in your loop you are resetting sum to
zero each time through, which is why at the end of the loop sum is the
same as the finishing value of i, not the sum of the values...




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Consequences of removing python3-venv

[Mats Wichmann]

On 5/9/19 9:52 AM, Jim wrote:
> My Linux mint18 system recently wanted to up date python 3.6. When I
> clicked install I got the following message:
> 
> The following 2 packages will be removed:
> python3-dev
> python3-venv
> 
> I have 2 virtual environments installed: python 3.5.2 & 3.6.7.
> 
> Listing the various pythons I have installed shows:
> 
> jfb@jims-mint18 ~ $ ls -ls /usr/bin/python*
>    0 lrwxrwxrwx 1 root root   9 Nov 23  2017 /usr/bin/python ->
> python2.7
>    0 lrwxrwxrwx 1 root root   9 Nov 23  2017 /usr/bin/python2 ->
> python2.7
> 3412 -rwxr-xr-x 1 root root 3492656 Nov 12 13:46 /usr/bin/python2.7
>    0 lrwxrwxrwx 1 root root   9 Jan  9  2017 /usr/bin/python3 ->
> python3.5
> 4360 -rwxr-xr-x 2 root root 4464368 Nov 12 10:27 /usr/bin/python3.5
>    0 lrwxrwxrwx 1 root root  33 Nov 12 10:27
> /usr/bin/python3.5-config -> x86_64-linux-gnu-python3.5-config
> 4360 -rwxr-xr-x 2 root root 4464368 Nov 12 10:27 /usr/bin/python3.5m
>    0 lrwxrwxrwx 1 root root  34 Nov 12 10:27
> /usr/bin/python3.5m-config -> x86_64-linux-gnu-python3.5m-config
> 4500 -rwxr-xr-x 2 root root 4604416 Oct 25  2018 /usr/bin/python3.6
>    0 lrwxrwxrwx 1 root root  33 Oct 25  2018
> /usr/bin/python3.6-config -> x86_64-linux-gnu-python3.6-config
> 4500 -rwxr-xr-x 2 root root 4604416 Oct 25  2018 /usr/bin/python3.6m
>    0 lrwxrwxrwx 1 root root  34 Oct 25  2018
> /usr/bin/python3.6m-config -> x86_64-linux-gnu-python3.6m-config
>    0 lrwxrwxrwx 1 root root  16 Mar 23  2016 /usr/bin/python3-config
> -> python3.5-config
>    0 lrwxrwxrwx 1 root root  10 Jan  9  2017 /usr/bin/python3m ->
> python3.5m
>    0 lrwxrwxrwx 1 root root  17 Mar 23  2016
> /usr/bin/python3m-config -> python3.5m-config
> 
> So will allowing the update harm my virtual environments?
> 
> I really don't want to reinstall them.

venv shouldn't need to be a separate package, since it's considered
"part of Python3", but I don't know what they're thinking exactly over
on the debian side. Maybe it's only for the command, and the module is
part of the regular distribution?  My pyvenv script says not to use it:

WARNING: the pyenv script is deprecated in favour of `python3.7 -m venv`

In any case, I believe python3-venv is a suitable virtual package, such
that it makes sure the one for the right Python exists - as long as it's
not trying to remove python3-venv you should be okay. Or, maybe Mint
have decided to actually remove the command and you'll have to fall back
to calling it as a module as noted above.

Removing the command won't remove your virtualenvs.  But, the Python
upgrade may cause some of your virtual environments might break, and
you'll have to refresh them - if they point to, rather than duplicate,
system stuff which is changing version.  There's a --upgrade option.  So
tread somewhat carefully.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Can't reinstall pip for Python 3.7.3

[Mats Wichmann]

On 5/8/19 5:36 AM, Zelphy wrote:
> Dear tutor members,
> 
> After spending a lot of time on forums trying to fix this problem, I come
> here hoping for a solution to my problem.
> In fact, after upgrading my version of Python ( I was on 3.6.5 and then on
> 3.7.3 ) I wanted to reinstall pip ( mainly in order to download Pygame )
> But, after downloading Python and the get-pip.py file, when I open it ( and
> it do the same thing every time ) the command prompt tell me "successfully
> uninstalled pip-19.1.1 and then Python stop to work.
> [image: Pip install.PNG]
> I checked the Path but when I open the Script folder, this folder is empty.
> But as I remember, normally there is, in this folder, a pip.exe and some
> other things.
> 
> [image: Path.PNG]
> 
> [image: path Environment variables.PNG]
> 
> I also linked here some screenshots that might help.

Screenshots don't usually make it through the mailing list software.

When installing Python on Windows, you don't need to install pip, so
don't fiddle with the get-pip instructions - it looks like you had it,
then it was uninstalled but not reinstalled.  Just avoid that.

If you do a fresh install and check, it should all be there. The only
thing is that the Scripts directory doesn't go into your environment
path, even if you said to add the path for Python.

Instead of fiddling with that, and assuming you installed the python
launcher (which is a good idea), invoke pip as a module instead of
trying to use pip.exe.  Here's to check if it's working at all:

py -m pip --version

here's to install something:

py -m pip install MyFavoritePackage


If you then need to _upgrade_ pip, you would do it this way:

py -m pip install --upgrade pip
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] don't steel my code Mister user

[Mats Wichmann]

On 5/4/19 8:35 AM, nathan tech wrote:
> It has to be said, after extensive research, and responses here, it 
> seems python was just not designed to be a commercial product.
> 
> Licenses are all well and good, but if you're hacking a product, you're 
> just not going to be stopped by a lisence.
> 
> Furthering to that, if I ever sold products it would be £5, or $7, and 7 
> bucks just isn't worth all the effort to make python difficult to hack.
> 
> Nothing is impossible, but, deterring the average user just for $7? Not 
> worth it.
> 
> Thanks anyway guys.
> 
> Nate


A few more comments to add, this is philosophical and business model
territory, not coding, thus not necessarily the space we here are good at :)

As Alan observed, the purpose of "compile" in Python is not to
obfuscate, it is to improve the chances that the program works right as
delivered  without running into dependency and path problems.
(especially on Windows, a tricky target anyway and you should have
limited confidence any given system has Python installed)

There are people who have been working for years on "protecting" code
written in Python. Some claim to have technologies with very good
success. Personally, I tend to take those claims with a grain of salt,
and won't provide any links; some internet searching should turn up some
ideas.

If you feel you have secrets to protect, or indeed simply want to
prevent people from using your code without paying there are several
different general approaches:

Licensing means - tell people they can't do that.
Technical means - the use of license keys, code obfuscation (*), etc.
Software as a service - put the "meat" of your technology as a web
service, give the application the user runs away for free but use some
sort of API/license key scheme to restrict who can actually use the service.
Better value - just make it not worth the time for people to cheat -
your product is so cheap the work to steal it costs more; you keep
adding new value such that reverse-engineers have to start over
frequently, etc.


There's a page on the Python wiki about this, which really doesn't have
that much information, but I will include anyway:

https://wiki.python.org/moin/Asking%20for%20Help/How%20do%20you%20protect%20Python%20source%20code%3F


* Note on code obfuscation: people can do surprisingly sophisticated
things here.  One approach that has been used is to change the bytecode
emitted so it can only be read by a custom interpreter which you supply.
Dropbox did this.  Of course, researchers figured out how to decode it
anyway. I didn't realize this was so old, time passes...

https://developers.slashdot.org/story/13/08/28/0048238/researchers-reverse-engineer-dropbox-cracking-heavily-obfuscated-python-app

This stuff definitely falls in the catgeory of "is it worth doing all
this to protect an inexpensive app"?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] pip issue

[Mats Wichmann]

On 5/2/19 6:14 PM, Cameron Simpson wrote:
> On 02May2019 17:24, Anil Duggirala  wrote:
>> I executed the pip3 install --user -r
>> contrib/requirements/requirements.txt (I actually did sudo before that).
> 
> Please don't use sudo for this. The notion "install" does not imply
> being root.

> The whole point of --user is to install packages in your personal
> account without troubling with the system packages. Doing that as root
> only installs the packages for root, generally a useless thing as you
> shouldn't be running normal stuff as root.
> 
> Just do pip3 as yourself.

also, despite the large volume of old text that says otherwise, it's
better not to invoke pip as a command, but as a module. Thus, instead of

pip3 install --user blah

do:

python3 -m pip install --user blah

assuming you wanted it to work for "python3". the latter way makes sure
the package ends up in a place that matches the Python you're going to
use - and you are going to use this code from another Python program,
no?  (that change is not going to fix your problem, it's just general
advice). Most Linux systems these days have several Python versions, and
the more you use Python the more likely it is you got something that set
up another Python version (virtualenv, bundled Python, etc). Mac users
often have two: the system one, and the one you actually use for your
own work.  Even Windows users end up with several Pythons, e.g. one you
installed, one that got installed with Visual Studio, etc.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] self.name is calling the __set__ method of another class

[Mats Wichmann]

On 5/2/19 9:25 AM, Arup Rakshit wrote:
> On 30/04/19 6:34 AM, David L Neil wrote:
>> As Steven explained, this is a complex environment where only those
>> with a good understanding of the meta abstractions would even want to
>> play (IMHO). Perhaps you would be better served by actually writing
>> some Python applications, and with such experience under-your-belt,
>> adding these 'advanced knowledge' ideas at some later time, if/when
>> needed?)
> 
> Thanks David for suggesting this. I am going to do that, writing an web
> application using Python Flask. It feels more easier than battling with
> such frustrated abstractions. :) I don't know why python exposed so many
> things, and getter/setter goes wild in that respect.  It was not thathard in 
> the other languages(Ruby, JS).

So let me make this comment: some of us (yeah, I'm guilty of this) like
to expose the underlying details when talking about something.  In my
case, I do that because it's how I myself learn - if the details make
beautiful sense, then the "normal way" complete with syntactic sugar
doesn't feel like a mystery and I'm much happier. Does that mean
everybody needs those details up front? Absolutely not. PyCon talks have
a high risk of doing this - great material, usually, but they got
accepted as talks because they are "revealing secrets". Attendees
already knew how to use the feature but come back from the conference
going "now I understand so much better" and so got their money's worth.

In normal everyday use, getters and setters in Python are simple.

(a) don't use them unless you actually need them, which is rarely - this
isn't C++, Java, etc. where you're supposed to encapsulate everything.
(b) if you do need them, "wrap" a regular attribute access in your class
definition.

So I wouldn't write this:

class C:
def __init__(self, x):
self.__x = x

def get_x(self):
return self.__x

def set_x(self, x):
self.__x = x

# But this:

class C:
def __init__(self, x):
self.x = x


Now if you later found that 'x' needed to be more complex (in this
trivial example, x is only allowed to be between 0 and 100), we can add
property decorators which provide some sort of encapsulation, but leaves
the same API - so the retrofit doesn't break clients:

class C:
def __init__(self, x):
self.x = x

@property
def x(self):
return self.__x

@x.setter
def x(self, x):
if x < 0:
self.__x = 0
elif x > 100:
self.__x = 100
else:
self.__x = x

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] feedparser in python

[Mats Wichmann]

On 4/28/19 6:26 PM, nathan tech wrote:
> Hello everyone,
> 
> Most recently, I have started work using feedparser.
> 
> I noticed, almost straight away, it's a  bit slow.
> 
> For instance:
> 
>      url="http://feeds.bbci.co.uk/news/rss.xml;
> 
>      f1=feedparser.parse(url)
> 
> 
> On some feeds, this can take a few seconds, on the talk python to me 
> feed, it takes almost 10!
> 
> This, obviously, is not ideal when running a program which checks for 
> updates every once in a while. Talk about slow!
> 
> 
> I tried using etag, and modified, but none of the feeds seem to ever 
> have them!
> 
> Similarly, this doesn't seem to work:
> 
>      f2=feedparser.parse(url, f.headers["date"])
> 
> What am I doing wrong?
> 
> Any help appreciated.
> 
> A greatly frustrated Nate

This is just an aside... programs which depend on fetching things from
the Web are candidates for various advanced programming techniques.

One is to not write synchronously, where you ask for some data, process
the data, and present results, as if everything happened instantly.
Instead various techniques like using callbacks, or multiple threads, or
multiprocessing, or Python's asynchronous facilities can be employed.
Documentation for several of those techniques use web communications as
their examples :)  In other words, think of writing your code so other
work can happen while waiting for a particular response (for example
firing off requests to other feeds), and thing of how your update
checking can happen in the background so the data is there when you want
to look at it.

Another is when you write your unit tests, mock the responses from the
internet servers so your tests don't suffer the same delays as
interactive use of the program will see.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help request ERROR installing beautifulsoup

[Mats Wichmann]

On 4/29/19 1:44 AM, Alan Gauld via Tutor wrote:
> On 28/04/2019 17:11, Dr. Luca T wrote:
> ^
>> SyntaxError: Missing parentheses in call to 'print'. Did you mean 
>> print("Unit tests have failed!")?
>> 
> 
>> I use windows 10, python 3.7.3 ...
> 
> The problem is you are running python 2 code using python 3.
> You need to find a python 3 version of your package.
> 

You definitely want beautifulsoup4, not just because the version 3 one
doesn't work with Python 3, but also because the authors tell you not to
use it:

"This package is OBSOLETE. It has been replaced by the beautifulsoup4
package. You should use Beautiful Soup 4 for all new projects."

By the way, if you are using pip to install, it is recommended to use it
through the Python interpreter you intend to use.  Since you're on
Windows, hopefully you've let your install set up the Python Launcher
and then it would look like:

py -m pip install beautifulsoup4


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] str.replace error

[Mats Wichmann]

On 4/25/19 10:29 PM, Steven D'Aprano wrote:
> On Thu, Apr 25, 2019 at 10:46:31AM -0700, Roger Lea Scherer wrote:
> 
>> with open('somefile') as csvDataFile:
>> csvReader = csv.reader(csvDataFile)
>> for row in range(100):
>> a = "Basic P1"
>> str.replace(a, "")
>> print(next(csvReader))
> 
> 
> I'm not quite sure what you expected this to do, but you've 
> (deliberately? accidently?) run into one of the slightly advanced 
> corners of Python: unbound methods. 

accidentally, I believe.

notice that the way the Python 3 page on string methods is written, you
_could_ read it as you are to use the literal 'str' but in fact you are
expected to substitute in the name of your string object.

For this specific case:
===
str.replace(old, new[, count])

Return a copy of the string with all occurrences of substring old
replaced by new. If the optional argument count is given, only the first
count occurrences are replaced.
===

So for the example above you're expected to do (without changing the
range call, which has been commented on elsewhere: you should just
iterate directly over the reader object, that's the way it's designed):

for row in range(100):
a = "Basic P1"
row.replace(a, "")

and then hopefully actually do something with the modified 'row', not
just go on to the next iteration and throw it away...

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Question on implmenting __getitem__ on custom classes

[Mats Wichmann]

On 4/23/19 8:57 AM, Arup Rakshit wrote:
> On 23/04/19 3:40 PM, Steven D'Aprano wrote:
>> Watch out here, you have a mutable default value, that probably doesn't
>> work the way you expect. The default value is created ONCE, and then
>> shared, so if you do this:
>>
>> a = MyCustomList()  # Use the default list.
>> b = MyCustomList()  # Shares the same default list!
>> a.append(1)
>> print(b.list)
>> # prints [1]
>>
>> You probably want:
>>
>>       def __init__(self, list=None):
>>   if list is None:
>>       list = []
>>   self.list = list
> 
> That is really a new thing to me. I didn't know. Why list=None in the
> parameter list is different than in the body of __init__ method? Can you
> elaborate this?
> 

It arises because a function definition is an executable statement, run
right then.  So a default value in the parameter list is created right
then, and then used as needed, and if that default is a mutable (list,
dictionary, etc) you get surprises.  When an empty list is created in
the body, it happens each time the function object is called (it's a
method, but it's still just a function object). You can write some
experiments to show yourself this in action, it usually makes it sink in
more than someone telling you.

And don't worry, this is one of the most famous of all Python beginner
traps, we've all fallen in it.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: uploading images in pygame

[Mats Wichmann]

On 4/16/19 1:54 PM, fatima butt wrote:
> please i am getting error..i am trying to upload image from openspaceart
> from internet . I have downloaded the image on my desktop and i am trying
> to upload this to the pygame.

Please provide us with more information. What are you trying (hint:
*code*), what kind of errors are you getting, etc.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Questions about the deprecation of standard library modules

[Mats Wichmann]

On March 30, 2019 9:13:16 PM MDT, boB Stepp  wrote:
>While reading in chapter 3 of "Learning Python, 5th ed." by Mark Lutz,
>I was playing around with reload() in the imp module.  In the
>interpreter I did a "from imp import reload" and then help(reload).
>This had a warning that it was deprecated.  After a search online I
>found that the entire imp library was deprecated since Python 3.4.
>The aforementioned book covers through Python 3.3.  This got me to
>wondering when the imp module would be removed from Python 3, which
>led me to finding PEP 4.  In it it says:
>
>Module name:   imp
>Rationale: Replaced by the importlib module.
>Date:  2013-02-10
>Documentation: Deprecated as of Python 3.4.
>
>There is no mention as to _when_ the module might be removed.  PEP 4
>says that a deprecated module may be removed as early as the next
>Python release after it becomes deprecated.  Also the PEP states that
>if a module exists in both Python 2.7 and Python 3.5, it will not be
>removed until Python 2.7 is no longer supported, which, I suppose,
>means next year.
>
>So my main question is how does one know in which Python version a
>deprecated module will be removed?  I'm not too concerned about the
>imp module, but _do_ want to know how the removal process works for
>deprecated standard library modules that I might be interested in.

bob, there's no real answer. if you look at tempfile, it says tempfile.mktemp() 
is deprecated since 2.3 (2003) and tempfile.template since 2.0 (2000). both 
still exist and work...  3.7 and much more so 3.8 become a lot more aggressive 
in warning about deprecated behavior, and the warnings alone can be an issue (a 
project I work on has its test suite break completely with 3.8 alpha because it 
tests stdout/stderr being as expected, and the warnings leak into stderr for 
the tests. yes imp is one of those).   So fix when you have to, I guess, but 
preferably be proactive and fix earlier... dont count on having X releases.
-- 
Sent from a mobile device with K-9 Mail. Please excuse my brevity.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] (no subject)

[Mats Wichmann]

On 3/26/19 12:26 PM, Matthew Herzog wrote:
> elif fileDate = datetime.strptime(name[0:8], DATEFMT).date():
>                    ^
> SyntaxError: invalid syntax

are you assigning (=) or comparing (==)?

you can't do both in one statement because of the side effect implications.

[except that starting with Python 3.8 you will be able to (see PEP 572),
but that will be a brand new operator,  := ]


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] (no subject)

[Mats Wichmann]

On 3/23/19 3:16 AM, Peter Otten wrote:

> Personally I would use a try...except clause because with that you can 
> handle invalid dates like _etc.xls gracefully. So
> 
> ERASED = "erased_"
> 
> 
> def strip_prefix(name):
> if name.startswith(ERASED):
> name = name[len(ERASED):]
> return name
> 
> 
> def extract_date(name):
> datestr = strip_prefix(name)[:8]
> return datetime.datetime.strptime(datestr, DATEFMT).date()
> 
> 
> for name in ...:
> try:
> file_date = extract_date(name)
> except ValueError:
> pass
> else:
> print(file_date)

I'd endorse this approach as well.. with a DATEFMT of "%Y%m%d", you will
get a ValueError for every date string that is not a valid date... which
you then just ignore (the "pass"), since that's what you wanted to do.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] My problem in simple terms

[Mats Wichmann]

On 3/21/19 11:54 PM, Edward Kanja wrote:
> Greetings,
> I'm referring to my question i sent earlier, kindly if you have a hint on
> how i can solve
> my problem i will really appreciate. After running regular expressions
> using python
> my output has lot of square brackets i.e. [][][][][][][][][]. How do i
> substitute this with empty
> string so as to have a clear output which i will latter export to an excel
> file.
> Thanks a lot.

I think you got the key part of the answer already: you're getting empty
lists as matches, which when printed, look like []. Let's try to be more
explicit:

$ python3
Python 3.7.2 (default, Jan 16 2019, 19:49:22)
[GCC 8.2.1 20181215 (Red Hat 8.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> help(re.findall)

Help on function findall in module re:

findall(pattern, string, flags=0)
Return a list of all non-overlapping matches in the string.

If one or more capturing groups are present in the pattern, return
a list of groups; this will be a list of tuples if the pattern
has more than one group.

Empty matches are included in the result.


re.findall *always* returns a list, even if there is no match.  If we
add more debug prints in your code so it looks like this:


import re

with open ('unon.txt') as csvfile:

for line in csvfile:

print("line=", line)
index_no=re.findall(r'(\|\s\d{5,8}\s)',line)
print("index_no (type %s)" % type(index_no), index_no)

names=re.findall(r'(\|[A-Za-z]\w*\s\w*\s\w*\s\w*\s)',line)
print("names (type %s)" % type(names), names)
#Address=re.findall(r'\|\s([A-Z0-9-,/]\w*\s\w*\s)',line)

duty_station=re.findall(r'\|\s[A-Z]*\d{2}\-\w\w\w\w\w\w\w\s',line)
print("duty_station (type %s)" % type(duty_station), duty_station)


You can easily see what happens as your data is processed - I ran this
on your data file and the first few times through looks like this:

line=
--

index_no (type ) []
names (type ) []
duty_station (type ) []
line= |Rawzeea NLKPP | VE11-Nairobi
   | 20002254-MADIZ| 00   |
00   |Regular Scheme B | 15-JAN-2019 To 31-DEC-2019 | No   |

index_no (type ) []
names (type ) ['|Rawzeea NLKPP   ']
duty_station (type ) ['| VE11-Nairobi ']
line=
||

index_no (type ) []
names (type ) []
duty_station (type ) []


You see each result of re.findall has given you a list, and most are
empty.  The first and third lines are separators, containing no useful
data, and you get no matches at all. The second line provided you with a
match for "names" and for "duty_station", but not for "index_no".  Your
code will need to be prepared for those sorts of outcomes.

Just looking at the data, it's table data, presumably from a
spreadsheet, but does not really present in a format that is easy to
process, because individual lines are not complete.   A separator line
with all dashes seems to be the marker between complete entries, which
then take up 14 lines, including additional marker lines which follow
slightly different patterns - they may contain | marks or leading spaces.

You will need to decide how regular your table data is and how to work
with it, most examples of handling table data assume that one row is a
complete entry, so you probably won't find a lot of information on this.
 In your case I'm looking at line 2 containing 8 fields, line 4
containing 9 fields, line 6 10 fields, and then lines 8-14 being
relatively free-form consisting of multiple lines.

Is there any chance you can generate your data file in a different way
to make it easier to process?



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Import error, no module named app script

[Mats Wichmann]

On 3/21/19 5:00 PM, nathan tech wrote:
> Hi guys,
> So I recently begun using Mac to program in python.
> For one of the modules I am going to be using, it calls app script in order 
> to send commands to voiceover, which is a screen reader for the MAC.
> The only problem is, it gives an error about no module named appscript. 
> Indeed, when I go into the python command and try import appscript, it says 
> no module named appscript.
> I tried pip install apscript, but it said requirement already satissfied.
> Hoping someone can help.
> Nate

You seem to have a disagreement in spelling - apscript vs. appscript. We
can't tell if that is just a transcription error on your part.

I'd be a bit discouraged by this page:

http://appscript.sourceforge.net/

which says rather loudly:

Please note that appscript is no longer developed or supported, and its
use is not recommended for new projects.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] LPTHW ex15 question

[Mats Wichmann]

On 3/21/19 2:42 PM, Alan Gauld via Tutor wrote:
> On 21/03/19 12:43, Jones, Kayla wrote:
>> ...getting an error message in powershell that I can't figure out. >
>> I've attached a screenshot to help.  Any suggestions would be 
> appreciated.
> 
> First suggestion is not to send images as attachments since
> the server strips them out as potential security risks.
> 
> Instead copy and paste the text into the mail message.
> 
> Also don't assume we know what the book or exercise are about.
> You need to tell us.

I presume the acronym is Learn Python The Hard Way, which many of us
don't have (I do not)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Merge a dictionary into a string

[Mats Wichmann]

On 3/16/19 11:39 AM, Valerio Pachera wrote:
> Consider this:
> 
> import collections
> d = OrderedDict(a='hallo', b='world')
> 
> I wish to get a single string like this:
> 
> 'a "hallo" b "world"'
> 
> Notice I wish the double quote to be part of the string.
> In other words I want to wrap the value of a and b.

So the question that comes to mind is "why"?  I don't mean that in the
negative sense as in you don't want to do that, but your use case may
drive the choice of possible solutions.

For example, I once got asked very nearly this question by someone who
it turned out wanted to serialize the dict into something that could
later be used to load up a dict. String is what he thought of, but in
this case json, or pickle, turned out to be a better solution for what
he wanted than a string.

If you only want to print the string for informational purposes, the
suggestions here will work well.  You can even define your own class
which inherits from OrderedDict and just provides a new definition of
the method which produces a string representation and gives you what you
want without calling anything, like this:

>>> class MyOrderedDict(OrderedDict):
... def __repr__(self):
... return " ".join(f'{k} "{v}"' for k, v in self.items())
...
>>>
>>> d = MyOrderedDict(a='hallo', b='world')
>>> print(d)
a "hallo" b "world"
>>> x = str(d)
>>> x
'a "hallo" b "world"'


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Merge a dictionary into a string

[Mats Wichmann]

On March 16, 2019 5:57:23 PM MDT, Alan Gauld via Tutor  wrote:
>On 16/03/2019 18:44, Peter Otten wrote:
>>
>> In Python 3.6 and above you can use f-strings:
>> 
> d = dict(a="hello", b="world")
> " ".join(f'{k} "{v}"' for k, v in d.items())
>> 'a "hello" b "world"'
>
>Cool, I'd missed f-strings. Time for some reading
>
>Thanks Peter,

f-strings are great, but a lot of people have to support multiple python 
versions so they're not a big option for everyone... yet.
-- 
Sent from a mobile device with K-9 Mail. Please excuse my brevity.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to create a structure from a Log file

[Mats Wichmann]

On March 9, 2019 6:08:03 AM MST, Asad  wrote:
>Hi All ,
>
>   I would like to know , how can I approach this problem to create
>a easy structure from the logfile using python so I can review the
>logfiles
>quite efficiently . Please share suggestion tip or already written
>codes.
>
>
>
>Thanks,
>___
>Tutor maillist  -  Tutor@python.org
>To unsubscribe or change subscription options:
>https://mail.python.org/mailman/listinfo/tutor

The great things about logfiles is they're (typically) all text, and they have 
regular, prredictable  contents.  Before you can do anything useful, though, 
you have to figure out the pattern of the contents.  That can vary wildly... in 
a system logfile there may be many programs that contribute so the patterns can 
take more work to figure out; if it's a logfile for one specific program, it 
may be easier.  What kinds of things are you looking for?  What kind of data 
are you expecting?

You should be able to find lots of  examples of logfile processing on the 
internet, but whether any of that is of any use to you beyond providing a model 
for techniques is up to whether your logfile is like any of the ones being 
processed in those examples.

As in other replies - tell us more.   Also, just a general comment, this list 
usually works out better if you have more specific questions (I tried AAA and 
instead of giving me BBB I got CCC),
-- 
Sent from a mobile device with K-9 Mail. Please excuse my brevity.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] directory structure with tests?

[Mats Wichmann]

On 3/6/19 3:17 PM, James Hartley wrote:
> It is preferable to sprinkle tests files throughout the directories of a
> project, or coalesce all tests in a test directory?

"It depends".

There are people who have strong opinions.

If you're going to use conventions like naming the test for the
functionality in foo.py as test_foo.py, it makes sense for the two to be
nearby each other. That can be the same directory, or a subdirectory (a
harness like Py.Test will find files named test_* and assume they're
tests whichever place they're in). Very large projects may well split
tests into more subdirectories based on their purpose or usage.

On the other hand, if your project is going to be turned into a package,
you might want to keep the tests in a separate directory, as it may be
somewhat easier to select what goes into the package if they're not
sprinkled together.

You may also end up handling unit tests and integration tests differently.

How's that for a definitive answer?  :)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Only appending one object to list, when I am expecting more than 1

[Mats Wichmann]

Without spending a lot of time on an answer, you can use default arguments in 
the cases where only the number of arts differs in your instance creation. If 
there's larger differences, look into @classmethod.

On February 27, 2019 5:25:02 AM PST, AdamC  wrote:
>That's great - bug found. Thanks. However the next question is, how do
>I
>create an instance of a class with variable parameters (i.e. with
>dateAdded
>already computed and stored, or with dateAdded created for the first
>time)?
>
>I hope that makes sense.
>
>Adam
>
>On Tue, 26 Feb 2019 at 18:24, Tobiah  wrote:
>
>>
>>
>> On 2/26/19 6:39 AM, AdamC wrote:
>> > Sorry folks - my code didn't work due to my debug var count to
>ensure
>> that
>> > I was looping properly.
>> >
>> > It should be:
>>
>> As was pointed out, the problem is not in your code.  It's in your
>> data.  You only have one record with a proper 'tpe' value, so that's
>> all you get in your media list.
>>
>>
>> Toby
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>
>-- 
>--
>You back your data up on the same planet?
>http://www.monkeez.org
>PGP key: 0x7111B833
>___
>Tutor maillist  -  Tutor@python.org
>To unsubscribe or change subscription options:
>https://mail.python.org/mailman/listinfo/tutor

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Doubt

[Mats Wichmann]

On 2/26/19 11:36 AM, Maninath sahoo wrote:

is it possible reverse inheritance?
mean child class to parent class



Well, it's Python, so you can do all sorts of things, some of them good 
ideas, some bad...


Like Alan says, the question as asked doesn't really make a lot of sense 
to us, but the way classes (really, user-defined types) work does give 
you a lot of flexibility.  Some people feel like we should express it as 
code reuse at a sophisticated level: if you see a method in another 
class that you'd like to use for your own class, you can inherit that 
method by listing the class in your definition.  Or, if you don't want 
to do so, you write your own method.  So while I wouldn't call it 
reverse inheritance, you as author of your class have all the control of 
what you do and don't take from other classes, and maybe it's not 
entirely accurate to call them child and parent classes because of that?


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to use "curses.resizeterm(nlines, ncols)"

[Mats Wichmann]

All is is_term_resized, resizeterm and the "internal" resize_term 
functions are recent additions :-) From "man 3 resizeterm":


  This extension of ncurses was introduced in mid-1995.  It  was 
  adopted in NetBSD curses (2001) and PDCurses (2003).


For some definition of "recent" :)

I have an odd relationship with curses, I was at UC Berkeley when Ken 
Arnold adapted it from bits of Bill Joy's vi - "we" needed a 
terminal-independent way to address things when the ADM3A terminal and a 
bit later the HP 2621 turned up, making such programming possible (he 
needed it for rogue, as I reacall); the version of curses we all use now 
was shepharded by Pavel Curtis, a classmate of mine from Berkeley High 
School.  That said I don't know any magic stories...  a number of Python 
modules have made an easier to use interface to a popular library but 
curses seems to be pretty much a direct transliteration, warts and all.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to use "curses.resizeterm(nlines, ncols)"

[Mats Wichmann]

On 2/24/19 1:30 PM, boB Stepp wrote:


So what am I misunderstanding?  Can someone show me a code snippet
that I can run which will demonstrate the usefulness and usage of
curses.resizeterm()?

TIA!



If it's snippets you want, I always look at programcreek.  These are 
always part of something bigger so they may not fit your request to have 
them be something you can run.


https://www.programcreek.com/python/example/57429/curses.is_term_resized

(I didn't get any hits when I tried resizeterm directly, even though the 
calls are there...)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Visual studio Code -Python

[Mats Wichmann]

On 2/23/19 5:23 AM, Asad wrote:

Hi All ,

          I am using : pdb.set_trace()
           can you all share some good tricks i using n ,s , l . The 
tedious part which I see is  if it is a loop like for loop then I need 
to do next till the length for the data is completed


for x in string :
      if re.search(x,line)

len(string)
     = 2500

therefore I need to press n 2500 time so that the loop completes and 
goes to another line of code . Any suggestions? how can run the for loop 
without doing n for 2500 times ?


pdb has an "until" command you can use to get it to run until the line 
after the loop, if that's what you are looking for.  sorry, it's a 
little hard to tell what you *are* looking for.



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] import failure

[Mats Wichmann]

On 2/22/19 9:55 AM, Alex Kleider wrote:


(p2) alex@one:$ python src/gdata/oauth/rsa.py
key_factory = '/home/alex/.virtualenvs/p2/local/lib/python2.7/site-packages/tlslite/utils/keyfactory.pyc'> 


(p2) alex@one:$ python
Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.

from tlslite.utils import keyfactory



Details: The initial run failed because of unavailability of a module.
A little reading led to the discovery that it could be easily installed
using pip but after doing so: Still the same failure.
Running the interpreter proves that the module is now available but the
test still fails.
Adding a bit of code to the bottom of the failing module shows that it
in fact can import the module in question.
Ran the interpreter again just to be sure that the function is available
and indeed it is.
I don't understand how this could be possible.

Any suggestions as to what the problem might be or how to investigate
further would be very much appreciated.

Cheers,
Alex


pip installs are specific to the interpreter, you're probably getting a 
mismatch there.


Rule one: install this way:

python -m pip install sheepdip

that way you're sure the pip matches the python and things go in the 
expected place.


Rule 2:
you can do some inspection by printing the values of sys.executable and 
sys.path both in your interactive environment where it works and in your 
script where it doesn't work.


My guess is you'll find a mismatch... but this is only surmise, 
something to try out, we can't see your precise environment.




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Visual studio Code -Python

[Mats Wichmann]

On 2/17/19 1:50 AM, Asad wrote:
> Hi All ,
> 
> I am using Visual Studio Code for Python . However I was using
> the debug option F5 i see it list the variables in my program neatly ,I set
> breakpoints it stops there  but I am unable  to preview each line of the
> execution of the code .
> 
> Thanks,

You'll need to go to the source for that... the Python extension should
have limited instructions, and a bunch of pointers for where to go find
out more, and some animations that intend to show how things work.

Best of luck!


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python Websocket Server

[Mats Wichmann]

On 2/14/19 9:44 AM, Simon Connah wrote:
> Hi,
> 
> I was wondering what the best practice for writing web socket servers in
> Python was in 2019? I found an old example on the web which used the
> tornado library but that was talking about Chrome 22 as the client which
> is ancient now so I'm not sure if things have changed?
> 
> Any suggestions on the best library to use would be greatfully accepted.
> I'd ideally like to be able to do it in an asynchronous manner.
> 
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor


Full Stack Python's page on the topic has quite a lot of resources:

https://www.fullstackpython.com/websockets.html


The thing to keep in mind with old tutorials is that the state of
Python's own async support has evolved a lot recently.  Before, you
absolutely had to depend on a framework which would do the work of
supporting concurrenct connections or you would go insane, but now
you'll start to see examples, I expect, of doing simple things without
bringing in one of the frameworks.  (If your intent is production work,
the well tested frameworks are still going to be the best bet)



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Putting a Bow on It

[Mats Wichmann]

On 2/11/19 6:48 AM, Chip Wachob wrote:
> Thanks.  These are both great helps to get me started.
> 
> The little bit of searching does leave me a little bit confused, but the
> reference to the book is somewhat helpful / encouraging.
> 
> I see a lot of people saying that certain approaches have been depreciated,
> then re-appreciated (?) then depreciated once more and so on..  that sure
> makes it confusing to me.  Unfortunately since I'm using someone's pre-made
> libraries, and that requires 2.7, I'm sort of locked at that version, but
> it seems like most, if not all, of these options will work for any version
> of Python.
> 
> These posts give me some keywords that should help me narrow the field a
> bit.
> 
> I realize that choosing a tool is always a case of personal preference.  I
> don't want to start a 'this is better than that' debate.
> 
> If the 'pros' out there have more input, I'm all ears.

I'm having the same problems, everybody seems to have an idea of what is
state of the art, and they don't often agree. And sadly, people do not
always date their blog entries so you can eliminate what is too old to
be useful to a "newbie" (a category I fall into with packaging)

There are really two classes of solution:

base tools for manually packaging.  The Python Packaging Authority is
supposed to be definitive for what the state of these is.

smart systems which automate some or all of the steps.  These are often
labeled with some sort of hypelabel - Python packaging finally done
right or some such.  (I've tried a couple and they have failed utterly
for the project I want to redo the packaging on. My impression is these
will usually fail if your project is not meant to be imported)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] help

[Mats Wichmann]

>> Error:
>> ./python2.py: line 1: syntax error near unexpected token `('
> 
> That is not a Python error, that's a complaint of your shell.
> If you make a Python script executable you also have to insert the proper 
> hash-bang line. In the case of Python 2
> 
> #!/usr/bin/python2
> 
> will probably work. Example shell session:
> 
> $ cat tmp.py
> def  demo():
> print "heureka"
> 
> demo()
> $ ./tmp.py
> ./tmp.py: line 1: syntax error near unexpected token `('
> ./tmp.py: line 1: `def  demo():'
or, of course, run it with python explicitly:

$ python tmp.py

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Remove soft line break

[Mats Wichmann]

On 2/4/19 10:13 AM, Valerio Pachera wrote:
> 
> I have a file with row that split at the 80th character.
> The next row start with a blank space, meaning that i part of the previous 
> row.
> 
> Example:
> 
> Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam non justo enim. 
> Viv
>  amus dapibus quis neque vitae ornare. Pellentesque at pharetra sapien, id 
> eleif
>  end lacus. Nullam ut semper enim, vulputate venenatis justo. Vestibulum 
> vehicul
>  a dolor sit amet ultricies vulputate. Aenean lobortis, nulla eu scelerisque 
> hen
> 
> What do you suggest to get the text on a single line?

joining the lines is simple - use an empty string as the thing to join
on, and if you want to ditch the beginning space there's a function for
that too (lstrip). But... if you don't have a precise way to know if a
word was split in the middle or on a space (or do you know this?), you
probably need human intervention to reassemble the original.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Not able to get to Script Mode

[Mats Wichmann]

On 1/29/19 2:07 PM, jetspacejt wrote:
> Using version 3.7.1
> Where is File  Edit  Shell...etc.
> Not at the top of my screen

You're looking (probably) for IDLE, not for Python itself
(IDLE is an editor/simple IDE written in Python, that is usually bundled
with Python).

On the wild guess you're using Windows (it always helps to tell us.
Please?), start typing IDLE in the seach box, that should get you there.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python - files

[Mats Wichmann]

On 1/26/19 1:20 AM, Asad wrote:
> Hi All ,
> 
>I would like to know how do I make and option file as an argument on
> command propmnt in python 

I don't know your context for asking this question. Alan has already
explained what you need to do for your issue, and whatever your needs it
is certainly worthwhile to understand how sys.argv works.


Just wanted to add, you find yourself having a need for processing a
non-trivial amount of arguments, this is a problem that is well
addressed in the Python community... the standard library now has three
modules for that - the "original" (UNIX/POSIX like) getopt module, the
currently preferred argparse module, and the optparse module it replaced
(which is now considered deprecated), as well as several very good
modules which are not in the standard library like docopt and click
(just naming two as examples, not endorsing anything).  So you don't
need to reinvent the wheel here if you have heavy duty needs - spend
your time on other parts of the problems you are solving.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] asking questions

[Mats Wichmann]

Just for a bit of amusement, one of the scenarios we see here is famous
enough it has its very own website:

http://xyproblem.info/

enjoy :)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Running Python 3 on Linux Mint

[Mats Wichmann]

On 1/14/19 4:08 PM, Cranky Frankie wrote:
> On Sun, Jan 13, 2019 at 2:04 PM Alan Gauld via Tutor 
> wrote:

>> "You don't say which Mint version but assuming its 17 or
>> greater then you can just use the software manager
>> (or Synaptic) and install the python3 packages."

> Thanks so much, I'm all set now.

Given the impending end of support for Python 2 - officially less than a
year now - distros have been surprisingly slow in switching to Python 3
being the default (meaning instead of having to specially ask for
python3, you would have to specially ask for python2). So don't feel bad
that this wasn't instantly obvious... Mint follows Ubuntu, which follows
Debian (mostly) so it's not exactly the Mint folks' problem, but even
they could have chosen to be proactive if they wanted to.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Doubt

[Mats Wichmann]

On 1/7/19 9:29 AM, Amit Yadav wrote:
> How can simply typing
> 
>  print "hello world"
> 
> work?
> Like without including any header file or import statements how can it work.

because you're using a function (or in your case - which looks like
Python 2.x because it is not written as a function call - a statement)
which is part of the language, there is no need to import anything.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Interpreter pasting Question

[Mats Wichmann]

On 12/28/18 10:13 AM, Avi Gross wrote:

> As one of my programming styles includes lots of hands-on incremental
> analysis of bits and pieces to get them working before combining them,  I
> have no time for an idyllic experience.

don't presume to know what environment will actually work for you, but
that working model sounds somewhat like Jupyter, which has a "notebook"
for recording live code snippets, commentary, etc. - and a way to clip
those bits and pieces into your combination.




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] look back comprehensively

[Mats Wichmann]

On 12/24/18 5:45 PM, Avi Gross wrote:


> As for the UNIX tools, one nice thing about them was using them in a
> pipeline where each step made some modification and often that merely
> allowed the next step to modify that. The solution did not depend on one
> tool doing everything.

I know we're wondering off topic here, but I miss the days when this
philosophy was more prevalent - "do one thing well" and be prepared to
pass your results on in a way that a different tool could potentially
consume, doing its one thing well, and so on if needed.  Of course
equivalents of those old UNIX tools are still with us, mostly thanks to
the GNU umbrella of projects, but so many current tools have grown so
many capabilities they no longer can interact with with other tools in
any sane way.  "pipes and filters" seems destined to be constrained to
the dustbin of tech history.

I'll shut up now...


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] look back comprehensively

[Mats Wichmann]

On 12/24/18 2:14 AM, Alan Gauld via Tutor wrote:
> On 24/12/2018 05:25, Asokan Pichai wrote:
> 
>> That said, sometimes text processing at the shell can precede or even
>> replace some of these. Of course that assumes Unix/Linux OS.

> In fact for most log file analysis I still use [ng]awk.
> Its hard to beat the simplicity of regex based event
> handling for slicing text files.

Sure... there's nothing wrong with using "the appropriate tool for the
job" rather than making everything look like a Python problem.  There's
Perl, too... the ultimate log analysis toolkit. If only I could read
what I wrote a week later :)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] look back comprehensively

[Mats Wichmann]

On 11/14/18 3:01 PM, Steven D'Aprano wrote:
> On Tue, Nov 13, 2018 at 11:59:24PM -0500, Avi Gross wrote:
>> I have been thinking about the thread we have had where the job seemed to be
>> to read in a log file and if some string was found, process the line before
>> it and generate some report. Is that generally correct?
> 
> If that description is correct, then the solution is trivial: iterate 
> over the file, line by line, keeping the previous line:
> 
> previous_line = None
> for current_line in file:
> process(current_line, previous_line)
> previous_line = current_line
> 
> 
> No need for complex solutions, or memory-hungry solutions that require 
> reading the entire file into memory at once (okay for, say, a million 
> lines, but not if your logfile is 2GB in size). If you need the line 
> number:

Absolutely, let's not go reading everything in in bulk, Python has tried
very hard to build elegant iterators all over the place to avoid "doing
the whole thing" when you don't have to - and it has helped heaps with
what years ago used to be an indictment of Python as being "too slow":
not doing work you don't need to do is always a good thing.

The general problem is pretty common, I think, and expands a bit beyond
the trivial case.  Log files may have a start and end marker for a case
you have to examine, and the number of lines between those may be fixed
(0, 1, 2, whatever - 0 being the most trivial case) or variable - I
think that's the situation that started this thread way back, and it
comes up lot. You can have a search on Stack{Exchange,Overflow}, a
non-trivial number of people have asked.

I just now have a different scenario, similar requirement... I happen to
want to scan a bunch of Python code to locate instances of the Python
idiom for ignoring certain possible/expected error conditions:

try:
block of code
except SomeError:
pass


to experiment with replacing those with contextlib.suppress and see if
the team of a particular project thinks that makes code more readable:

from contextlib import suppress
...

with suppress(SomeError):
block of code


This is pretty similar - I want to identify a multi-line sequence that
starts with "try:", has one or more lines, then ends with, in this case,
a two-line sequence where the first line starts with "except" and is
immediately followed by "pass" - but to make it more exciting, is can
then not then followed by either "else" or "finally", because if the try
block has either of those clauses, it is not a candidate for using
suppress instead.  Regexes aren't necessarily helpful on multiline
patterns, even if you ignore the jokes about regexes ("now you have two
problems")

As common as this is, I suspect there are elegant solutions that go
beyond everyone rolling their own.  I'm thinking that maybe pyparsing
has the tools to help with this kind of problem...  I may take a look
into that over then next few days since I just ended up with a personal
interest.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python

[Mats Wichmann]

On 12/20/18 8:49 AM, Mary Sauerland wrote:
> Hi, 
> 
> I want to get rid of words that are less than three characters but I keep 
> getting errors. I tried multiple ways but keep getting errors. 

Just a quick note or two:
> 
> Here is my code:
> 
> f1_name = "/Users/marysauerland/Documents/file1.txt"
> #the opinions
> f2_name = "/Users/marysauerland/Documents/file2.txt"
> #the constitution
> 
> 
> def read_words(words_file):
> return [word.upper() for line in open(words_file, 'r') for word in 
> line.split()]
> 
> 
> read_words(f1_name)

^^^ this line is meaningless.  "everything is an object" in Python. your
function returns a list object - which you don't do anything with. you
should assign a name to it, like:

constitution_words = read_words(f1_name)

Since no name is assigned to that object, Python sees it has no
references, and it is just lost.

and then...

> #performs the function on the file
> set1 = set(read_words(f1_name))
if you saved the object returned from the earlier call to the function,
then you don't need to call the function again, instead you convert the
saved list object to a set object.  We can't tell whether you have an
eventual use for the unfiltered list of words, or only the set of unique
words, the answer to that determines how you write this section.

picking a more descriptive name than set1 would be a good idea (and
f1_name as well, and others - when writing software, the hard part is
maintenance, where you or others have to go in later and fix or change
something.  using meaningful names really helps with that, so it's a
good habit to get into).

since you have sets consisting of words from your two documents, you may
as well use set operations to work with them.  Do you know the set
operation to find all of the members of one set that are also in another
set? hint: in set theory, that is called the intersection.

you say you are trying to remove short words, but there seems to be no
code to do that.  instead you seem to be solving a different problem?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Obfuscated Python [was Long Lines techniques]

[Mats Wichmann]

On 12/13/18 11:15 PM, Avi Gross wrote:
> Steven,
> 
> There are dunderheads who will maliciously misuse things. Yes, we know what
> __add__ is supposed to do. But if someone codes a class with one that
> ignores addition to a collection if it has reached a maximum size, or does
> addition modulo 16 or makes it play a happy birthday tune, while
> subtracting, it may not be trivial for the innocent user of your code to
> make much sense of it. 
> 
> There are documented IDEAS about what many of those dunder methods might
> mean but no enforcement. 

As Raymond Hettinger likes to say, Python is a Consenting Adults
language.  You're not prevented from doing things, even if it might
outrage a few...


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Trouble in dealing with special characters.

[Mats Wichmann]

On 12/7/18 3:20 AM, Steven D'Aprano wrote:

>> How to know whether in a given string(sentence) is there any that is not
>> ASCII character and how to replace?
> 
> That's usually the wrong solution. That's like saying, "My program can't 
> add numbers greater than 100. How do I tell if a number is greater than 
> 100, and turn it into a number smaller than 100?"

yes, it's usually the wrong solution, but in the case of quote marks it
is *possible* is is the wanted solution: certain text editing products
(cough cough Microsoft Word) are really prone to putting in typographic
quote marks.  Everyone knows not to use Word for editing your code, but
that doesn't mean some stuff doesn't make it into a data set we forced
to process, if someone exports some text from an editor, etc. There are
more quoting styles in the world than the English style, e.g. this one
is used in many languages: „quoted text“  (I don't know if that will
survive the email system, but starts with a descended double-quote mark).

It's completely up to what the application needs; it *might* as I say be
appropriate to normalize text so that only a single double-quote and
only a single single-quote (or apostrophe) style is used.  Or it might not.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor