Re: [Tutor] Searching through files for values

[Alan Gauld]

On 14/08/15 05:07, Jason Brown wrote:


for file_list in filenames:

with open(file_list) as files:
 for items in vals:
 for line in files:


Others have commented on your choice of names.
I'll add one small general point.
Try to match the plurality of your names to the
nature of the object. Thus if it is a collection
of items use a plural name.

If it is a single object use a single name.

This has the effect that for loops would
normally look like:

for single name in plural name:

This makes no difference to python but it makes it a lot
easier for human readers - including you - to comprehend
what is going on and potentially spot errors.

Also your choice of file_list suggests it is a list object
but in fact it's not, its' a single file, so simply reversing
the name to list_file makes it clearer what the nature of
the object is (although see below re using type names).

Applying that to the snippet above it becomes:

for list_file in filenames:
with open(list_file) as file:
for item in vals:
for line in file:

The final principle, is that you should try to name variable
after their purpose rather than their type. ie. describe the
content of the data not its type.

Using that principle file might be better named as data
or similar - better still what kind of data (dates,
widgets, names etc), but you don't tell us that...

And of course principles are just that. There will be cases
where ignoring them makes sense too.

HTH
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Searching through files for values

[Jason Brown]

(accidentally replied directly to Cameron)

Thanks, Cameron.  It looks like that value_file.close() tab was
accidentally tabbed when I pasted the code here.  Thanks for the suggestion
for using 'with' though!  That's will be handy.

To test, I tried manually specifying the list:

vals = [ 'value1', 'value2', 'value3' ]

And I still get the same issue.  Only the first value in the list is looked
up.

Jason

On Thu, Aug 13, 2015 at 7:32 PM, Cameron Simpson c...@zip.com.au wrote:

 On 13Aug2015 16:48, Jason Brown zero...@gmail.com wrote:

 I'm trying to search for list values in a set of files.  The goal is to
 generate a list of lists that can later be sorted.  I can only get a match
 on the first value in the list:

 contents of value_file:
 value1
 value2
 value3
 ...

 The desired output is:

 file1 value1
 file1 value2
 file2 value3
 file3 value1
 ...

 Bit it's only matching on the first item in vals, so the result is:

 file1 value1
 file3 value1

 The subsequent values are not searched.


 Rhat is because the subsequent values are never loaded:

 filenames = [list populated with filenames in a dir tree]
 vals = []
 value_file = open(vars)
 for i in value_file:
vals.append(i.strip())
value_file.close()


 You close value_file inside the loop i.e. immediately after the first
 value.  Because the file is closed, the loop iteration stops.  You need to
 close it
 outside the loop (after all the values have been loaded):

value_file = open(vars)
for i in value_file:
vals.append(i.strip())
value_file.close()

 It is worth noting that a better way to write this is:

with open(vars) as value_file:
for i in value_file:
vals.append(i.strip())

 Notice that there is no .close(). The with construct is the pynthon
 syntax to use a context manager, and open(vars) returns an open file,
 which is also a context manager. A context manager has enter and exit
 actions which fire unconditionally at the start and end of the with, even
 if the with is exited with an exception or a control like return or
 break.

 The benefit of this is after the with, the file will _always get
 closed. It is also shorter and easier to read.

 for file_list in filenames:
with open(file_list) as files:
 for items in vals:
 for line in files:
 if items in line:
 print file_list, line


 I would remark that file_list is not a great variable name. Many people
 would read it as implying that its value is a list. Personally I would have
 just called it filename, the singular of your filenames.

 Cheers,
 Cameron Simpson c...@zip.com.au
 ___
 Tutor maillist  -  Tutor@python.org
 To unsubscribe or change subscription options:
 https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Does composition only work with particular instances of objects?

[Alan Gauld]

On 14/08/15 05:31, boB Stepp wrote:

I was looking at an example illustrating composition from the book,
Introducing Python by Bill Lubanovic on p. 140:


class Bill:

 def __init__(self, description):
 self.description = description


class Tail:

 def __init__(self, length):
 self.length = length


class Duck:

 def __init__(self, bill, tail):
 self.bill = bill
 self.tail = tail
 def about(self):
 print('This duck has a', bill.description, 'bill and a',
   tail.length, 'tail.')

Here I was mildly surprised that bill and tail were not Bill and Tail,
and in the about method that self.bill was not used in place of
bill.description, etc.

Continuing:


tail = Tail('long')
bill = Bill('wide orange')
duck = Duck(bill, tail)
duck.about()

This duck has a wide orange bill and a long tail.


This is a rubbish example, or at the kindest an incomplete one.
The problem here is that the class about() method is relying on the 
existence of global variables called bill and tail.
In any sane example it should be using the attributes self.bill and 
self.tail.


If they really did want all instances to share the same bill and tail 
they should have made them class variables rather than instance ones and 
used those in about().


So, in normal use of composition you, Bob, would be right and they 
should use self.bill and self.tail.



So I naively thought I could do the following:


bill0 = Bill('narrow rainbow')
tail0 = Tail('ginormous')


And was surprised by:


duck.about()

This duck has a wide orange bill and a long tail.

duck0 = Duck(bill0, tail0)
duck0.about()

This duck has a wide orange bill and a long tail.


This is because of the hard coded references to the global
vars bill and tail. If they used self.bill and self.tail
everything would work as you expected.

So, unless the book explains why this is bad practice and
goes on to show a good example, I must conclude its a very
bad example.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Does composition only work with particular instances of objects?

[boB Stepp]

On Fri, Aug 14, 2015 at 2:50 AM, Alan Gauld alan.ga...@btinternet.com wrote:

 So, unless the book explains why this is bad practice and
 goes on to show a good example, I must conclude its a very
 bad example.

I found the errata pages for the book
(http://www.oreilly.com/catalog/errata.csp?isbn=0636920028659) and a
Jeff Bienstadt submitted this on Aug 05, 2015.  Mr. Bienstadt did a
fine job of making his points.  The author acknowledged his errors
with:

Thanks for catching this. You are completely correct, and your
example code proves it. I regret the error. I would blame my cat, but
this one wasn't his fault.

So obviously the quality control process let this one slip through the cracks.



-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to effectively use a Student class with SQLite [Was Re: How to design object interactions with an SQLite db?]

[Alan Gauld]

On 14/08/15 03:16, boB Stepp wrote:


Yes, that's a standard problem in any HR type application. Names suck as
database keys. But names are how humans interact.


HR = Human Resources?


Sorry, yes. Anything involving people.


the case of students with duplicate names, she might forget to enter
one (or more, if there are that many).


You can't really mitigate for that kind of error.
Imagine popping up a dialog for every single record saying Are you sure 
you didn't forget somebody else of the same name? How annoying would 
that get?! Even if you only did it during bulk data input it would still 
be a pain.



place and edit that student's data.  It seems that I would always have
to display a certain amount of student information to make it
*obvious* to the user which student she is editing.  Or is there an
easier way?


No, you need to identify which subset of fields are sufficient to always 
identify a student. And that might mean all of them!



I'm guessing this stands for Model-view-controller as in

https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller


Correct. MVC is the standard UI OOP pattern, its been around a long time 
(with various minor variants) and it works. The main simplification is 
to sometimes join the View and Controller

into a singe object. Microsoft did this with their Windows
object model.


is the controller serving as the API between the data (model) and the
display (view)?


No, or only partly. The controller captures and processes user input. It 
controls the state machine that determines which actions are

valid and which are not depending on the states of the Model
and the current Views(there may be multiple views of the same
model) The controller is sometimes known as the policy of
the UI - the rules of how it works. As such the controller is
the thing that determines that an action is valid and makes the
call to the Models UI. In most MVC implementations the Model
will have a list of all open views and send a state update to
those views so they can update themselves. In a few cases the Controller 
does that bit sand the Model replies only to the

controller.

If you are using a network model where different devices may
have views onto the same object then you must use the first
option. In a single desktop app the second option tends to
be more efficient.

In all cases its the Model that exposes the API that provides
the business value/logic of the system. The View and Controller
are purely UI focussed.


So is the idea here to decouple the program logic
from the (possibly complicated) UI, is to have a module (controller?)
which processes data resulting from the program logic (model) and then
decides whether this should go to the menu portion of the display, or
the part that generates a pop-up window, etc.?  And likewise
user-generated events go to the controller module and it decides which
data states get altered in the model?


Pretty close. Any window that displays data(state) of an Model is a 
view. You could have a graphical view and a table view of the same
data for example, and both could be on screen together. Or you could 
have two devices accessing the same data model over a network at the 
same time.


In both cases the Model has multiple views and one or more controllers.


correctly, the program logic (model) and controller need to be
designed in parallel?


Not always, but if you use the use case approach the controller
implements the sequence of events and the model implements the
actual actions. You can extract the actions from the use case and 
implement that separately. But in bigger projects the UI and

database teams will likely work in parallel and with close
co-operation.


lines in other projects I have done.  I will have to search for most
applicable materials to my current project.


A very simple example is the login use case

Actors - User, SUD(System under design), UserDB
Preconditions - None
Postcondition - SUD Home screen displayed to user

1) User starts application
2) SUD displays login screen with username and password fields.
   submit is greyed out
3) User fills in name and password
4) SUD validates field content and enables submit button
5) user hits submit
6) SUD validates user name and password
7) SUD displays home screen

1A1 - SUD fails to launch
1A2 User retries
4A1 User fails to fill in all fields
4A2 SUD does not enable submit
4B1 User fills a field with invalid characters
4B2 SUD presents error message
6A1 User does not exist
6A2 SUD notifies user of error
6B1 Password is invalid
6B2 SUD notifies user of error
6C1 Password has expired
6C2 SUD initiates change password use case.
6D1 After 3 unsuccessful login submissions the SUD
locks the user account, logs a security threat
and notifies the sys admin.

Notice its very common to have more error cases than happy path steps!
Notice too that some steps have multiple error cases.



Hmm.  I was thinking that I would need all the student objects
generated 

Re: [Tutor] Searching through files for values

[Peter Otten]

Jason Brown wrote:

 (accidentally replied directly to Cameron)
 
 Thanks, Cameron.  It looks like that value_file.close() tab was
 accidentally tabbed when I pasted the code here.  Thanks for the
 suggestion
 for using 'with' though!  That's will be handy.
 
 To test, I tried manually specifying the list:
 
 vals = [ 'value1', 'value2', 'value3' ]
 
 And I still get the same issue.  Only the first value in the list is
 looked up.

The problem is in the following snippet:

 with open(file_list) as files:
  for items in vals:
  for line in files:
  if items in line:
  print file_list, line
 

I'll change it to some meaningful names:

with open(filename) as infile:
for search_value in vals:
for line in infile:
if search_value in line:
print filename, has, search_value, in line, line.strip()

You open infile once and then iterate over its lines many times, once for 
every search_value. But unlike a list of lines you can only iterate once 
over a file:

$ cat values.txt
alpha
beta
gamma
$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type help, copyright, credits or license for more information.
 lines = open(values.txt)
 for line in lines: print line.strip()
... 
alpha
beta
gamma
 for line in lines: print line.strip()
... 


No output in the second loop. The file object remembers the current position 
and starts its iteration there. Unfortunately you have already reached the 
end, so there are no more lines. Possible fixes:

(1) Open a new file object for every value:

for filename in filenames:
for search_value in vals:
with open(filename) as infile:
for line in infile:
if search_value in line:
print filename, has, search_value, 
print in line, line.strip()

(2) Use seek() to reset the position of the file pointer:

for filename in filenames:
with open(filename) as infile:
for search_value in vals:
infile.seek(0)
for line in infile:
if search_value in line:
print filename, has, search_value, 
print in line, line.strip()

(3) If the file is small or not seekable (think stdin) read its contents in 
a list and iterate over that:

for filename in filenames:
with open(filename) as infile:
lines = infile.readlines()
for search_value in vals:
for line in lines:
if search_value in line:
print filename, has, search_value, 
print in line, line.strip()

(4) Adapt your algorithm to test all search values against a line before you 
proceed to the next line. This will change the order in which the matches 
are printed, but will work with both stdin and huge files that don't fit 
into memory. I'll leave the implementation to you as an exercise ;)


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] SQLite, Python and SQL injection attacks

[boB Stepp]

I was just looking at the sqlite3 docs at

https://docs.python.org/3/library/sqlite3.html?highlight=sqlite#module-sqlite3

and found the following cheery news:

Usually your SQL operations will need to use values from Python
variables. You shouldn’t assemble your query using Python’s string
operations because doing so is insecure; it makes your program
vulnerable to an SQL injection attack ...

There followed this recommendation:

Instead, use the DB-API’s parameter substitution. Put ? as a
placeholder wherever you want to use a value, and then provide a tuple
of values as the second argument to the cursor’s execute() method...

I have to be honest -- I would have fallen into this potential trap if
I had not read this.  It is not clear to me yet how the recommendation
avoids this issue.  Does the placeholder enforce some sort of type
checking so that arbitrary SQL strings will be rejected?

Having seen this example, are there any other security surprises that
I need to avoid by adopting certain coding techniques when I am using
Python with SQLite?

-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] cannot get a label message to display immediately

[boB Stepp]

On Fri, Aug 14, 2015 at 11:32 AM, Bill Allen walle...@gmail.com wrote:

 I am working in Tkinter.   The scenario is that I click a button that
 starts a function running.   No problem there.   However, the function may
 take some time to run and I do not want the user to be worried.   I am
 wanting to immediately set a label when the function starts to say Please
 Wait.  However, the label does not show up until the function completes.
 How do I get both actions to happen essentially at the same time, the
 writing of the label and the execution of the function?  I have no code to
 show on this one because I am lost in the weeds, not sure of the general
 way to go on this.

I am on the path to learning myself, but couldn't you, as part of the
function, have it either set the label itself or call another function
that does this, and then execute the main part of your function?
Also, just before your function returns its result, it could
clear/rewrite the label.

Additionally, tkinter has the ability to change the cursor to an
hourglass.  You could handle this analogously to what I already said.

HTH,


-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SQLite, Python and SQL injection attacks

[Alan Gauld]

On 14/08/15 19:40, boB Stepp wrote:


Instead, use the DB-API’s parameter substitution. Put ? as a
placeholder wherever you want to use a value, and then provide a tuple
of values as the second argument to the cursor’s execute() method...


This is not a Sqlite issue its true of any database.


I have to be honest -- I would have fallen into this potential trap


Me too, the first time I used a SQL database.
But it didn't take long before a more enlightened colleague
advised me of my ignorance! :-)


I had not read this.  It is not clear to me yet how the recommendation
avoids this issue.  Does the placeholder enforce some sort of type
checking so that arbitrary SQL strings will be rejected?


Yes, it parses the inputs to detect potential issues,
such as rogue semi colons etc.


Having seen this example, are there any other security surprises that
I need to avoid by adopting certain coding techniques when I am using
Python with SQLite?


As I say, it's not just SQLite, its any database.

And the same is true of handling URLs etc you should always
use library parsing and escaping routines to build them.
Especially when inserting data from users or received
data files.

hth
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] cannot get a label message to display immediately

[Alan Gauld]

On 14/08/15 17:32, Bill Allen wrote:

I am working in Tkinter.   The scenario is that I click a button that
starts a function running.   No problem there.   However, the function may
take some time to run


That's the problem right there. You should never kick of an event 
handler that takes a long time to run. Either:

1) Kick of a separate thread to do the back end processing
2) break the function into short chunks and use a timer
(after() in Tkinter) to repeatedly fire the function
(this is the traditional GUI approach)
3) In Python 3.4 use asyncore to create an asynchronous event
loop and feed the functions into that.

Any of these will stop the GUI hanging and enable intermediate
updates (eg a progress bar or countdown) to happen.


How do I get both actions to happen essentially at the same time, the
writing of the label and the execution of the function?


Using the simplest (but least efficient) timer approach convert
code like:

def long_handler():
for item in data:
processItem(item)
# wait. for the loop to end

to

def long_handler()
   update_status()# change the GUI
   getItem(data)  # fetch one item from data
   processItem(item)  # process one item,
   if data:   # is there any more?
  after(20, long_handler)  # then go again after 20ms


This then processes the data items at a rate of 50/second until they 
complete. You can reduce the delay but there reaches a limit where

the returns reduce.

Using threads and/or asyncore should allow you to process the
data in one go in the background with control still returning
to the GUI. But its more complex to set up and test and asyncore
is very new and network focused (Py3.4 only) although in principle
is generic . At this point I'd recommend threads if you have
large data loads to process and asyncore if yuu have a lot of networking 
to do.


HTH
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SQLite, Python and SQL injection attacks

[Emile van Sebille]

On 8/14/2015 11:40 AM, boB Stepp wrote:

I was just looking at the sqlite3 docs at

https://docs.python.org/3/library/sqlite3.html?highlight=sqlite#module-sqlite3

and found the following cheery news:

Usually your SQL operations will need to use values from Python
variables. You shouldn’t assemble your query using Python’s string
operations because doing so is insecure; it makes your program
vulnerable to an SQL injection attack ...


See http://bobby-tables.com/ for more info.

Emile



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] cannot get a label message to display immediately

[Alan Gauld]

On 14/08/15 17:50, Alex Kleider wrote:


Might it be possible to insert the code that posts the 'label' into the
beginning of the function's code block?


That doesn't work because the GUI won't redraw itself until
the event handler finishes and returns control to the Tkinter
event loop. That's why you must avoid long running event
handlers.

You can force a redraw periodically from within the handler
but that doesn't really help much - the GUI is still frozen
for the user which is the biggest issue!.


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] cannot get a label message to display immediately

[Bill Allen]

I am working in Tkinter.   The scenario is that I click a button that
starts a function running.   No problem there.   However, the function may
take some time to run and I do not want the user to be worried.   I am
wanting to immediately set a label when the function starts to say Please
Wait.  However, the label does not show up until the function completes.
How do I get both actions to happen essentially at the same time, the
writing of the label and the execution of the function?  I have no code to
show on this one because I am lost in the weeds, not sure of the general
way to go on this.


Thanks,
--Bill Allen
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] cannot get a label message to display immediately

[Alex Kleider]

On 2015-08-14 09:32, Bill Allen wrote:

I am working in Tkinter.   The scenario is that I click a button that
starts a function running.   No problem there.   However, the function 
may

take some time to run and I do not want the user to be worried.   I am
wanting to immediately set a label when the function starts to say 
Please
Wait.  However, the label does not show up until the function 
completes.

How do I get both actions to happen essentially at the same time, the
writing of the label and the execution of the function?  I have no code 
to
show on this one because I am lost in the weeds, not sure of the 
general

way to go on this.


Thanks,
--Bill Allen


Might it be possible to insert the code that posts the 'label' into the
beginning of the function's code block?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] cannot get a label message to display immediately

[Laura Creighton]

In a message of Fri, 14 Aug 2015 11:32:59 -0500, Bill Allen writes:
I am working in Tkinter.   The scenario is that I click a button that
starts a function running.   No problem there.   However, the function may
take some time to run and I do not want the user to be worried.   I am
wanting to immediately set a label when the function starts to say Please
Wait.  However, the label does not show up until the function completes.
How do I get both actions to happen essentially at the same time, the
writing of the label and the execution of the function?  I have no code to
show on this one because I am lost in the weeds, not sure of the general
way to go on this.


Thanks,
--Bill Allen

Have you ever used threads before?

The standard way to handle this problem is to run your gui in one thread
(the main thread) and then spawn a separate thread to handle any real
work that needs doing.

http://code.activestate.com/recipes/82965-threads-tkinter-and-asynchronous-io/

is one way to handle it, but if you've never used threads before, I
suspect it will be hard to understand.  I'm going to be busy for
several hours -- maybe somebody else here can explain.

Laura

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to skip a single file when using shutil.make_archive()

[Ben Finney]

Anthony Papillion papill...@gmail.com writes:

 I'm creating an archive of a directory using shutil.make_archive and
 need to skip a single file if it is present in that directory. Is
 there a way to do this or should I be looking to ZipFile to meet this
 need?

You can create a hierarchy of files the way you want it, and then use
‘shutil.make_archive’ once the tree is the way you want it.

* Use ‘tempfile.mkdtemp’ to create a unique temporary working directory,
  and bind its name to ‘working_dir’.

* Use ‘shutil.copytree’ to copy the entire hierarchy from its permanent
  location to the temporary ‘working_dir’ location.

* Use other ‘shutil’ functions to manipulate the files in ‘working_dir’
  the way you want.

* Use ‘shutil.make_archive’ to create an archive of the files from
  ‘working_dir’.

* Use ‘shutil.rmtree’ to remove the ‘working_dir’.

-- 
 \ “All television is educational television. The question is: |
  `\   what is it teaching?” —Nicholas Johnson |
_o__)  |
Ben Finney

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SQLite, Python and SQL injection attacks

[Cameron Simpson]

On 14Aug2015 13:40, boB Stepp robertvst...@gmail.com wrote:

I was just looking at the sqlite3 docs at
https://docs.python.org/3/library/sqlite3.html?highlight=sqlite#module-sqlite3
and found the following cheery news:

Usually your SQL operations will need to use values from Python
variables. You shouldn’t assemble your query using Python’s string
operations because doing so is insecure; it makes your program
vulnerable to an SQL injection attack ...

There followed this recommendation:

Instead, use the DB-API’s parameter substitution. Put ? as a
placeholder wherever you want to use a value, and then provide a tuple
of values as the second argument to the cursor’s execute() method...

I have to be honest -- I would have fallen into this potential trap if
I had not read this.  It is not clear to me yet how the recommendation
avoids this issue.  Does the placeholder enforce some sort of type
checking so that arbitrary SQL strings will be rejected?


Well, better to say that it transcribes the values correctly, possibly with 
some type checking. You run the same risk constructing shell command lines too, 
which is why shell=True is generally discourages with subprocess.Popen.


So if you have:

 SELECT FROM tablename WHERE columnvalue = ?

and you have it a python string like foo;bah, the SQL API will take care of 
quoting the string so that the ; is inside the quotes. Likewise if the string 
contains SQL end of string markers (quotes). And if the value cannot be 
transcribed the API should raise an exception.


IN this way you know that the structure of the query has been preserved 
correctly. _And_ you do not need to worry about quoting values (or otherwise 
transcribing them) correctly; that is a solved and debugged problem.


You code is simpler and robust.

Cheers,
Cameron Simpson c...@zip.com.au

The Fano Factor, where theory meets reality.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to skip a single file when using shutil.make_archive()

[Anthony Papillion]

Many thanks Ben! That is exactly what I was looking for and it's super
easy. Thanks again!

On Fri, Aug 14, 2015 at 5:36 PM, Ben Finney ben+pyt...@benfinney.id.au
wrote:

 Anthony Papillion papill...@gmail.com writes:

  I'm creating an archive of a directory using shutil.make_archive and
  need to skip a single file if it is present in that directory. Is
  there a way to do this or should I be looking to ZipFile to meet this
  need?

 You can create a hierarchy of files the way you want it, and then use
 ‘shutil.make_archive’ once the tree is the way you want it.

 * Use ‘tempfile.mkdtemp’ to create a unique temporary working directory,
   and bind its name to ‘working_dir’.

 * Use ‘shutil.copytree’ to copy the entire hierarchy from its permanent
   location to the temporary ‘working_dir’ location.

 * Use other ‘shutil’ functions to manipulate the files in ‘working_dir’
   the way you want.

 * Use ‘shutil.make_archive’ to create an archive of the files from
   ‘working_dir’.

 * Use ‘shutil.rmtree’ to remove the ‘working_dir’.

 --
  \ “All television is educational television. The question is: |
   `\   what is it teaching?” —Nicholas Johnson |
 _o__)  |
 Ben Finney

 ___
 Tutor maillist  -  Tutor@python.org
 To unsubscribe or change subscription options:
 https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] How to skip a single file when using shutil.make_archive()

[Anthony Papillion]

Hello Everyone,

I'm creating an archive of a directory using shutil.make_archive and need
to skip a single file if it is present in that directory. Is there a way to
do this or should I be looking to ZipFile to meet this need?

Thanks
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] try and file existence

[Mark Lawrence]

On 15/08/2015 02:28, Clayton Kirkwood wrote:

try:
 fp = open( user_preferences )
except( PermissionError ):


You need a pass statement here if you don't intend doing anything with 
the error, but see my comments at the bottom.



else:
 with open(user_preferences ) as f:

I originally only had the bottom open statement. Ran but file didn't exist,
and my run failed with file doesn't exist. I figured I'd check to see if the
file existed. This is one of those situations where a search of
documentation for fd_exist (which I thought I'd seen once), or exist turns
up either nothing or nothing relevant. I finally found that the try: clause
with the open statement might help and I copied the snippet to my code. I am
getting an indentation error: expected an indent block. What is wrong, and
what is the best way to find out if a file exists?

TIA,

Clayton



There's nothing to stop you using multiple except statements with one 
try.  So something like this is how I'd go about it.


try:
with open(user_preferences) as f:
do_something()
except PermissionError:
whatever()
except FileNotFoundError:
oh_heck()
etc.

Seee this for an explanation of exception handling 
https://docs.python.org/3/tutorial/errors.html#handling-exceptions.  A 
full list of the exceptions you'd need to consider is here 
https://docs.python.org/3/library/exceptions.html#os-exceptions


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] cannot get a label message to display immediately

[Bill Allen]

On Fri, Aug 14, 2015 at 3:06 PM, Alan Gauld alan.ga...@btinternet.com
wrote:


 That's the problem right there. You should never kick of an event handler
 that takes a long time to run. Either:
 1) Kick of a separate thread to do the back end processing
 2) break the function into short chunks and use a timer
 (after() in Tkinter) to repeatedly fire the function
 (this is the traditional GUI approach)
 3) In Python 3.4 use asyncore to create an asynchronous event
 loop and feed the functions into that.

 ...
 def long_handler()
update_status()# change the GUI
getItem(data)  # fetch one item from data
processItem(item)  # process one item,
if data:   # is there any more?
   after(20, long_handler)  # then go again after 20ms


 Follow my photo-blog on Flickr at:
 http://www.flickr.com/photos/alangauldphotos


 Alan and everyone that responded,

Excellent information!  It was the concepts that I was falling short on an
this helped me a great deal.  In my particular situation, I found using the
after() method indeed worked just fine and was quite simple to implement.
In my case, as simple as this:

def processing(*args):   #my initial button click calls this
'''  display messages in the information message_frame while the data
is processed '''
info.set('PROCESSING, PLEASE WAIT...')   #the label message I was
wanting to get out there to the user
root.after(1000, process_part)  #the long running data process


Thanks again!
Bill Allen
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] try and file existence

[Cameron Simpson]

On 14Aug2015 18:28, Clayton Kirkwood c...@godblessthe.us wrote:

try:
   fp = open( user_preferences )
except( PermissionError ):
else:
   with open(user_preferences ) as f:

I originally only had the bottom open statement. Ran but file didn't exist,
and my run failed with file doesn't exist. I figured I'd check to see if the
file existed. This is one of those situations where a search of
documentation for fd_exist (which I thought I'd seen once), or exist turns
up either nothing or nothing relevant. I finally found that the try: clause
with the open statement might help and I copied the snippet to my code. I am
getting an indentation error: expected an indent block. What is wrong, and
what is the best way to find out if a file exists?


In purely syntactic terms you need some code in the suite under the except 
clause, and you don't want the brackets:


   try:
   fp = open( user_preferences )
   except PermissionError as e:
   print(open(%r) fails: %s % (user_preferences, e))
   else:
   with open(user_preferences ) as f:

In logical terms, the with is not wanted - you're opening the file again.  
Leaving aside the logical issue there, this structure (test then operate) is 
also racy: suppose the file has its attributes changed or is removed between 
the first open and the second.


Next: you're catching PermissionError. That normally means that you have not 
got rights for opening the file; you will get a different exception if the file 
does not exist. You're being too precise if you want both.


But maybe you don't. You need to decide 

Finally, the usual Python pattern is not to catch exceptions _at all_ unless 
you have a deliberate polciy plan for what to do. Normally you would have some 
function looking like this:


 def read_prefs(filename):
   prefs = {}
   with open(filename) as fp:
 ... read preferences, save in prefs dict for example ...
   return prefs

If the file is missing or you have no permissions, that will raise an 
exception. Let it!


Then in an outer layer of your program you might catch the exception, where you 
can make a policy decision because you have a wider view of what is going on:


 try:
   prefs = read_prefs(prefs_filename)
 except FileNotFoundError as e:
   print(warning: file not found: %r: %s % (prefs_filename, e))
   # proceed with empty preferences, not an error
   prefs = {}

This bit of code catches _only_ FileNotFoundError on the (presumed) policy that 
a missing preferences file is ok - your program will proceed with default 
behaviours - but any _other_ kind of exception is not expected - let your 
program abort! Do not proceed!


Cheers,
Cameron Simpson c...@zip.com.au

Capitalism is the extraordinary belief that the nastiest of men, for the
nastiest of reasons, will somehow work for the benefit of us all.
- John Maynard Keynes
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] try and file existence

[Clayton Kirkwood]

try:
fp = open( user_preferences )
except( PermissionError ):
else:
with open(user_preferences ) as f:

I originally only had the bottom open statement. Ran but file didn't exist,
and my run failed with file doesn't exist. I figured I'd check to see if the
file existed. This is one of those situations where a search of
documentation for fd_exist (which I thought I'd seen once), or exist turns
up either nothing or nothing relevant. I finally found that the try: clause
with the open statement might help and I copied the snippet to my code. I am
getting an indentation error: expected an indent block. What is wrong, and
what is the best way to find out if a file exists?

TIA,

Clayton

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] try and file existence

[Steven D'Aprano]

On Fri, Aug 14, 2015 at 06:28:09PM -0700, Clayton Kirkwood wrote:
 try:
 fp = open( user_preferences )
 except( PermissionError ):
 else:
 with open(user_preferences ) as f:


try:
fp = open(user_preferences)
except (IOError, OSError) as e:
handle_error()
else:
with fp as f:
handle_file()


[...]
 what is the best way to find out if a file exists?

Try to open it and see what happens. If the open() succeeds, then the 
file exists and can be read. If it fails, then either the file doesn't 
exist, or it can't be read. Inspect the error to find out which.

There is also os.path.exists(filename), but you should avoid using that 
if possible. The problem is this:

if os.path.exists(filename):
# file exists *right now*
# but a millisecond later, some other program deletes it...
# and now it doesn't exist any more
with open(filename) as f:  # gives an error
...



This is called a time of check to time of use bug, and it is a major 
cause of security bugs in software. Remember, even if you're the only 
*person* using your computer, there could be hundreds of other programs 
running in the background, and one of those might delete the file after 
you've checked its existence.

Also, just because the file *exists* doesn't mean you can open it. 
Perhaps the file is locked, unreadable, you don't have permissions, or 
maybe even the disk is faulty and the file is corrupt and unreadable.


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to effectively use a Student class with SQLite [Was Re: How to design object interactions with an SQLite db?]

[Laura Creighton]

In a message of Thu, 13 Aug 2015 23:42:33 -0500, boB Stepp writes:
Many of my wife's students do have their own email accounts, but,
alas, not all of them.  I have not totally thought this through yet,
but the student data will include their parents' names and some of
their data.  But it will be my luck that two students will have the
same name, John Allan Smith, with their dads having the same name!
But I guess I can list both parents' names.  Surely that would enable
the user to reliably pick the correct student?

As an aside, when discussing my wife's user requirements for this
project, I found out that some of her students have, shall we say, a
variety of parents:  birth parents, multiple step-parents, parents who
are not allowed to have contact with their children, legal guardians
who are not parents, etc.  Ay, yi, yi!


-- 
boB

You've found the 'variety of parents' problem.  Listing the parents'
names will only let your wife know she has the correct student if she
habitually thinks of the parent names when she thinks of the student.
I suspect her internal labelling is more likely to be along the lines
of 'the short one', 'the one who plays the cello', 'the one who used
to have difficulty reading' and 'the one whose water-pistol I
confiscated in the first week of class'.

So you may be better off letting the teacher specify some tags she can use
and apply to any student, which can be of use when you need to
tell one student from another, and the name just isn't doing it for you.
(Perhaps because you have several students with that name, but also because
this is a student you taught many years ago.  The name is vaguely
familiar but the details have blurred over time.  Water-Pistol will
evoke better memories than parents' name in this case, as if you can
barely remember the child you most likely have lost the parents altogether.)

Laura
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to effectively use a Student class with SQLite [Was Re: How to design object interactions with an SQLite db?]

[boB Stepp]

On Fri, Aug 14, 2015 at 4:49 AM, Laura Creighton l...@openend.se wrote:

 You've found the 'variety of parents' problem.  Listing the parents'
 names will only let your wife know she has the correct student if she
 habitually thinks of the parent names when she thinks of the student.

This came to mind because currently, even with three grades in the
same classroom, the total number of students tends to be small
compared to traditional schools.  Also, Montessori parents seem on
average to be very active in their comms with the teacher.  But...

 I suspect her internal labelling is more likely to be along the lines
 of 'the short one', 'the one who plays the cello', 'the one who used
 to have difficulty reading' and 'the one whose water-pistol I
 confiscated in the first week of class'.

 So you may be better off letting the teacher specify some tags she can use
 and apply to any student, which can be of use when you need to
 tell one student from another, and the name just isn't doing it for you.
 (Perhaps because you have several students with that name, but also because
 this is a student you taught many years ago.  The name is vaguely
 familiar but the details have blurred over time.  Water-Pistol will
 evoke better memories than parents' name in this case, as if you can
 barely remember the child you most likely have lost the parents altogether.)

... I have been thinking in terms of only my wife using the software.
If I have the good (or mis-) fortune to create a successful and utile
bit of software, I might find others using the program.  So your
points suggest I should look for a more flexible approach that any
potential user will find effective.



-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor