Re: [Tutor] Question on a select statement with ODBC
-Original Message- From: Tutor [mailto:tutor-bounces+a.bull=pubdmgroup@python.org] On Behalf Of Alan Gauld Sent: Wednesday, October 22, 2014 12:13 PM To: tutor@python.org Subject: Re: [Tutor] Question on a select statement with ODBC On 22/10/14 16:06, Al Bull wrote: I don't think I explained the problem properly. I have several hundred thousand records in the ORD table. There are many instances of records with identical ORD_DBASUB values. Where duplicates exist, I only want to keep the most current record. Ah, OK thats very different. You can do it in SQL but it gets messy and depends on the details of the ODBC SQL, which I don't know... It would involve a nested select with an inner join I suspect. This code works except in very specific cases. Take the following example: ORD_DBASUB DATE 1) 10360 2004-11-02 2) 10360 2004-09-03 3) 10334 2004-04-05 4) 10334 2004-03-08 Record #3 is correctly saved, but record #4 is not removed.It appears that ROW is being moved to the next entry after the ord_rows.remove That's correct you should never modify the collection that you are iterating over with a for loop. Instead convert to using a while loop and only increment the index if you don't remove an thing. Alternatively make a copy of the collection and iterate over that, but a while is usually preferable IMHO. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ [Al Bull] Quick question then... Does this do the trick? Currentrecord = 1 While currentrecord = len(ord_rows): if savedbasub == currentrecord.ord_dbasub: ord_rows.remove(currentrecord) delcount += 1 else: savedbasub = currentrecord.ord_dbasub currentrecord =+ 1 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] yet another misunderstanding on my part
!-Original Message-
!From: Tutor [mailto:tutor-bounces+crk=godblessthe...@python.org] On
!Behalf Of Steven D'Aprano
!Sent: Wednesday, October 22, 2014 4:03 AM
!To: tutor@python.org
!Subject: Re: [Tutor] yet another misunderstanding on my part
!
!On Tue, Oct 21, 2014 at 09:54:49PM -0700, Clayton Kirkwood wrote:
!
! col_position, code, description = 0, [], [] key_name =
! raw_table.replace('\t','\n') for each_line in key_name.splitlines():
! if ':' in each_line:
!code[col_position], description.append() =
! each_line.split(':') #neither works; first one is out of range error,
! 2nd, can't assign to #function. I've used square brackets around
! various sections, and it doesn't like it
!
!The trick to learning how to program is NOT to randomly make changes to
!your code and hope that you get lucky, but to understand why you are
!getting the errors you get.
Trust me, I iterate and look for examples instead of randomly running
around.
Regarding the index out of range, I know what it meant, I was just kind of
surprised that Python didn't automatically create the node and stuff a value
in.
!
!Start here:
!
!position = 0
!codes = []
!codes[position] = 999
!
!
!This fails with an error:
!
!IndexError: list assignment index out of range
!
!
!What does that mean? It's an English sentence, a little bit terse but
!still understandable:
!
!list assignment index -- what's that? It's the index used inside the
!square brackets, being used for list assignment. In this case, the index
!is 0.
!
!out of range -- what's that mean? It tells you that the index you have
!provided (in this case, 0) is too big. Let's experiment to see what too
!big means:
!
!py alist = ['a', 'b', 'c'] # three items alist[0] = 'A'
!py alist[1] = 'B'
!py alist[2] = 'C'
!py alist[3] = 'D'
!Traceback (most recent call last):
! File stdin, line 1, in module
!IndexError: list assignment index out of range
!
!
!Aha! You can only assign to items in a list which already exist. In the
!experiment, I had a list with three items, item 0, item 1 and item 2,
!and assigning to index 0, 1 and 2 succeeded. But assigning to item 3
!(which doesn't exist) fails.
!
!Go back to your list. It is an empty list, [], which means it has *no*
!items in it. Since there are no items in it, you cannot assign to any
!(non-existent) item.
!
!If you can't assign to an item, since there aren't any, you have to
!append to the list.
!
!I'm not sure where you got the idea of writing
!
!description.append() = something
!
!
!from. There's nothing in Python that I've ever seen that suggests that
!would work, and the error message should be clear:
!
!py x() = 23
! File stdin, line 1
!SyntaxError: can't assign to function call
freshfruit = [' banana', ' loganberry ', 'passion fruit ']
[weapon.strip() for weapon in freshfruit]
['banana', 'loganberry', 'passion fruit']
From 5.1.3 in the Python3 tutorial. Although not an = assignment, some value
is being stored in weapon after being strip of whitespace. I realize they
are different, maybe just not sure how different:) In this case, somehow
Python seems to keep track of the location so that the modified
value(dropping spaces) replaces the original location.
!
!
!Try this instead:
!
!py descriptions = []
!py descriptions.append(something blue) descriptions.append(something
!py round) descriptions.append(something flat)
!py print(descriptions)
!['something blue', 'something round', 'something flat']
!
!
!Like the name suggests, append appends something to the end of the
!list. So your code, which started like this:
!
!# doesn't work
!col_position, code, description = 0, [], [] key_name =
!raw_table.replace('\t','\n') for each_line in key_name.splitlines():
!if ':' in each_line:
!code[col_position], description.append() = each_line.split(':')
!
!
!could be written something like this:
!
!# note plural names
!codes, descriptions = [], []
!key_name = raw_table.replace('\t','\n')
!for each_line in key_name.splitlines():
!if ':' in each_line:
!code, description = each_line.split(':')
!codes.append(code)
!descriptions.append(description)
So two questions remain. Why can't codes.append(code) just replace the code
in the previous line and descriptions.append(description) replace the
description in the previous line. Previous emails have seen something like
value.strip() = some value . Second question, why can't a numeric index be
used to make assignment to a specific location like a[1] = some value? If
the mechanism is to use a.index(1,some value), it seems somewhat clumsy.
It is clear that once the array is created, it is possible to access the
value by numeric indexing such as:
print( col_position, code[col_position], description[col_position])
which produces:
83 r7 Price / EPS Estimate Next Year
84 s7 Short Ratio
Also,
# create a list of 2-tuples like (number, square)
[(x, x**2) for x in range(6)]
[(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)]
Why does this
Re: [Tutor] Passing Data to .DLL
It's working!
I don't understand the line
rx_buf = (c_uint8 * rx_buf_size).from_buffer_copy(string_buf)
or where .from_buffer_copy() came from but it works...
I promise I will not knowingly pass Python strings to C.
Thanks.
-Original Message-
From: eryksun [mailto:eryk...@gmail.com]
Sent: Wednesday, October 22, 2014 4:16 AM
To: Wilson, Pete
Cc: tutor@python.org
Subject: Re: [Tutor] Passing Data to .DLL
On Tue, Oct 21, 2014 at 7:04 PM, Wilson, Pete pete.wil...@atmel.com
wrote:
ProcessIncomingSerialData_t = CFUNCTYPE(None, POINTER(c_uint8),
c_uint16) process_incoming_serial_data =
pt_dll.ProcessIncomingSerialData
process_incoming_serial_data.argtypes
= [ProcessIncomingSerialData_t]
ProcessIncomingSerialData takes two parameters: uint8_t *rx_buf and
uint16_t size.
process_incoming_serial_data.argtypes = [POINTER(c_uint8),
c_uint16]
process_incoming_serial_data.restype = None
The buffer parameter isn't const, so pass a copy of string_buf.
size = len(string_buf)
rx_buf = (c_uint8 * size).from_buffer_copy(string_buf)
process_incoming_serial_data(rx_buf, size)
I'm going to meander off topic a bit to give you a warning...
Python strings are immutable by contract, so don't pass them directly
to C functions that might modify them. String immutability makes
interning possible, sometimes just locally in the code object and
sometimes globally in the interpreter. Here's an example of both in
CPython 2.7:
import abc
from ctypes import *
def test():
s = 'abc'
a = cast(s, POINTER(c_char * 3))[0]
a[:] = 'cba'
print 'abc'
test.__code__.co_consts
(None, 'abc', 3, 0, 'cba')
Notice that the 2nd constant in co_consts is 'abc'. This gets stored to
s and later printed. In between I use ctypes to reverse it. So what
gets printed? If you guessed cba, you're right.
test()
cba
Look at the constants now:
test.__code__.co_consts
(None, 'cba', 3, 0, 'cba')
So how about referencing the abc module?
abc
Traceback (most recent call last):
File stdin, line 1, in module
NameError: name 'abc' is not defined
OK, but do you think we can use cba instead?
cba
Traceback (most recent call last):
File stdin, line 1, in module
NameError: name 'cba' is not defined
This is yet another problem with mucking with the internal state of an
immutable object. Equal objects are supposed to hash the same. 'cba'
is now equal to the interned 'abc' string that I reversed, but it
probes to a different slot in the dict's hash table.
test.__code__.co_consts[1] == 'cba'
True
hash(test.__code__.co_consts[1]) == hash('cba')
False
OTOH, a new 'abc' string probes to the same slot in the hash table, but
it's no longer equal (i.e. it gets handled as a hash collision).
test.__code__.co_consts[1] == 'abc'
False
hash(test.__code__.co_consts[1]) == hash('abc')
True
This breaks dict lookup unless we use the interned string itself:.
globals()[test.__code__.co_consts[1]]
module 'abc' from '/usr/lib/python2.7/abc.pyc'
So the moral is only pass Python strings to C functions that promise
(scout's honor) to not modify them. If the parameter isn't const, err
on the side of caution; copy the string to a ctypes buffer.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Passing Data to .DLL
No worries, if I could spell I would have been a Lawyer. Pete -Original Message- From: eryksun [mailto:eryk...@gmail.com] Sent: Wednesday, October 22, 2014 5:36 PM To: Wilson, Pete Cc: tutor@python.org Subject: Re: [Tutor] Passing Data to .DLL On Wed, Oct 22, 2014 at 6:05 PM, eryksun eryk...@gmail.com wrote: from_buffer_copy is similar, accept instead of sharing the buffer That should be ex-cept (conjunction for an exception clause), not ac- cept (verb, to receive). I missed that in my initial proofread. It takes a while to clear my mental buffer enough for a fresh look. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] yet another misunderstanding on my part
Clayton Kirkwood c...@godblessthe.us Wrote in message:
(Somehow, your email program seems to be using the exclamation
point to identify quoted lines, instead of the standard
greater-than symbol. Is that something you can correct, prrhaps
using settings?)
!-Original Message-
!From: Tutor [mailto:tutor-bounces+crk=godblessthe...@python.org] On
!Behalf Of Steven D'Aprano
freshfruit = [' banana', ' loganberry ', 'passion fruit ']
[weapon.strip() for weapon in freshfruit]
['banana', 'loganberry', 'passion fruit']
From 5.1.3 in the Python3 tutorial. Although not an = assignment, some value
is being stored in weapon after being strip of whitespace. I realize they
are different, maybe just not sure how different:) In this case, somehow
Python seems to keep track of the location so that the modified
value(dropping spaces) replaces the original location.
!
!
!Try this instead:
!
!py descriptions = []
!py descriptions.append(something blue) descriptions.append(something
!py round) descriptions.append(something flat)
!py print(descriptions)
!['something blue', 'something round', 'something flat']
!
!
!Like the name suggests, append appends something to the end of the
!list. So your code, which started like this:
!
!# doesn't work
!col_position, code, description = 0, [], [] key_name =
!raw_table.replace('\t','\n') for each_line in key_name.splitlines():
!if ':' in each_line:
!code[col_position], description.append() = each_line.split(':')
!
!
!could be written something like this:
!
!# note plural names
!codes, descriptions = [], []
!key_name = raw_table.replace('\t','\n')
!for each_line in key_name.splitlines():
!if ':' in each_line:
!code, description = each_line.split(':')
!codes.append(code)
!descriptions.append(description)
So two questions remain. Why can't codes.append(code) just replace the code
in the previous line and descriptions.append(description) replace the
description in the previous line. Previous emails have seen something like
value.strip() = some value .
That last line won't work either.
If you did try to do it on one line, you'd need to put the
argument to append inside the parens, not on the other side of an
equal sign. Something like (untested )
codes.append (each_line.split()[0])
descriptions.append (each_line.split()[1]
Second question, why can't a numeric index be
used to make assignment to a specific location like a[1] = some value? If
the mechanism is to use a.index(1,some value),
The index() method does not change the object a, at least not for
list objects. So that is not a replacement for subscripted
assignment.
The form a[1] = some_value works fine, as long as there is
already an object in that list element. In other words, it can be
used to replace an item, but not to change the size. Python does
not support sparse lists, so rather than permitting a size
increase of exactly one, it was decided that the syntax would not
permit any size change. And to grow by 1, append works very
well.
If you know ahead of time what size you need, you could prefill it
with something like:
a = [] * 20
But you might want to check later that you replaced them all.
it seems somewhat clumsy.
It is clear that once the array
We have been talking about lists here, not arrays. The type
array.array behaves similarly, but not the same.
is created, it is possible to access the
value by numeric indexing such as:
print( col_position, code[col_position], description[col_position])
which produces:
83 r7 Price / EPS Estimate Next Year
84 s7 Short Ratio
Also,
# create a list of 2-tuples like (number, square)
[(x, x**2) for x in range(6)]
[(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)]
Why does this not need the .append or .insert? Square brackets around the
whole line?
Those square brackets, with the for keyword inside, comprise a
more advanced technique called a list comprehension. It's
syntactic sugar for a for loop with an append. But the list
object being built is anonymous, so it can frequently be used
inside other expressions.
I recommend ignoring list comprehensions till you can reliably
write the loop.
You seem determined to cram things onto a single line that would
probably be more readable as separate ones. And intermediate
variables give an opportunity to make up useful names, to make
the code easier to read.
--
DaveA
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Question on a select statement with ODBC
Al Bull wrote: Quick question then... Does this do the trick? I may have misunderstood your original question; do you want to delete records from the database or from the Python list? Your code below will do the latter, once you have fixed the bugs. Currentrecord = 1 While currentrecord = len(ord_rows): if savedbasub == currentrecord.ord_dbasub: ord_rows.remove(currentrecord) delcount += 1 else: savedbasub = currentrecord.ord_dbasub currentrecord =+ 1 As the general idea is correct I'll encourage you to try to fix these bugs yourself. The first thing you have to resolve: is currentrecord an index or an object representing a database row? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] yet another misunderstanding on my part
On 22/10/14 19:48, Clayton Kirkwood wrote: Regarding the index out of range, I know what it meant, I was just kind of surprised that Python didn't automatically create the node and stuff a value in. But what should Python do in this case aNewList[100] = 42 Should Python create a 1 million element list with only one value? That would be a slow operation. What should the other values be set to? None, presumably? !description.append() = something ! ! !from. There's nothing in Python that I've ever seen that suggests that !would work, and the error message should be clear: freshfruit = [' banana', ' loganberry ', 'passion fruit '] [weapon.strip() for weapon in freshfruit] ['banana', 'loganberry', 'passion fruit'] From 5.1.3 in the Python3 tutorial. Although not an = assignment, some value is being stored in weapon after being strip of whitespace. I realize they are different, maybe just not sure how different:) This is a list comprehension with a very specific syntax. It is equivalent to aList = [] for weapon in freshfruit: aList.append(weapon.strip()) It is very different from assigning a value to a function call. In this case, somehow Python seems to keep track of the location so that the modified value(dropping spaces) replaces the original location. No it does not replace the original location it is appended to the list. So two questions remain. Why can't codes.append(code) just replace the code in the previous line and descriptions.append(description) replace the description in the previous line. Because the append() adds it at the end it doesn't replace anything. value.strip() = some value . Second question, why can't a numeric index be used to make assignment to a specific location like a[1] = some value? It can if that entry already exists. listA = [0,1,2,3,4,5] #initialize with elements listA[3] = 9 print(listA) # - [0,1,2,9,4,5] listB = [] listB[3] = 9 # error because listB has no elements yet. listB += [0,1,2,3,4] # add some elements listB[3] = 9 # OK now. It is clear that once the array is created, Its not an array, its a list. Most specifically it is not a C style block of memory that is just filled in with a set of bytes. It is a dynamic sequence of objects of potentially mixed(and changing) type. # create a list of 2-tuples like (number, square) [(x, x**2) for x in range(6)] [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)] Why does this not need the .append or .insert? Square brackets around the whole line? Because the list comprehension implicitly does an append for you (see above) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] yet another misunderstanding on my part
Dave Angel da...@davea.name Wrote in message: Clayton Kirkwood c...@godblessthe.us Wrote in message: Second question, why can't a numeric index be used to make assignment to a specific location like a[1] = some value? If the mechanism is to use a.index(1,some value), The index() method does not change the object a, at least not for list objects. So that is not a replacement for subscripted assignment. In particular, the index () method searches the list. The form a[1] = some_value works fine, as long as there is already an object in that list element. In other words, it can be used to replace an item, but not to change the size. Python does not support sparse lists, so rather than permitting a size increase of exactly one, it was decided that the syntax would not permit any size change. And to grow by 1, append works very well. If you know ahead of time what size you need, you could prefill it with something like: a = [] * 20 Oops. I meant: A = [None] * 20 But you might want to check later that you replaced them all. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Is there a convenient table of Python 3.4 exceptions?
I am reading a brief intro to exception handling in Mark Summerfield's Programming in Python 3, 2nd ed. He gives the basic syntax as: try: try_suite except exception1 as variable1: exception_suite1 ... except exceptionN as variableN: exception_suiteN My current understanding is that exception1, ... , exceptionN should match one of Python's exception classes or subclasses in order to perform a match and execute the corresponding exception suite. I found this in the docs: Exceptions are identified by class instances. The except clause is selected depending on the class of the instance: it must reference the class of the instance or a base class thereof. The instance can be received by the handler and can carry additional information about the exceptional condition. Note Exception messages are not part of the Python API. Their contents may change from one version of Python to the next without warning and should not be relied on by code which will run under multiple versions of the interpreter. I have so far been unable to find a list of these class/subclass names. Of course I can force an error to occur in the interpreter and see what comes up for each type of error I wish to catch. Is there such a table or list? -- boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Is there a convenient table of Python 3.4 exceptions?
boB Stepp robertvst...@gmail.com writes: My current understanding is that exception1, ... , exceptionN should match one of Python's exception classes or subclasses in order to perform a match and execute the corresponding exception suite. Correct. Bear in mind that “… or subclasses” covers user-defined exception classes, so you'll often see non-standard exception classes in use by particular code bases, that are nevertheless subclasses of the exception classes in the standard library. I have so far been unable to find a list of these class/subclass names. The standard library documentation's chapter on exceptions URL:https://docs.python.org/3/library/exceptions.html shows URL:https://docs.python.org/3/library/exceptions.html#exception-hierarchy. But you won't find an exhaustive list of what can raise those exceptions. Of course I can force an error to occur in the interpreter and see what comes up for each type of error I wish to catch. Yes, that's the right thing to do; that, and read the documentation for whatever library code you are invoking. It should say if there are important situations where a particular exception will be raised. There will, of course, be a great many other situations that can raise an exception. This is a normal part of the flow of code in Python, and the standard library does a lot of it. More importantly, though, you should consider *why* you're attempting to catch a lot of exception classes. Will you be meaningfully handling every one of those situations? That's rather doubtful. Instead, you should consider which exceptional states your code can meaningfully handle, discover what excpetion classes are raised for those few situations, and catch *only* those classes. Any other exception that gets raised isn't something you can do anything useful with, so it should propagate back up to higher levels, either to be handled or to exit with a useful error message. -- \ “Science is a way of trying not to fool yourself. The first | `\ principle is that you must not fool yourself, and you are the | _o__) easiest person to fool.” —Richard P. Feynman, 1964 | Ben Finney ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Is there a convenient table of Python 3.4 exceptions?
I have so far been unable to find a list of these class/subclass names. Of course I can force an error to occur in the interpreter and see what comes up for each type of error I wish to catch. Is there such a table or list? Hi Bob, You can find the ones used in the Standard Library here: https://docs.python.org/3.4/library/exceptions.html Usually the documentation should say what kinds of exceptions to expect. If you find an exception to this, please bring it up, and one of us can investigate: maybe the documentation can be improved. If you have questions, please feel free to ask! Good luck. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Is there a convenient table of Python 3.4 exceptions?
(Danny, please preserve attribution lines for the quoted text.) Danny Yoo d...@hashcollision.org writes: Of course I can force an error to occur in the interpreter and see what comes up for each type of error I wish to catch. Is there such a table or list? […] Usually the documentation should say what kinds of exceptions to expect. If you find an exception to this, please bring it up, and one of us can investigate: maybe the documentation can be improved. That's rather too sweeping a statement, and it's definitely not true for Python's standard library documentation:: import signal signal.getsignal(None) Traceback (most recent call last): File stdin, line 1, in module TypeError: an integer is required (got type NoneType) signal.getsignal(50) Traceback (most recent call last): File stdin, line 1, in module ValueError: signal number out of range There's nothing in the documentation of ‘signal.getsignal’ which says it will raise either of those exceptions. I'd say it's a safe bet there are numerous other exception classes that will be raised calling that function, too. That's just one, chosen quickly and arbitrarily to prove the point. The same goes for the vast majority of functions in the standard library; most possible exception classes go unmentioned in the documentation for that function1, because exceptions are a normal part of operating on Python values (think of the wealth of possible exception classes that can come from passing in a file-like object to many functions). Indeed, most functions can't possibly have exhaustive documentation for what exception classes might be raised, because most of those possible exceptions are to be raised by the code implementing the *parameter* values, not raised by the code implementing that function. Should every function in every module of the standard library document every exception class that could be raised by that function? I'd say clearly not; the result would be uselessly cluttered. Where should the line be drawn? I think, in the case of the standard library, the line is already drawn at the right place. The documentation should describe situations that *someone already familiar with Python* – and with the types of objects they're passing to a function – would not know exactly what exception class might be raised. But for most functions, most of the exception classes they might raise are not documented, because when they are raised it's obvious. Is this a hurdle for newcomers? Yes, and that's why the standard library API documentation is not a tutorial. We have separate tutorial documentation for that. It's not a problem with the documentation of ‘signal.getsignal’ that it doesn't have an exhaustive list of all exception classes that might be raised. This is Python, not Java; exceptions are used prolifically as a flow control method, for code to let its call stack know something unusual occurred. -- \ “I went camping and borrowed a circus tent by mistake. I didn't | `\ notice until I got it set up. People complained because they | _o__) couldn't see the lake.” —Steven Wright | Ben Finney ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
