Re: psss...I want to move from Perl to Python

[James Harris]

On 29/01/2016 09:12, Ulli Horlacher wrote:

Steven D'Aprano  wrote:


Every time I make a half-hearted attempt to learn enough Perl syntax to get
started, I keep running into the differences between $foo, %foo and @foo
and dire warnings about what happens if you use the wrong sigil


I have started learning Python several times and surrendered because my
brain was too Perl hardcoded after 30 years, but NOW I was successful :-)


I nearly gave up with Python at the very beginning before I realised not 
to mix tabs and spaces.



(I still find Perl syntax better...)


Perl may be easier to 'think in' for those who are familiar with it. 
Perl syntax is shorter. And expressions have fewer elements as some 
operands are implied.


Python's constructions are more explicit, making the effect more 
apparent in the source.



About the variables in short:

$foo is a scalar (number, string, reference, file handle)
@foo is an array
%foo is a hash (dictionary in Python slang)

and yes, you can use them all together in same code, they are different.


IIRC it's not quite that simple in that the type depends on the context 
that Perl thinks is current. I won't go into detail as this is not a 
Perl group but I found Perl horrible to work with. It is slick but also 
cryptic.


James

--
https://mail.python.org/mailman/listinfo/python-list

List comprehensions and evaluation of elements therein

[James Harris]

A list comprehension has various components. Anyone know when each of 
the elements is evaluated? In the form


 [v0 for v0 in expr0 if expr1]

If v0 appears in expr0 or expr1 the evaluation order matters.

I think of the above as being a rewrite of

 results = []
 for v0 in expr0:
   if expr1:
 results.append(v0)
 return results

Further,

 [v0, v2 for v0 in expr0 if expr1 for v2 in expr2 if expr3]

leads to

 results = []
 for v0 in expr0:
if expr1:
   for v2 in expr2:
  if expr3:
 results.append((v0, v2))
 return results

First of all, is that a correct analog of the list comprehension?

With this latter expansion the values of v0 and v2 could appear in expr4 
or expr5. Again, the evaluation order would matter.


James

--
https://mail.python.org/mailman/listinfo/python-list

Re: Lightwight socket IO wrapper

[James Harris]

"Dennis Lee Bieber" <wlfr...@ix.netcom.com> wrote in message 
news:mailman.12.1442794762.28679.python-l...@python.org...

On Sun, 20 Sep 2015 23:36:30 +0100, "James Harris"
<james.harri...@gmail.com> declaimed the following:




There are a few things and more crop up as time goes on. For example,
over TCP it would be helpful to have a function to receive a specific
number of bytes or one to read bytes until reaching a certain 
delimiter
such as newline or zero or space etc. Even better would be to be able 
to

use the iteration protocol so you could just code next() and get the
next such chunk of read in a for loop. When sending it would be good 
to

just say to send a bunch of bytes but know that you will get told how
many were sent (or didn't get sent) if it fails. Sock.sendall() 
doesn't

do that.


Note that the "buffer size" option on a TCP socket.recv() gives you
your "specific number of bytes" -- if available at that time.


"If" is a big word!

AIUI the buffer size is not guaranteed to relate to the number of bytes 
returned except that you won't/shouldn't(!) get more than the buffer 
size.



I wouldn't want to user .recv(1) though to implement your "reaching a
certain delimiter"... Much better to read as much as available and 
search

it for the delimiter.


Yes, that's what I do at the moment. I keep a block of bytes, add any 
new stuff to it and scan it for delimiters.



I'll confess, adding a .readln() FOR TCP ONLY, might
be a nice extension over BSD sockets (might need to allow option for
whether line-ends are Internet standard  or some other marker, 
and
whether they should be converted upon reading to the native format for 
the

host).


Akira Li pointed out that there is just such an extension: makefile. 
Scanning to  is what I do just now as that includes  too and 
I leave them on the string. IIRC file.readline works in the same way.



I thought UDP would deliver (or drop) a whole datagram but cannot find
anything in the Python documentaiton to guarantee that. In fact
documentation for the send() call says that apps are responsible for
checking that all data has been sent. They may mean that to apply to
stream protocols only but it doesn't state that. (Of course, UDP
datagrams are limited in size so the call may validly indicate
incomplete transmission even when the first part of a big message is
sent successfully.)


Looking in the wrong documentation 

You probably should be looking at the UDP RFC. Or maybe just

http://www.diffen.com/difference/TCP_vs_UDP

"""
Packets are sent individually and are checked for integrity only if 
they
arrive. Packets have definite boundaries which are honored upon 
receipt,

meaning a read operation at the receiver socket will yield an entire
message as it was originally sent.
"""


I would rather see it in the Python docs because we program to the 
language standard and there can be - and often are, for good reason - 
areas where Python does not work in the same way as underlying systems.


Even if the IP layer has to fragment a UDP packet to meet limits of 
the
transport media, it should put them back together on the other end 
before
passing it up to the UDP layer. To my knowledge, UDP does not have a 
size
limit on the message (well -- a 16-bit length field in the UDP 
header). But
since it /is/ "got it all" or "dropped" with no inherent confirmation, 
one
would have to embed their own protocol within it -- sequence numbers 
with
ACK/NAK, for example. Problem: if using LARGE UDP packets, this 
protocol
would mean having LARGE resends should packets be dropped or arrive 
out of

sequence (and since the ACK/NAK could be dropped too, you may have to
handle the case of a duplicated packet -- also large).


Yes, it was the 16-bit limitation that I was talking about.


TCP is a stream protocol -- the protocol will ensure that all data
arrives, and that it arrives in order, but does not enforce any 
boundaries

on the data; what started as a relatively large packet at one end may
arrive as lots of small packets due to intermediate transport limits 
(one
can visualize a worst case: each TCP packet is broken up to fit 
Hollerith
cards; 20bytes for header and 60 bytes of data -- then fed to a reader 
and
sent on AS-IS). Boundaries are the end-user responsibility... line 
endings
(look at SMTP, where an email message ends on a line containing just a 
".")

or embedded length counter (not the TCP packet length).


Yes.

Receiving no bytes is taken as indicating the end of the 
communication.

That's OK for TCP but not for UDP so there should be a way to
distinguish between the end of data and receiving an empty datagram.


I don't believe UDP supports a truly empty datagram (length of 0) --
presuming a sending stack actually sends one, the receiving stack will
probably drop it as there is no data to pass on to a client (there

Re: Lightwight socket IO wrapper

[James Harris]

"Marko Rauhamaa" <ma...@pacujo.net> wrote in message 
news:8737y6cgp6@elektro.pacujo.net...

"James Harris" <james.harri...@gmail.com>:


I agree with what you say. A zero-length UDP datagram should be
possible and not indicate end of input but is that guaranteed and
portable?


The zero-length payload size shouldn't be an issue, but UDP doesn't 
make

any guarantees about delivering the message. Your UDP application must
be prepared for some, most or all of the messages disappearing without
any error indication.

In practice, you'd end up implementing your own TCP on top of UDP
(retries, timeouts, acknowledgements, sequence numbers etc).


The unreliability of UDP was not the case in point here. Rather, it was 
about whether different platforms could be relied upon to deliver 
zero-length datagrams to the app if the datagrams got safely across the 
network.


James

--
https://mail.python.org/mailman/listinfo/python-list

Re: Lightwight socket IO wrapper

[James Harris]

"Akira Li" <4kir4...@gmail.com> wrote in message 
news:mailman.18.1442804862.28679.python-l...@python.org...

"James Harris" <james.harri...@gmail.com> writes:
...

There are a few things and more crop up as time goes on. For example,
over TCP it would be helpful to have a function to receive a specific
number of bytes or one to read bytes until reaching a certain
delimiter such as newline or zero or space etc.


The answer is sock.makefile('rb') then `file.read(nbytes)` returns a
specific number of bytes.


Thanks, I hadn't seen that. Now I know of it I see references to it all 
over the place but beforehand it was in hiding


It is exactly the type of convenience wrapper I was expecting Python to 
have but expected it to be in another module. It looks as though it will 
definitely cover some of the issues I had.



`file.readline()` reads until newline (b'\n') There is Python Issue:
"Add support for reading records with arbitrary separators to the
standard IO stack"
 http://bugs.python.org/issue1152248
See also
 http://bugs.python.org/issue17083

Perhaps, it is easier to implement read_until(sep) that is best suited
for a particular case.


OK.

...

When sending it would be good to just say to send a bunch of bytes 
but
know that you will get told how many were sent (or didn't get sent) 
if

it fails. Sock.sendall() doesn't do that.


sock.send() returns the number of bytes sent that may be less than 
given.

You could reimplement sock.sendall() to include the number of bytes
successfully sent in case of an error.


I know. As mentioned, I wondered if there were already such functions to 
save me using my own.


I thought UDP would deliver (or drop) a whole datagram but cannot 
find

anything in the Python documentaiton to guarantee that. In fact
documentation for the send() call says that apps are responsible for
checking that all data has been sent. They may mean that to apply to
stream protocols only but it doesn't state that. (Of course, UDP
datagrams are limited in size so the call may validly indicate
incomplete transmission even when the first part of a big message is
sent successfully.)

Receiving no bytes is taken as indicating the end of the
communication. That's OK for TCP but not for UDP so there should be a
way to distinguish between the end of data and receiving an empty
datagram.


There is no end of communication in UDP and therefore there is no end 
of

data. If you've got a zero bytes in return then it means that you've
received a zero length datagram.

sock.recvfrom() is a thin wrapper around the corresponding C
function. You could read any docs you like about UDP sockets.

http://stackoverflow.com/questions/5307031/how-to-detect-receipt-of-a-0-length-udp-datagram


As mentioned to Dennis just now, I would prefer to write code to conform 
with the documented behaviour of Python and its libraries, as long as 
they were known to be reliable implementations of what was documented, 
of course.


I agree with what you say. A zero-length UDP datagram should be possible 
and not indicate end of input but is that guaranteed and portable? 
(Rhetorical.)  It seems not. Even the Linux man page for recv says: "If 
no  messages  are  available  at  the  socket, the receive calls wait 
for a message to arrive, unless the socket is nonblocking" In that 
case, of course, what it defines as a "message" - and whether it can be 
zero length or not - is not stated.



The recv calls require a buffer size to be supplied which is a
technical detail. A Python wrapper could save the programmer dealing
with that.


It is not just a buffer size. It is the maximum amount of data to be
received at once i.e., sock.recv() may return less but never more.


My point was that we might want to request the entire next line or next 
field of input and not know a maximum length. *C* programmers are used 
to giving buffers fixed sizes often because then they can avoid fiddling 
with memory management but Python normally does that for us. I was 
suggesting that the thin wrapper around the socket recv() call is too 
thin! The makefile() approach that you mentioned seems more Pythonesque, 
though.



You could use makefile() and read() if recv() is too low-level.


Yes.

James

--
https://mail.python.org/mailman/listinfo/python-list

Re: A little test for you Guys

[James Harris]

On Tuesday, September 22, 2015 at 7:45:00 PM UTC+1, Lj Fc wrote:
> you have 10 minutes Good luck!!

A good set of questions, IMO. Am answering as someone coming back to Python 
after a few years.

> 1. What is PEP8 ?

Coding guidelines, I think.

> 2. What are the different ways to distribute some python source code ?

I don't know what that's getting at as it specifically mentions source code 
apart from tar/gzip or zip. Maybe git or other scm?

> 2 Lists
> 
> Let's define the function plural :
> 
> def plural(words):
> plurals = []
> for word in words:
>plurals.append(word + 's')
> return plurals
> 
> for word in plural(['cabagge','owl','toy']):
> print word
> 
> Question : How could the code of the function plural be optimised?

I would go for

  [word + 's' for word in words]

> 3 Dictionaries
> 
> Here are two dictionnaries :
> 
> input = {
> 'foo1': 'bar1',
> 'chose': 'truc',
> 'foo2': 'bar2',
> }
> output = {
> 'bar1': 'foo1',
> 'truc': 'chose',
> 'bar2': 'foo2'
> }
> 
> Question : Propose a function that returns output when you provide input ?

  def f(input):
output = {}
for k,v in input.items():
  output[v] = k
return output

> 4 Iterators
> 
> Let's consider this program :
> 
> def program_1():
> yield 1
> yield 2
> yield 3
> 
> g = program_1()
> a = list(g)
> b = list(g)
> c = g()
> 
> Question : At the end of the program,
> 
> 1. What is the type of g ?
> 2. What is the value of a ?
> 3. What is the value of b ?
> 4. What is the value of c ?

Good one. I checked this and only got 1 and 2 right.

> 5 Decorators

No idea!

James
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A little test for you Guys

[James Harris]

 wrote in message 
news:5218c7f9-74ea-4ca0-abd1-46a9bcd3d...@googlegroups.com...


...


Pretty sure this guy is asking us to do his homework.  :-P


Maybe (and I hope not) but asking what PEP8 is could be easily found on 
the internet and asking what the values would be at the end of the 
program in question 4 could be easily found by trying it.


James

--
https://mail.python.org/mailman/listinfo/python-list

Lightwight socket IO wrapper

[James Harris]

I guess there have been many attempts to make socket IO easier to handle 
and a good number of those have been in Python.


The trouble with trying to improve something which is already well 
designed (and conciously left as is) is that the so-called improvement 
can become much more complex and overly elaborate. That can apply to the 
initial idea, for sure, but when writing helper or convenience functions 
perhaps it applies more to the temptation to keep adding just a little 
bit extra. The end result can be overly elaborate such as a framework 
which is fine where such is needed but is overkill for simpler 
requirements.


Do you guys have any recommendations of some *lightweight* additions to 
Python socket IO before I write any more of my own? Something built in 
to Python would be much preferred over any modules which have to be 
added. I had in the back of my mind that there was a high-level 
socket-IO library - much as threading was added as a wrapper to the 
basic thread module - but I cannot find anything above socket. Is there 
any?


A current specific to illustrate where basic socket IO is limited: it 
normally provides no guarantees over how many bytes are transferred at a 
time (AFAICS that's true for both streams and datagrams) so the 
delimiting of messages/records needs to be handled by the sender and 
receiver. I do already handle some of this myself but I wondered if 
there was a prebuilt solution that I should be using instead - to save 
me adding just a little bit extra. ;-)


James

--
https://mail.python.org/mailman/listinfo/python-list

Re: Lightwight socket IO wrapper

[James Harris]

"Akira Li" <4kir4...@gmail.com> wrote in message 
news:mailman.37.1442754893.21674.python-l...@python.org...

"James Harris" <james.harri...@gmail.com> writes:


I guess there have been many attempts to make socket IO easier to
handle and a good number of those have been in Python.

The trouble with trying to improve something which is already well
designed (and conciously left as is) is that the so-called 
improvement

can become much more complex and overly elaborate. That can apply to
the initial idea, for sure, but when writing helper or convenience
functions perhaps it applies more to the temptation to keep adding
just a little bit extra. The end result can be overly elaborate such
as a framework which is fine where such is needed but is overkill for
simpler requirements.

Do you guys have any recommendations of some *lightweight* additions
to Python socket IO before I write any more of my own? Something 
built
in to Python would be much preferred over any modules which have to 
be

added. I had in the back of my mind that there was a high-level
socket-IO library - much as threading was added as a wrapper to the
basic thread module - but I cannot find anything above socket. Is
there any?


Does ØMQ qualify as lightweight?


It's certainly interesting. It's puzzling, too. For example,

 http://zguide.zeromq.org/py:hwserver

The Python code there includes

 message = socket.recv()

but given that this is a TCP socket it doesn't look like there is any 
way for the stack to know how many bytes to return. Either ZeroMQ layers 
another end-to-end protocol on top of TCP (which would be no good) or it 
will be guessing (which would not be good either).


There are probably answers to that query but there is a lot of 
documentation, including on reliable communication, and that in itself 
makes ZeroMQ seem overkill, even if it can be persuaded to do what I 
want.


I am impressed that they show code in many languages. I may come back to 
it but for the moment it doesn't seem to be what I was looking for. And 
it is not built in.



A current specific to illustrate where basic socket IO is limited: it
normally provides no guarantees over how many bytes are transferred 
at

a time (AFAICS that's true for both streams and datagrams) so the
delimiting of messages/records needs to be handled by the sender and
receiver. I do already handle some of this myself but I wondered if
there was a prebuilt solution that I should be using instead - to 
save

me adding just a little bit extra. ;-)


There are already convenience functions in stdlib such as
sock.sendall(), sock.sendfile(), socket.create_connection() in 
addition

to BSD Sockets API.

If you want to extend this list and have specific suggestions; see
 https://docs.python.org/devguide/stdlibchanges.html


That may be a bit overkill just now but it's a good suggestion.


Or just describe your current specific issue in more detail here.


There are a few things and more crop up as time goes on. For example, 
over TCP it would be helpful to have a function to receive a specific 
number of bytes or one to read bytes until reaching a certain delimiter 
such as newline or zero or space etc. Even better would be to be able to 
use the iteration protocol so you could just code next() and get the 
next such chunk of read in a for loop. When sending it would be good to 
just say to send a bunch of bytes but know that you will get told how 
many were sent (or didn't get sent) if it fails. Sock.sendall() doesn't 
do that.


I thought UDP would deliver (or drop) a whole datagram but cannot find 
anything in the Python documentaiton to guarantee that. In fact 
documentation for the send() call says that apps are responsible for 
checking that all data has been sent. They may mean that to apply to 
stream protocols only but it doesn't state that. (Of course, UDP 
datagrams are limited in size so the call may validly indicate 
incomplete transmission even when the first part of a big message is 
sent successfully.)


Receiving no bytes is taken as indicating the end of the communication. 
That's OK for TCP but not for UDP so there should be a way to 
distinguish between the end of data and receiving an empty datagram.


The recv calls require a buffer size to be supplied which is a technical 
detail. A Python wrapper could save the programmer dealing with that.


Reminder to self: encoding issues.

None of the above is difficult to write and I have written the bits I 
need myself but, basically, there are things that would make socket IO 
easier and yet still compatible with more long-winded code. So I 
wondered if there were already some Python modules which were more 
convenient than what I found in the documentation.


James

--
https://mail.python.org/mailman/listinfo/python-list

Re: Shutting down a cross-platform multithreaded app

[James Harris]

"Laura Creighton" <l...@openend.se> wrote in message 
news:mailman.5.1442609448.21674.python-l...@python.org...
In a message of Fri, 18 Sep 2015 20:09:19 +0100, "James Harris" 
writes:

Set the daemon flag on the worker threads, so when the main thread
exits, the workers also exit.


Interesting idea, and I did not know that a *thread* could be a 
daemon.

Unfortunately, I think what you suggest would kill the threads stone
dead and not allow them to close connections.


Can you stick your worker threads into a Queue.  When the main thread 
exits

have it tell the queue to clean itself up?

see:
http://code.activestate.com/recipes/82965-threads-tkinter-and-asynchronous-io/

The main thread doesn't have to be a gui ...

(but the author of that recipe and I are now drunkly celebrating a 
birthday

so maybe I ought not to be posting this idea ...)


:-)

I am not sure. The polling every 100ms or similar in periodicCall() is 
something I want to avoid. I think I have a way to do this without any 
polling.


James

--
https://mail.python.org/mailman/listinfo/python-list

Re: Shutting down a cross-platform multithreaded app

[James Harris]

"Chris Angelico" <ros...@gmail.com> wrote in message 
news:mailman.8.1442612439.21674.python-l...@python.org...
On Sat, Sep 19, 2015 at 3:17 AM, James Harris 
<james.harri...@gmail.com> wrote:
Needless to say, on a test Windows machine AF_UNIX is not present. 
The only

cross-platform option, therefore, seems to be to use each subthread's
select()s to monitor two AF_INET sockets: the one to the client and a
control one from the master thread. I would seem to need IP socket 
pairs
between the master thread and the subthreads. If the master thead 
receives a

shutdown signal it will send a shutdown command to each subthread.

The above seems logical but would use quite a few IP sockets. I 
cannot think

of a better way, though. Any comments on the ideas above?


If you're using select() to monitor the sockets, you don't actually
then have to _do_ anything with the shutdown socket. You could have a
single socket that sends the shutdown signal to all your workers.


I don't understand how a single socket could send the signal to all the 
workers. I did consider some form of multicast but thought it too 
complicated (and possibly infeasible).


Re. not understanding the single sending socket idea that you mention 
perhaps I had better explain a bit of what I had in mind:


1. A worker thread would have a TCP socket connection to its client, and 
another socket for communicating with the master. That second socket has 
to be AF_INET for portability. It could be TCP or UDP. A connected UDP 
socket may be most appropriate and seems worth trying.


2. If something happens to the master thread so that it determines that 
the application should shut down it would iterate over the sockets to 
the workers and tell each one to shut down. It would then shut itself 
down. (Am not sure at the moment whether to wait for the worker 
threads.)


3. A worker thread, being told to shutdown (basically a single byte 
received from the master thread) would tell its client to close the TCP 
connection and then it would wait a little while for it to do so - maybe 
a second or two. When the client closes the TCP connection (or the 
timeout wait period expires) the worker thread will close its end and 
exit.



Bear in mind, though, that Windows has no protection against other
processes shutting you down. You can restrict it to 127.0.0.1 (of
course) but any program running on the same computer as the server -
regardless of user permissions etc - will be able to connect to your
sockets.


I was thinking of a connected UDP socket. That way, AIUI, at least in 
the absence of forged datagrams, only the master thread will be able to 
communicate with the worker, due to connected UDP sockets demulitplexing 
datagrams based on their source as well as their destination, i.e. on a 
5-tuple (UDP, source IP, source port, destination IP, destination port).



So it might be best to do something like this (all on the
main thread):

1) Open a listening socket
2) Connect to the listening socket
3) Accept a connection
4) Close the original listening socket
5) Spin off all your threads, passing them the socket from step 2
6) To terminate them all, write a byte to the socket from step 3.


That sounds similar to what I had in mind but I am not sure why you 
would close the listening socket. Connections could come in at any time 
and threads could therefore be needed at any time so I was thinking that 
the master thread (the one with the listening TCP socket) would just sit 
waiting for new connection requests (or an interrupting signal).


In reality, due to Windows not recognising signals while in the accept() 
call I think there would be a real master thread and a listening thread 
but I have omitted that in the descriptions above. As far as the normal 
worker threads are concerned they would be ready to be told to shut down 
by the listening thread, and it would be ready to be told to shut down 
by the master thread. Still with me? ;-)



This will make it difficult for ordinary userspace code to mess with
you. It'd still be possible, I think, for something with raw sockets
access to feign a termination signal; I have no idea what protections
Windows offers you there.


Yes, something which could forge a packet could tell a worker to close 
down. I don't think there's any significant problem here, thought, 
because:


* other programs are similarly vulnerable to forged packets
* the only forgery effect is to tell a worker thread to stop - not a big 
loss

* the shutdown protocol would/should cause the client to re-request
* a forger would have to know the specific port number used by the 
master thread to communicate with that particular worker, and the port 
number that worker was using.


Overall, I think it would be more than robust enough.

Notwithstanding your comment about a single socket, above, no one seems 
so far to have objected to the number of sockets this would need, which 
was my main concern, so that's g

Re: Shutting down a cross-platform multithreaded app

[James Harris]

"Chris Angelico" <ros...@gmail.com> wrote in message 
news:mailman.13.1442657702.21674.python-l...@python.org...
On Sat, Sep 19, 2015 at 7:49 PM, James Harris 
<james.harri...@gmail.com> wrote:

"Chris Angelico" <ros...@gmail.com> wrote in message
news:mailman.8.1442612439.21674.python-l...@python.org...


...


If you're using select() to monitor the sockets, you don't actually
then have to _do_ anything with the shutdown socket. You could have 
a

single socket that sends the shutdown signal to all your workers.



I don't understand how a single socket could send the signal to all 
the

workers. I did consider some form of multicast but thought it too
complicated (and possibly infeasible).


The way I'm describing it, the workers never actually read from the
socket. Once that socket becomes readable, they immediately shut down,
without making the socket no-longer-readable.


Understood. Good idea. Initial thoughts on it: Would work for threads. 
Would save on the number of sockets required. Would not work for 
processes if the model was ever changed. Would make it easier for rogue 
packets to shut a worker down. Would not allow any way to distinguish 
between shutdown priorities (not something I have mentioned). Definitely 
feasible. I'll keep it in mind.



Bear in mind, though, that Windows has no protection against other
processes shutting you down. You can restrict it to 127.0.0.1 (of
course) but any program running on the same computer as the server -
regardless of user permissions etc - will be able to connect to your
sockets.


...

That sounds similar to what I had in mind but I am not sure why you 
would

close the listening socket. Connections could come in at any time and
threads could therefore be needed at any time so I was thinking that 
the
master thread (the one with the listening TCP socket) would just sit 
waiting

for new connection requests (or an interrupting signal).


TCP sockets work on the basis of a master socket and any number of
spawned sockets. The master is what gives you an open port; each
spawned socket represents one connection with one client. Once you
have an established connection, the master should be able to be closed
without disrupting that. No other process will be able to connect to
you, but you'll still be able to use one end of the socket to make the
other end readable.


Agreed but I need the listening socket to remain open and listening for 
new connections (at least until the whole program is told to shut down).



This will make it difficult for ordinary userspace code to mess with
you. It'd still be possible, I think, for something with raw sockets
access to feign a termination signal; I have no idea what 
protections

Windows offers you there.



Yes, something which could forge a packet could tell a worker to 
close down.

I don't think there's any significant problem here, thought, because:

* other programs are similarly vulnerable to forged packets
* the only forgery effect is to tell a worker thread to stop - not a 
big

loss
* the shutdown protocol would/should cause the client to re-request
* a forger would have to know the specific port number used by the 
master
thread to communicate with that particular worker, and the port 
number that

worker was using.

Overall, I think it would be more than robust enough.


With UDP, any process that can send a UDP packet can flood the system
with them until your workers shut down. You wouldn't even notice until
it succeeds.


Is that true? You seem to be describing a non-forged attack but to get 
the source UDP port right wouldn't the attacker have to be runing on the 
same machine *and* to bind to the same port that the machine had 
allocated to my program? I might be wrong but I don't think the UDP 
stack would allow the same port to be bound again before the original 
had been closed.



With TCP, at least an attacker would need raw socket
access. It's still not as protected as a Unix domain socket, but it's
a bit harder for someone to do.

Notwithstanding your comment about a single socket, above, no one 
seems so
far to have objected to the number of sockets this would need, which 
was my

main concern, so that's good!


Sure. Sockets are pretty cheap. Even if you had one for every worker,
there's room for you to have thousands (maybe tens of thousands) of
workers without a problem. I think you'll run into other scaling
problems with that many workers on one computer :)


Let's see. If I stick with my original plan then each worker would have 
a TCP socket and a UDP socket. The "listener" thread would have its 
single listening TCP socket plus it would have a UDP socket for each 
worker thread. Total of three sockets per worker, two of which would be 
UDP sockets with port numbers assigned and thus consumed.


If I go with a single "shutdown socket" then I would have just one 
socket per worker. That would use fewer sockets, for sure.


James

--
https://mail.python.org/mailman/listinfo/python-list

Re: Shutting down a cross-platform multithreaded app

[James Harris]

"Paul Rubin" <no.email@nospam.invalid> wrote in message 
news:87zj0jd1ta@jester.gateway.sonic.net...

"James Harris" <james.harri...@gmail.com> writes:



I have a multithreaded app that I want to be able to shut down easily
such as by hitting control-c or sending it a signal.


Set the daemon flag on the worker threads, so when the main thread
exits, the workers also exit.


Interesting idea, and I did not know that a *thread* could be a daemon. 
Unfortunately, I think what you suggest would kill the threads stone 
dead and not allow them to close connections.


That's a particular problem with TCP connections and would require the 
OS to keep TCP state around for a while. I would rather close the TCP 
connections or, rather, encourage the other end to close the connection 
so that the worker threads could then close the sockets in a way that 
would not hold on to resources.


For anyone who is interested see the TCP state diagram such as the one 
at


 http://no-shoveling.com/wp-content/uploads/2013/11/TCPfsm.png

The key transition is the way the server exits the ESTABLISHED state. If 
the server closes its end of the connection first the transition goes 
via the line labelled appl:close, send: FIN. In that case the socket 
will end up in the TIME_WAIT state wherein it can wait 2MSL or 2 maximum 
segment lifetimes before becoming free.


According to

 https://en.wikipedia.org/wiki/Maximum_segment_lifetime

MSL is arbitrarily defined to be two minutes. That means a TCP endpoint 
could sit in TIME_WAIT for a horribly long four minutes...!


So, I would rather get the other end to send the first FIN, if possible. 
On the TCP state diagram that is the exit from ESTABLISHED labelled 
recv:FIN, send ACK. My end can then shutdown the socket, which would 
send a FIN, and wait for a final ACK.


Bottom line: I need to do a controlled cleanup.

James

--
https://mail.python.org/mailman/listinfo/python-list

Shutting down a cross-platform multithreaded app

[James Harris]

Well, this is fun ... for some definition of the word. ;-(

I have a multithreaded app that I want to be able to shut down easily 
such as by hitting control-c or sending it a signal. What follows is the 
way I have come up with given the common elements of different 
environments. Suggestions for improvement would be welcome or you may 
just find the convolutions and machinations interesting.


The first issue is that only the main thread can receive signals, 
according to


 https://docs.python.org/2/library/signal.html

It says: "the main thread will be the only one to receive signals (this 
is enforced by the Python signal module, even if the underlying thread 
implementation supports sending signals to individual threads)".


That's OK. I can get the main thread to accept suitable signals but then 
I need some way for it to tell the other threads to shut themselves down 
too. In most (probably all) cases they will be sitting waiting for 
network IO.


I could have the main thread set a value in a global variable and then 
have the sub-threads check the global in between accesses of the network 
in a polling loop (using select() with a timeout). But as has already 
been pointed out to me in the thread "Signal SIGINT ignored during 
socket.accept" such polling does not sit well with hosted OSes and it 
can keep the CPU from remaining at rest in a low-power mode.


I can, however, use select() to monitor two file descriptors. One would 
be the socket the thread is using to communicate with the client. The 
other would be a control connection from the master thread.


Now to make this cross platform According to the opening paragraph 
in the following link Windows select() won't work on arbitrary file 
descriptors but only works for sockets.


 https://docs.python.org/2/library/select.html

Well, that can be dealt with. I thought of using AF_UNIX or something 
else but it seems there is nothing else which could be considered 
universal and, according to the next link, if socket.AF_UNIX is not 
defined then even the Unix protocol is not supported.


 https://docs.python.org/2/library/socket.html

Needless to say, on a test Windows machine AF_UNIX is not present. The 
only cross-platform option, therefore, seems to be to use each 
subthread's select()s to monitor two AF_INET sockets: the one to the 
client and a control one from the master thread. I would seem to need IP 
socket pairs between the master thread and the subthreads. If the master 
thead receives a shutdown signal it will send a shutdown command to each 
subthread.


The above seems logical but would use quite a few IP sockets. I cannot 
think of a better way, though. Any comments on the ideas above?


James

--
https://mail.python.org/mailman/listinfo/python-list

Re: Signal SIGINT ignored during socket.accept

[James Harris]

"Grant Edwards"  wrote in message 
news:msum6c$hv$1...@reader1.panix.com...

On 2015-09-11, Chris Angelico  wrote:


This is what I meant when I said you would be polling. Effectively,
you wake up your program every half-second, check if Ctrl-C has been
pressed, and if it hasn't, you go back to sleep again. This is pretty
inefficient.


Though it offends one's engineering sensibilities[1], it's just not
that inefficient.


Indeed.


I'd bet money you won't even be able to measure the
difference in CPU usage.


You would win your bet.


Waking up twice per second and immediately
calling select() again on any hardware/OS built in the past 50 years
is going completely negligible (as long as you can ignore the smell).


CPU usage is not the only issue but it is a consideration. I tested it 
before posting the code and couldn't see any significant increase in CPU 
use. To give it a better test I have just left running for a couple of 
hours or so. I am pleased to say it currently reports a cumulative total 
of 0:00:00, i.e. it has not clocked up even a second of CPU time yet!


...


[1] If offenses to engineering sensibility were of concern, then
   he wouldn't be using Windows in the first place.  ;)


LOL. I know that's tongue in cheek but I tend to favour portability over 
most other things. So running on Windows as well as Unix is, in my book, 
a Good Thing.


James

--
https://mail.python.org/mailman/listinfo/python-list

Re: Signal SIGINT ignored during socket.accept

[James Harris]

"James Harris" <james.harri...@gmail.com> wrote in message 
news:msv21t$n1m$1...@dont-email.me...


"Grant Edwards" <invalid@invalid.invalid> wrote in message 
news:msum6c$hv$1...@reader1.panix.com...


...


Waking up twice per second and immediately
calling select() again on any hardware/OS built in the past 50 years
is going completely negligible (as long as you can ignore the smell).


CPU usage is not the only issue but it is a consideration. I tested it 
before posting the code and couldn't see any significant increase in 
CPU use. To give it a better test I have just left running for a 
couple of hours or so. I am pleased to say it currently reports a 
cumulative total of 0:00:00, i.e. it has not clocked up even a second 
of CPU time yet!


I am beginning to wonder if time is being accounted properly. It has now 
been running 8 hours and still shows CPU time as zero.


James

--
https://mail.python.org/mailman/listinfo/python-list

Signal SIGINT ignored during socket.accept

[James Harris]

I have a listening socket, self.lsock, which is used in an accept() call 
as follows


 endpoint = self.lsock.accept()

The problem is that if control-C is pressed it is not recognised until 
something connects to that socket. Only when the accept() call returns 
is the signal seen.


The question, then, is how to get the signal to break out of the 
accept() call. This is currently on Windows but I would like it to run 
on Unix too. I see from the web that this type of thing is a common 
problem with the underlying C libraries but I cannot quite relate the 
posts I have found to Python.


Any ideas?

James

--
https://mail.python.org/mailman/listinfo/python-list

Re: Signal SIGINT ignored during socket.accept

[James Harris]

"Chris Angelico" <ros...@gmail.com> wrote in message 
news:mailman.332.1441910212.8327.python-l...@python.org...
On Fri, Sep 11, 2015 at 4:24 AM, James Harris 
<james.harri...@gmail.com> wrote:


I have a listening socket, self.lsock, which is used in an accept() 
call as

follows

 endpoint = self.lsock.accept()

The problem is that if control-C is pressed it is not recognised 
until
something connects to that socket. Only when the accept() call 
returns is

the signal seen.

The question, then, is how to get the signal to break out of the 
accept()
call. This is currently on Windows but I would like it to run on Unix 
too. I

see from the web that this type of thing is a common problem with the
underlying C libraries but I cannot quite relate the posts I have 
found to

Python.


What version of Python are you using? Also (in case it matters), what
version of Windows?


Good point. It turns out that it does matter. I have one implementation 
which fails (Windows) and one which works (Linux). The Linux one breaks 
out on Control-C. The Windows one does not recognise Control-C until the 
accept() call returns.


The implementations are:

$ uname -srm
Linux 3.18.7-v7+ armv7l
$ python -V
Python 2.7.3

And

c:\>ver
Microsoft Windows XP [Version 5.1.2600]
c:\>python -V
Python 2.7.9


Have you tested on any Unix system? I just tried on my Linux, and
Ctrl-C interrupted the accept() straight away,


Thanks.


so this is quite probably a Windows-only issue.


That turns out to be exactly right.

Can you produce an absolute minimal demo program? I'd try something 
like this:


import socket
s = socket.socket()
s.listen(1)
s.accept()


Yes:

port = 8880
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("", port))
s.listen(1)
endpoint = s.accept()


which is what worked for me (interactively, Python 2.7.9 and 3.6.0a0,
Debian Linux).


On Linux I get

$ python socktest.py
^CTraceback (most recent call last):
 File "socktest.py", line 6, in 
   endpoint = s.accept()
 File "/usr/lib/python2.7/socket.py", line 202, in accept
   sock, addr = self._sock.accept()
KeyboardInterrupt
$

On Windows I get

S:\>python socktest.py
Traceback (most recent call last):
 File "socktest.py", line 6, in 
   endpoint = s.accept()
 File "C:\Python27\lib\socket.py", line 202, in accept
   sock, addr = self._sock.accept()
KeyboardInterrupt

S:\>

However, on Windows the recognition of Control-C does not happen until 
after something connects to the socket.


I will carry on researching it but maybe the above gives a clue to those 
in the know...!


James

--
https://mail.python.org/mailman/listinfo/python-list

Re: Signal SIGINT ignored during socket.accept

[James Harris]

"Chris Angelico" <ros...@gmail.com> wrote in message 
news:mailman.337.1441913195.8327.python-l...@python.org...
On Fri, Sep 11, 2015 at 5:11 AM, James Harris 
<james.harri...@gmail.com> wrote:


...

However, on Windows the recognition of Control-C does not happen 
until after

something connects to the socket.


...


This is a known problem on Windows.


...


It's entirely possible that a blocking socket-accept call will
continue to block. There is one rather silly option, and that's to use
select() to effectively poll for Ctrl-C... or, possibly better, have a
separate program that shuts down your server (by connecting to it,
which thus breaks the stranglehold).


Thanks for your help, Chris. Using select() is a very good option. I 
tried first without a timeout and even then this version of Windows does 
not recognise Control-C until after the select() call returns (which 
needs similar prompting as with the accept() call. However, select() 
with a timeout allows the code to work both on Windows and Linux. 
Hooray!


For anyone else who is looking for this the earlier test code was 
changed to


port = 8880
import select
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setblocking(0)
s.bind(("", port))
s.listen(1)
while 1:
 ready = select.select((s,), (), (), 0.5)
 #print '(ready %s)' % repr(ready)
 if (ready[0]):
   try:
 endpoint = s.accept()
   except socket.error, details:
 print 'Ignoring socket error:', repr(details)
 continue
   print '(endpoint %s)' % repr(endpoint)
   if endpoint:
 break

James

--
https://mail.python.org/mailman/listinfo/python-list

Re: converting strings to hex

[James Harris]

Mark H Harris harrismh...@gmail.com wrote in message 
news:533e1b2e.5040...@gmail.com...
 On 4/3/14 9:10 PM, dave em wrote:

 I am taking a cryptography class and am having a
 tough time with an assignment similar to this.


 hi Dave, if your instructor wanted you to work on this with other people 
 she would have made it a group project and ordered pizza for everyone.

 I'll give you some credit, that last couple of folks that wanted help with 
 their homework here could not bring themselves to admit they wanted help 
 with their homework.   :)

YMMV but I thought the OP had done a good job before asking for help and 
then asked about only a tiny bit of it. Some just post a question!

HAL, please help me with my homework!

I'm sorry, Dave, I can't do that...

HAL!!?

I'm sorry, Dave, I just can't do that...

You might find this interesting.

  http://sundry.wikispaces.com/transcript-2001

James


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: With this artifact, everyone can easily invent new languages

[James Harris]

Simeon Chaos simeon.ch...@gmail.com wrote in message 
news:d7878ab7-2f6d-4bc4-9a28-3ea567bdf...@googlegroups.com...
 Thank you, James. I didn't know this group before. I'll post this message 
 there.

You're welcome. It can be hard to find apt groups on Usenet because there 
are so many. I don't think there was ever a group for programming language 
design (or invention) so that one which was about miscellaneous languages 
picked up the topic.

By the way, you could use comp.lang.misc simply for an announcement such as 
you posted here. If you want to start a discussion about an aspect of your 
design, however, the best option is to post a specific topic.

James


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to get Mac address of ethernet port?

[James Harris]

Andriy Kornatskyy andriy.kornats...@live.com wrote in message 
news:mailman.5329.1389450993.18130.python-l...@python.org...
 Sam,

 How about this?

 from uuid import getnode as get_mac
 '%012x' % get_mac()

AIUI that will return a mac address even if there isn't one. That may or may 
not suit the OP.

To the OP, depending on what you want to do remember that a machine can have 
more than one mac address and that a mac address can differ from the 
burned-in address (BIA) as some cards allow the effective mac address to be 
changed in software. So it's possible that two machines could show the same 
mac address.

James


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: With this artifact, everyone can easily invent new languages

[James Harris]

Simeon Chaos simeon.ch...@gmail.com wrote in message 
news:bb7d8d30-845a-4a3d-9b03-dee71ef42...@googlegroups.com...
 ? 2014?1?11UTC+8??10?17?33?,Chris Angelico??:
  On Sat, Jan 11, 2014 at 11:59 AM, Simeon Chaos wrote:
 
   All along, the design of programming languages is a complex task. 
   Because we need to understand the esoteric compiler theory and 
   technology, and one of the most critical and very difficult part is to 
   define the rules of the new language and to parse with them.To solve 
   this problem, there have been many theories , techniques and tools . 
   These tools can be roughly divided into two categories: one is parser 
   generator, another is to write the parser by hand with or without a 
   parser library.

 Yes, it's complex to design a new language. So don't let the tool stand in 
 the way. There is a saying: Grinding a chopper will not hold up the work 
 of cutting firewood.

To the OP: this is a suitable topic for comp.lang.misc which is used for 
discussions about programming language design and implementation such as 
parse mechanisms.

James


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird problem with UDP and gevent

[James Harris]

Roy Smith r...@panix.com wrote in message 
news:l3riea$82$1...@panix2.panix.com...
 I'm running:

 Ubuntu Precise
 Python 2.7.3
 django 1.4.5
 gunicorn 0.17.4
 gevent 1.0dev (rc3)

 I haven't been able to pin this down exactly, but it looks like if I
 do (inside of a custom logging.Handler subclass):

   # Paraphrased from the actual code
 remote_addr = (localhost, 9700)
  self.socket = socket.socket(type=socket.SOCK_DGRAM)
payload = ...
 self.socket.connect(remote_addr)
self.socket.send(payload)

 I get intermittant hangs in the connect() call.  If I rewrite this as:

 remote_addr = (localhost, 9700)
self.socket = socket.socket(type=socket.SOCK_DGRAM)
payload = ...
self.socket.sendto(payload, remote_addr)

 everything works fine.  Has anybody seen anything like this?  I'm
 guessing this is some kind of gevent bug.

Those are two different things. You would normally use connect() on a 
SOCK_STREAM socket. It requires that the remote endpoint, in this case 
localhost:9700, has an open socket listening for connections. sendto() is 
the right thing to use with SOCK_DGRAM.

James


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to streamingly read text file and display whenever updated text

[James Harris]

galeom...@gmail.com wrote in message 
news:04ee91f9-1cbf-4364-bca3-da25aa4db...@googlegroups.com...


 #!/usr/bin/python
 import time
 f = open('/home/martin/Downloads/a.txt')

Looks like you are on Unix so you can do this from the shell

  tail -F /home/martin/Downloads/a.txt

James


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Handling 3 operands in an expression without raising an exception

[James Harris]

? nikos.gr...@gmail.com wrote in message 
news:l20utg$278$1...@dont-email.me...
  26/9/2013 12:07 ??, ?/? Antoon Pardon ??:
 Experiment and find out for yourself. That is the only way you will
 acquire the experience and understanding you need. Sure we could
 spoon feed you the line you need, but that will only result in you
 using that line without any understanding of what you are doing so
 that if in a few months time something goes wrong, you again will
 have no understanding of what goes on and still will have no clue
 on how to handle a problem.

 I have tried code and also provided alternative code to solve my problem 
 which also doesn't solve it.

 So, you cannot accuse me that i'm not trying, that would be the case to 
 just ask for a line without trying anything for myself, which i did twice.

Agreed. You did post the work you had already done which seems reasonable to 
me.

For your first example, because you are worried about key errors maybe you 
could code something like the following.

  try:
city = gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
  except KeyError:
try:
  city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] )
except KeyError:
  city = ??? 

Does that help?

James


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python TUI that will work on DOS/Windows and Unix/Linux

[James Harris]

Michael Torrie torr...@gmail.com wrote in message 
news:mailman.229.1378858009.5461.python-l...@python.org...

...

 A toolkit (that's old, arguably), that might help you is TVision, a port
 of the old Turbo Vision library that formed the foundation for Borland's
 old DOS IDEs back in the day (check wikipedia).  And it looked quite
 beautiful back then, actually.  There is a Python binding for it here:

 https://pypi.python.org/pypi/PyTVision

That looks very good.

For the record, I have been emailed about npyscreen and urwid. And I have 
found a hex editor with the kind of interface I had in mind. Here are some 
links for anyone else who is interested.

  http://www.sigala.it/sergio/tvision/images.html
  http://www.npcole.com/npyscreen/
  http://excess.org/urwid/examples.html
  http://www.hexedit.com/hex-edit-shots.htm

James


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python TUI that will work on DOS/Windows and Unix/Linux

[James Harris]

James Harris james.harri...@gmail.com wrote in message 
news:kvmvpg$g96$1...@dont-email.me...
 Am looking for a TUI (textual user interface) mechanism to allow a Python 
 program to create and update a display in text mode. For example, if a 
 command prompt was sized 80x25 it would be made up of 80 x 25 = 2000 
 characters. The Python program would need to be able to write to any of 
 those 2000 characters at any time though in practice the display would 
 normally be arranged by dividing it up into non-overlapping rectangular 
 regions.

 I have seen that there are various libraries: urwid, newt, console, dialog 
 etc. But they seem to be either for Unix or for DOS, not for both. I am 
 looking for a library that will run under either.

In case anyone else is following this, people have emailed me directly 
suggesting ncurses, pdcurses and these:

Pygcurse (http://inventwithpython.com/pygcurse/)
UniCurses (http://sourceforge.net/projects/pyunicurses/)

Naturally, all of these are centred on curses. I have been reading up on it 
and must say that the whole curses approach seems rather antiquated. I 
appreciate the suggestions and they may be what I need to do but from what I 
have seen of curses it was designed principally to provide common ways to 
control cursor-based terminals. That was a-la-mode in the days when we had 
terminals with different cursor control strings and I remember programming 
VT100 and VT52 monitors or terminals like them. But now it seems cumbersome.

I haven't thought too much about it so this is not a design proposal but it 
might be better to divide a display up into non-overlapping windows and 
treat each one separately. Writes to one would not be able to affect the 
others. A given window could allow writes to fixed locations or could behave 
as a glass teletype, writing to the bottom of the window and scrolling as 
needed, or could behave as a viewing port into a data structure. Something 
like that may be more useful to a programmer even if it has to use curses 
underneath because that's all that the OS provides.

James


-- 
https://mail.python.org/mailman/listinfo/python-list

Python TUI that will work on DOS/Windows and Unix/Linux

[James Harris]

Am looking for a TUI (textual user interface) mechanism to allow a Python 
program to create and update a display in text mode. For example, if a 
command prompt was sized 80x25 it would be made up of 80 x 25 = 2000 
characters. The Python program would need to be able to write to any of 
those 2000 characters at any time though in practice the display would 
normally be arranged by dividing it up into non-overlapping rectangular 
regions.

I have seen that there are various libraries: urwid, newt, console, dialog 
etc. But they seem to be either for Unix or for DOS, not for both. I am 
looking for a library that will run under either.

Furthermore, some libraries are complex, providing widgets of all kinds. I 
am looking for something much simpler and the lighter-weight it is the 
better. At least at this stage I pretty much just want to divide the screen 
up into panels.

Input from keyboard would be essential. Input from a mouse would be nice to 
have.

Especially if you have had a similar requirement in the past but even if 
not, is there any cross-platform system you would recommend?

James


-- 
http://mail.python.org/mailman/listinfo/python-list

RFD: comp.sys.raspberry-pi

[James Harris]

  REQUEST FOR DISCUSSION (RFD)
unmoderated group comp.sys.raspberry-pi

This is a formal Request for Discussion (RFD) for the creation of the
unmoderated newsgroup comp.sys.raspberry-pi.

NEWSGROUPS LINE:
comp.sys.raspberry-pi   Raspberry Pi computers  related hardware and
software.

RATIONALE:

* Discussions on this topic have so far been widely and thinly spread
on Usenet
  - e.g. google site:groups.google.com/group/comp raspberry pi
* Device has been on sale for over a year (since 29 Feb 2012)
* Circa 1 million units sold so far
  - http://www.wired.com/design/2013/01/raspberry-pi-million-boards/
* The maker's web forum, while it is a web site and not Usenet,
currently reports over 299,000 posts and 33,000 threads giving an
indication of the levels of interest in discussion
  - http://www.raspberrypi.org/phpBB3/
* The Raspberry Pi is complex and used by communities having technical
issues to discuss
* Various third-party add-ons are available
* The device has some distinctive features, e.g.,
  * Low cost - base retail prices US$25/35 depending on model
  * Header for interfacing with external electronics
  * Can generate 1080p30 High Definition video
* For further specs see http://en.wikipedia.org/wiki/Raspberry_Pi
* People have been using the device for a variety of unusual purposes
  - http://www.raspberrypi.org/phpBB3/viewforum.php?f=15

All-in-all there seems to be many reasons why a Usenet group would be
well used but to allow the Big 8 Management Board to make an informed
decision if you would read and/or post to the proposed newsgroup
please indicate your interest by replying on this thread. Followups
are set to news.groups.proposals where discussions about the proposal
can be carried out.


CHARTER:

Newsgroup comp.sys.raspberry-pi is for discussion of issues connected
with Raspberry Pi computer systems including set up, programming,
software, operating systems, interfacing, related hardware and
projects.

Posters are expected to abide by normal Usenet standards of decorum,
and to ignore articles intended to disrupt the group.  The usual
suspects are prohibited (spam, binaries, direct advertising, etc.)


PROCEDURE:

For more information on the newsgroup creation process, please see:

  http://www.big-8.org/dokuwiki/doku.php?id=policies:creation

Those who wish to influence the development of this RFD and its final
resolution should subscribe to news.groups.proposals and participate
in the relevant threads in that newsgroup.  This is both a courtesy to
groups in which discussion of creating a new group is off-topic as
well as the best method of making sure that one's comments or
criticisms are heard.

All discussion of active proposals should be posted to
news.groups.proposals

To this end, the followup header of this RFD has been set to
news.groups.proposals.

If desired by the readership of closely affected groups, the
discussion may be crossposted to those groups, but care must be taken
to ensure that all discussion appears in news.groups.proposals as
well.

We urge those who would like to read or post in the proposed newsgroup
to make a comment to that effect in this thread; we ask proponents to
keep a list of such positive posts with the relevant message ID:

Barney Fife, 4jgdnb60fsmzha7znz2dnuvz_rwdn...@sysmatrix.net)

Such lists of positive feedback for the proposal may constitute good
evidence that the group will be well-used if it is created.

DISTRIBUTION:

This document has been posted to the following newsgroups:

  news.announce.newgroups
  news.groups.proposals
  comp.lang.forth
  comp.lang.python
  comp.os.linux.misc


PROPONENT:

James Harris james.harris.1Agmail.com (replace A with @ - but please
discuss the proposal on Usenet!)

CHANGE HISTORY:
2013-03-09  Initial RFD

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Shebang line on Windows?

[James Harris]

On Feb 22, 6:40 pm, Zachary Ware zachary.ware+pyl...@gmail.com
wrote:

 On Fri, Feb 22, 2013 at 12:16 PM, Walter Hurry walterhu...@lavabit.com 
 wrote:

  I use FreeBSD or Linux, but my son is learning Python and is using
  Windows.

  My question is this: Would it be good practice for him to put #!/usr/bin/
  env python at the top of his scripts, so that if made executable on *nix
  they will be OK? As I understand it this will have no effect on Windows
  itself.

 Adding the shebang line on Windows would be excellent practice.

A word of warning unless this has since been resolved: Whenever I have
tried adding the shebang line on Windows and running it on Unix the
latter has complained about the carriage return at the end of the
line. This means that Unix does not work when invoked as follows.
(And, yes, the file has had chmod +x applied.)

  ./program.py

It is, of course, OK when run as

  python program.py

but that removes some of the benefit of the shebang line.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What was your strategy?

[James Harris]

On Nov 14, 10:32 pm, Jorge Biquez jbiq...@icsmx.com wrote:
 Hello all.
 Quick question. I know some of you are with Python since started,
 some other maybe later.

 I was wondering if you can share what was the strategy you followed
 to master Python (Yes I know I have to work hard study and practice a
 lot). I mean did you use special books, special sites, a plan to
 learn each subject in a special way. I would like to know, if
 possible, comments specially from some of you who in the past had
 other languages, frameworks and platforms and left (almost) all of
 them and stayed with Python.

IMHO there's no one solution. What works for a person depends on how
that person learns. Options: books, online free course lecture videos,
class instruction, preexisting code, supplied documentation, online
tutorials etc.

I find it useful to have at least two sources, e.g. two books, as each
author brings a slightly different approach and often make different
assumptions (and make different mistakes). Comparing two or more makes
it easier to see through the differences. But make sure each is
reputable in its own right.

For example, I used: Learning Python, and Python in a Nutshell (and
WxPython in action for the GUI stuff). I'd recommend at least the
Nutshell book as a reference.

If books work for you check Amazon or similar for feedback of others.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Does everyone keep getting recruiting emails from google?

[James Harris]

On 14 Oct, 16:49, Daniel Fetchinson fetchin...@googlemail.com wrote:
 I keep getting recruiting emails from charlesngu...@google.com about
 working for google as an engineer. The messages are pretty much the
 same and go like this:

...

 I'm guessing I'm not the only one on this list to get these emails and
 suspect that pretty much everyone gets them. Is that the case? If yes,
 what's the point of spamming a more-or-less random set of people who
 although are probably interested in IT-related stuff but who can
 otherwise also be a set of dogs. Aren't enough people applying without
 this?

I had one - or, rather, two - because I'd posted to a Unix admin
newsgroup. Could have been legit but wasn't of interest so I didn't
follow it up so can't add any more.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: if the else short form

[James Harris]

On 5 Oct, 06:52, Lawrence D'Oliveiro l...@geek-
central.gen.new_zealand wrote:
 In message
 e8b46ea8-8d1e-4db9-91ba-501fd1a44...@g18g2000yqk.googlegroups.com, James

 Harris wrote:
  On 29 Sep, 18:20, Seebs usenet-nos...@seebs.net wrote:

  On 2010-09-29, Tracubik affdfsdfds...@b.com wrote:

  button = gtk.Button((False,, True,)[fill==True])

  Oh, what a nasty idiom.

  I'm surprised you don't like this construct. I hadn't seen it until I
  read the OP's question just now. However, it's meaning was immediately
  apparent.

 I’ve used it a lot, from habit because I only started heavily using Python
 with version 2.4.

 I’m still not sure I’m comfortable with “true-part if cond else false-
 part”, when just about every other language manages to standardize on
 “cond ? true-part : false-part”.

For the bit you are not comfortable with do you mean

  (false-part, true-part)[cond]

Of course, this is just an expression containing a selection.
Arbitrarily complex tests can be dealt with in languages where if
statements and case statements can be expressions. IIRC the great
Algol 60 allowed if statements to return a value. I can't say I can
see why a number of subsequent languages don't allow this.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: if the else short form

[James Harris]

On 29 Sep, 18:20, Seebs usenet-nos...@seebs.net wrote:
 On 2010-09-29, Tracubik affdfsdfds...@b.com wrote:

  Hi all,
  I'm studying PyGTK tutorial and i've found this strange form:

  button = gtk.Button((False,, True,)[fill==True])

  the label of button is True if fill==True, is False otherwise.

  i have googled for this form but i haven't found nothing, so can any of
  you pass me any reference/link to this particular if/then/else form?

 Oh, what a nasty idiom.

 Here's the gimmick.
         (False,, True,)
 is a tuple.  That means you can index it.  For instance:
         (False,, True,)[0]
 is the string False,.

 So, what is the numeric value of fill == True?  Apparently, at least
 at the time this was written, it was 0 if fill was not equal to True,
 and 1 if fill was equal to True.

 Let me say, though, that I'm a C programmer, so I'm coming from a language
 where the result of 0-or-1 for test operators is guaranteed, and I still
 wouldn't use this in live code.  It's insufficiently legible.

I'm surprised you don't like this construct. I hadn't seen it until I
read the OP's question just now. However, it's meaning was immediately
apparent.

I should say where I'm coming from. Contrast the following C and
Python:

  text = fill == true ? True, : False,;   (C)
  text = (False,, True,)[fill == true](Python)

I never liked C's ?: construct partly because it doesn't scale well.
To accept more than two options it requires the programmer to build a
small hierarchy which can be awful to read and may be best expressed
as a separate function. I'd rather have a language change a predicate
to a small integer and use that to index a set of results - and this
is exactly what the OP's tutorial does.

I'm not saying you are wrong, merely expressing a different view and
explaining why.

As another hypothetical example where sgn() returns -1, 0 or +1

  position = (less, equal, greater)[sgn(a - b) + 1]

Though where the list gets much longer it would be good to be able to
label the cases for legibility.

cc. comp.lang.misc

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sharing a program I wrote

[James Harris]

On 5 May, 04:25, Scott scott.freem...@gmail.com wrote:
 James,

 Thanks for the comprehensive reply. I would like to post it to
 comp.lang.python but the main file is 169 lines long and the file for
 functions is 316 lines long. I'm thinking that is a little long for
 this format.

You're welcome. Yes, it sounds a bit long for a direct post. In
general, comments are more forthcoming on smaller pithier code
fragments.

 Maybe I can put them up on a basic web page or file
 sharing site and just post a link. Is that well received on this
 forum?

I can't answer for how it would be received here but once you have
addressed any Python issues you may want to let folks on either or
both of

  comp.dcom.sys.cisco
  comp.dcom.net-management

know about your utility.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sharing a program I wrote

[James Harris]

On 4 May, 22:40, Scott scott.freem...@gmail.com wrote:
 I'm looking for suggestions on what to do (and how to do it) if I want
 to share a program that I wrote in Python. There seem to be quite a
 few places to post code and I don't know how to choose.

Perhaps look at the options and then select whichever suits your needs
best.

 I wrote a program (script?) that takes a text file containing the
 output of  the show access-list command on a Cisco PIX/ASA/FWSM
 firewall and any number of text files containing syslog output from
 the same firewall and creates a report showing which access-list rules
 allowed which actual connections. It is written in Python 2.6 and runs
 on Windows.

 Since this is obviously something mankind has long been waiting for I
 am thinking about sharing it - but since I am new to Python and
 programming in general I am not at all familiar with dealing with
 source code.

 I'm sure that improvements and additions could be made if it was
 reviewed by actual programmers but I wouldn't exactly call it a
 project either. Of course I'd love to add a gui interface...

An option if you want reviews - and your code is short - is to post it
here to comp.lang.python.

 I've seen pypi. It seems to index code that is posted on all sorts of
 sites - including pypi itself? And what is a package anyway? I've
 seen sourceforge. It looks like a good home for big applications or
 multi-developer projects. Freshmeat? Google code? My own website? Your
 blog?

Yes, many of these are well suited for significant projects. I set up

  http://codewiki.wikispaces.com/

Its focus is not on code size per se but on promoting sharing and
updates. However, *smaller* pieces of code are preferred, i.e. code
that can be viewed online and learned from. For example, see the
Python programs at

  http://codewiki.wikispaces.com/index_by_language#x-Portable%20Python


 Another detail is that my program uses a library that was written by
 someone else. It is the most excellent netaddr written by David P. D.
 Moss and it lives at code.google.com. It uses the New BSD License.
 Since this library is required would I simply provide a link to it?
 Would I post the actual library? Do I have to post a copy of his
 copyright info anywhere? Please don't tell me I have to write some
 kind of installer that takes care of providing that.

Unless you are worried that the original copy of the library may be
deleted or lost it should be enough to post a link. Then the web site
that contains the code would be responsible for explaining its licence
and will include other relevant documentation.

For my own code on codewiki I include installation instructions as
text, where necessary.

 I really just want anyone who might need a little networking/security
 tool like this to be able to find it. Any advice?

That was similar to my motive. Interestingly the highest number of
page hits on the site is for a networking utility.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python database of plain text editable by notepad or vi

[James Harris]

On 26 Mar, 14:58, Jon Clements jon...@googlemail.com wrote:
 On 26 Mar, 09:49, James Harris james.harri...@googlemail.com wrote:
...
  I'm toying with some
  ideas for a way to help generate source code (in various languages,
  not just Python). If it goes ahead the text files would be mainly
  marked-up code snippets - with or without symbols that need to be
  replaced.

  Rather than write one single monolithic app I thought to split it into
  reusable components. One part being data access could perhaps be an
  existing database (and I'll take a look at jkn's suggestion).

  Think of the database as similar to an associative array stored on
  disk. The only difference is I may want to play fast and loose with
  the keys in some ways - e.g. check for partial key matches or return a
  list of part-matched keys. The language name could be part of the key
  but I'd also need to store variants for specific language versions.
  I'm not sure yet how it will all pan out. As I say, just throwing
  around some ideas.
...
 Thanks for the explanation.

No problem. Thanks for taking an interest!

 Who admins and, who's editing this data?

For the app I have in mind a programmer would admin and edit the
files.

 I couldn't 100% guarantee that I could modify a text file and always
 put the right
 delimiter in the right place and remember to escape the relevant chars
 (and I'm
 probably not the 'average' user).

Apart from any markup each file would be just source code or, maybe,
individual fragments of source code so there would be few delimiters
to get right. And any that were wrong could be detected either by the
code writer or the compiler when used.

 Any opposition to just putting it in a 'proper' DB, then 'blobbing'
 the values?
 (or just integrate a procedure/script/function whatever your chosen
 RDBMS calls to choose it).
 Or in some systems, 'externally referencing'... loads of DB's have
 free front-ends,
 and there are lots of Python libraries.

I've thought of that. Like Ethan, though, I'd prefer simple text for
this. Databases, for good reasons, tend to obscure text. For one thing
it makes them more secure but it also makes them less transparent and
harder to examine and edit. For this app simple text files seem to be
the best option at the moment.


 I think perhaps, all I'm saying is, I'd choose a different approach.
 I'd provide a front-end, rather than choose to re-write the wheel over
 DB's.

Agreed. That was my reason for asking the question initially.

 Be nice to know how you get on, if you'd be so kind?

I don't know yet if it will be feasible but if I do eventually write
something I'll report back.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python database of plain text editable by notepad or vi

[James Harris]

On 26 Mar, 10:04, Harishankar v.harishan...@gmail.com wrote:
...
  Think of the database as similar to an associative array stored on
  disk. The only difference is I may want to play fast and loose with
  the keys in some ways - e.g. check for partial key matches or return a
  list of part-matched keys. The language name could be part of the key
  but I'd also need to store variants for specific language versions.
  I'm not sure yet how it will all pan out. As I say, just throwing
  around some ideas.
...
 I am not sure exactly what you need, but would you consider using
 something like ConfigParser module provided by Python? It appears to be
 something similar to what you need.

Thanks, I'll take a look at it.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Python database of plain text editable by notepad or vi

[James Harris]

I am looking to store named pieces of text in a form that can be
edited by a standard editor such as notepad (under Windows) or vi
(under Unix) and then pulled into Python as needed. The usual record
locking and transactions of databases are not required.

Another way to look at it is to treat the separate files as entries in
a dictionary. The file name would be the key and the lines of the file
the value.

Anyone know of a database (with a Python interface) which will allow
text files to be treated as database fields? If not I can just write
it but I thought it best to ask if there was an existing solution
first.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Anything like Effective Java for Python?

[James Harris]

On 10 Mar, 15:19, kj no.em...@please.post wrote:
 Subject line pretty much says it all: is there a book like Effective
 Java for Python.  I.e. a book that assumes that readers are
 experienced programmers that already know the basics of the language,
 and want to focus on more advanced programming issues?

I don't know about the Java book you mention but I find Python in a
Nutshell published by O'Reilly to be a good reference.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Named loops for breaking

[James Harris]

On 10 Mar, 06:29, Gabriel Genellina gagsl-...@yahoo.com.ar wrote:
 En Tue, 09 Mar 2010 18:41:10 -0300, Daniel Klein bri...@gmail.com  
 escribi :

  Basically I'm wondering if there are any plans to implemented named
  loops in Python, so I can tell a break command to break out of a
  specific loop in the case of nested loops.

 See PEP3136 [1] and its rejection note [2]
 I think you may find some more discussion in the python-ideas list.

  Currently I'm using boolean
  flag variables, but I find that very clumsy. I know this idea of
  breaking out of specific loops from javascript; I guess java does it
  too. It just seems a very Pythonian idea to me: clear, promotes code
  legibility, seems obvious.

 Although I've occasionally missed the feature myself, I agree with Guido's  
 arguments against it.

I don't agree with Guido's second reason in particular. He writes in
the link Gabriel provided

   [2]http://mail.python.org/pipermail/python-3000/2007-July/
008663.html

G However, I'm rejecting it on the basis that code so complicated to
G require this feature is very rare. In most cases there are existing
G work-arounds that produce clean code, for example using 'return'.
G While I'm sure there are some (rare) real cases where clarity of
the
G code would suffer from a refactoring that makes it possible to use
G return, this is offset by two issues:

G 1. The complexity added to the language, permanently. This affects
not
G only all Python implementations, but also every source analysis
tool,
G plus of course all documentation for the language.

Guido may have a point here about source tools but I don't think the
language becomes more complex. If anything it would become more
orthogonal - i.e. fewer arbitrary restrictions. And documentation is
needed for any change so saying that documentation would need to be
updated is an odd reason to reject a change.

G 2. My expectation that the feature will be abused more than it will
be
G used right, leading to a net decrease in code clarity (measured
across
G all Python code written henceforth). Lazy programmers are
everywhere,
G and before you know it you have an incredible mess on your hands of
G unintelligible code.

Breaking out of an inner loop is just as natural as breaking out of
the immediately enclosing loop. ISTM that if one is allowed the other
should be also.

 You have several alternatives: refactor the loop  
 into an auxiliary function, use a specific exception, add boolean flags,  
 or repeat the test at the outer levels. (I usually choose the first).

The auxiliary function idea (Guido's preference as well as Gabriel's)
works but it may require accessing of variables which don't appear in
the function interface, and the return in that function is no
different from the break dropping through multiple levels. Return does
exactly that (as well as setting a result value).

There are often times when it *is* better to factor out the code to a
different function but adding a function just to enable a break from
an inner loop doesn't seem to me a good reason.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Executing Python code on another computer

[James Harris]

On 19 Feb, 15:52, SiWi wimmersi...@googlemail.com wrote:
 Hello community,
 I googled for an answer of the following problem, but I couldn't find
 anything.
 I've got a netbook and my fast workstation compter, which I usually
 use for developing.
 But I'd also like to take my netbook around the house and to develop
 Python programms on it.
 The problem is that naturally a netbook is not the fastest computer
 you could find.

 So I wondered if it was possible to send the Python code I'm
 developing on the netbook to the workstation pc via wlan, let the
 script execute on the workstation pc and write the output back on the
 netbook.

 Is there any possibilty to achieve that goal?

Yes. Assuming you can cope with the relatively small netbook screen
here are some options:

1. Telnet (ok within a home and where no graphics needed)
2. Ssh (ok where no graphics needed)
3. An X-Windows server on your netbook (ok under Linux but good
Windows X Servers may be limited or nonexistent)
4. VNC (e.g. RealVnc) to get a remote view of the workstation's
screen.

I use telnet and RealVnc for purposes similar to those you describe.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Available for use: Tabs management software

[James Harris]

In case someone else finds it useful here is a program I wrote a year
or so ago to help manage tab characters. It will convert tabs to runs
of spaces, convert runs of spaces to tabs, or count or check for tab
characters as required. It supports tab stops at regular and irregular
positions.

  http://codewiki.wikispaces.com/tabs.py

It is written in Python and you can either use it from the command
line or call the functions from another Python script. Documentation
is included on the web page.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Available for use: Tabs management software

[James Harris]

On 7 Feb, 21:25, James Harris james.harri...@googlemail.com wrote:
 In case someone else finds it useful here is a program I wrote a year
 or so ago to help manage tab characters. It will convert tabs to runs
 of spaces, convert runs of spaces to tabs, or count or check for tab
 characters as required. It supports tab stops at regular and irregular
 positions.

  http://codewiki.wikispaces.com/tabs.py

 It is written in Python and you can either use it from the command
 line or call the functions from another Python script. Documentation
 is included on the web page.

After posting this I realised the code had a dependence on a separate
debugging module. I've commented-out all such references and tested it
in isolation from other code. Everything behaved as it should so if
you download it you should find it just works. The only dependence is
on Python (i.e. Python 2.x; I gather 3.x needs print statements and
maybe other stuff to be changed). Any issues, feel free to post them
here.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Language mavens: Is there a programming with if then else ENDIF syntax?

[James Harris]

On 3 Dec, 20:56, Michael Torrie torr...@gmail.com wrote:
 Steve Ferg wrote:
  Is there any particular reason why this might be a *bad* language-
  design idea?

 Syntactically, using braces, begin/end blocks, python white space, and
 your if/elif/then/endif structures, amounts to the same thing; they are
 all equivalent.  Thus from a language pov, there's no compelling reason
 to do one over the other.

 One good reason to avoid the if/elif/then/endif syntax and its related
 friends in a language (while/wend, do/loop, for/next) is that they
 require parser support for what amounts to three or four versions of
 precisely the same construct: the block.  Thus from a parser or compiler
 point of view, it's far better and easier to have blocks begin and end
 with something that is consistent and recognizable without a lot of
 look-ahead or context.

I'm not disagreeing with the above paragraph but in the interests of
even handedness an advantage of the other method is ease for a human
to match begins and ends of large blocks. With statement or block
brackets such as begin ... end (Pascal) and { ... } (C)
because they are all the same it can be hard to see where each
construct ends. C programmers sometimes write

} /* for */
  } /* while */

or similar to indicate which construct the terminating brace is
closing.

 I once started to write a parser for the QuickBasic language but found
 very quickly that the grammar required a fair amount of token
 look-ahead.  For example, if I found the token End, I had to look at
 least one token ahead to find out if this was an end if or just an
 end.  Also end could, if I recall, have an optional number parameter
 that would set the exit errorlevel code.  Certainly QuickBasic was not
 context-free and was not LL(1).

 I'm not sure, but Python's grammar is LL(1) I think, and probably darn
 close to context-free.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy way to play single musical notes in Python

[James Harris]

On 15 Nov, 05:41, r rt8...@gmail.com wrote:
 On Nov 14, 6:21 pm, James Harris james.harri...@googlemail.com
 wrote:

  Is there a simple way to play musical notes in Python? Something like
    voice.play(c4)

 Uhh, tksnack is pretty easy to use IMO, see this link... 
 http://www.daniweb.com/code/snippet216655.html

 No python does not have access to cross platform soundcard
 capabilities built into the language. I think there is a wrapper for
 csound somewhere. But there are many 3rd party modules that do have
 capabilities to some extent. You could make calls to the underlying OS
 machinery and there is the winsound module (not exactly what you want
 though). Just look over this Google splooge...

 http://www.google.com/search?hl=enrlz=1C1CHMI_enUS340US340q=Python+...

As I say I was hoping to avoid tk. Thanks for the feedback though. If
nothing else is suggested I'll have to try snack.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Language mavens: Is there a programming with if then else ENDIF syntax?

[James Harris]

On 16 Nov, 16:54, Steve Ferg steve.ferg.bitbuc...@gmail.com wrote:
 This is a question for the language mavens that I know hang out here.
 It is not Python related, except that recent comparisons of Python to
 Google's new Go language brought it to mind.

 NOTE that this is *not* a suggestion to change Python.  I like Python
 just the way it is.  I'm just curious about language design.

 For a long time I've wondered why languages still use blocks
 (delimited by do/end, begin/end, { } , etc.) in ifThenElse statements.

 I've often thought that a language with this kind of block-free syntax
 would be nice and intuitive:

     if condition then
         do stuff
     elif condition then
         do stuff
     else
         do stuff
     endif

 Note that you do not need block delimiters.

 Obviously, you could make a more Pythonesque syntax by using a colon
 rather then then for the condition terminator.  You could make it
 more PL/I-like by using do, etc.

 You can write shell scripts using if ... fi, but other than that I
 don't recall a language with this kind of syntax.

 Does anybody know a language with this kind of syntax for
 ifThenElseEndif?

 Is there any particular reason why this might be a *bad* language-
 design idea?

There are some. For example, Ada uses similar. See

  http://en.wikipedia.org/wiki/Ada_%28programming_language%29#Control_structures

These other newsgroups may be of interest:

  comp.programming
  comp.lang.misc

The latter is used by people designing programming languages where you
can find knowledgeable comments aplenty.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Easy way to play single musical notes in Python

[James Harris]

Is there a simple way to play musical notes in Python? Something like

  voice.play(c4)

to play C in octave 4 would be ideal. I included a voice parameter as
I'd like to play proper notes, not just beeps. This is for recognition
of pitch. For example, the program plays a note and the user tries to
identify the note played.

There are many options at

  http://wiki.python.org/moin/PythonInMusic

but which to choose? They generally seem too complex. I'm looking for
something really basic. It would help if it was cross platform and
didn't need Tkinter as the snack library does. I presume pure Python
is not possible.

Any suggestions?

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy way to play single musical notes in Python

[James Harris]

On 15 Nov, 00:12, James Harris james.harri...@googlemail.com wrote:

 Is there a simple way to play musical notes in Python? Something like

   voice.play(c4)

 to play C in octave 4 would be ideal. I included a voice parameter as
 I'd like to play proper notes, not just beeps. This is for recognition
 of pitch. For example, the program plays a note and the user tries to
 identify the note played.

 There are many options at

  http://wiki.python.org/moin/PythonInMusic

 but which to choose? They generally seem too complex. I'm looking for
 something really basic. It would help if it was cross platform and
 didn't need Tkinter as the snack library does. I presume pure Python
 is not possible.

 Any suggestions?

Oh, it would be OK if the system allowed numeric pitches such as

  voice.play(440)

to play a note at 440 Hertz.

Anything like the above should be good enough.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Feedback wanted on programming introduction (Python in Windows)

[James Harris]

On 28 Oct, 08:58, Alf P. Steinbach al...@start.no wrote:
 * tm:

  On 28 Okt., 07:52, Alf P. Steinbach al...@start.no wrote:
  [Cross-posted comp.programming and comp.lang.python]

  Looking at your topic '(Python in Windows)', without taking a
  glimpse at your actual introduction, I have the following to say:
  I think it is not a good idea to teach programming with a focus
  on a specific operating system. Programming should IMHO be taught
  without reference to an operating system. Otherwise you just teach
  how to write unportable programs.

 I think you're trolling a little. :-)

Whether tm is promoting his own language or not I agree with some of
his comments. I was going to avoid giving any feedback because most of
my thoughts are, I'm afraid, negative but seeing your response to tm
here I changed my mind. You asked for feedback. Hopefully you are
looking for genuine criticism and honest opinions rather than flattery
'cause I'm not going to flatter.

If you want to teach programming then target programming concepts
(albeit using Python as a means to express them) and as tm says, avoid
teaching either a particular OS or a particular set of bundled tools.

If you want to teach programming under Windows then change the title
to say so.

Sorry but I find the overall tone too patronising. Phrases like send
your browser galloping in the direction of are offputting. With this
and other phrases it sounds like you are either

1) writing this for young children, or
2) having more fun writing it than your readers will have reading it
or,
3) being self-indulgent and not planning to help others at all.

I know you don't mean any of these. Hopefully you can change the
approach to suit. There are many of these jocular phrases and they
appear in both chapters.

Given that this is a Windows-based course it's good that you include
teaching on Notepad rather than just the IDE.

The x squared graph is a good example to show that some fun can be had
in a small space.

I wouldn't condemn but I would query the use of Activestate's
distribution. A vanilla Python may have been better if you want to
teach transportable skills. Teaching Activestate reminds me of how
Compuserve bundled their own stuff with Internet access so people
thought the Internet was what they saw on Compuserve.

You get way too deep into Python in places (for a beginner's course in
programming). For example, from now on I’ll always use from
__future__ in any program that uses print.

The MIT course that you mention takes a more apt approach for teaching
*programming*. For example, it explains some things like floating
point numbers in Python not being able to express 0.1 perfectly in
binary but that's appropriate as other languages have the same issue.

As you say, you are an experienced programmer who is learning Python
and the chapters read that way. They rush in to all kinds of gotchas
and mechanisms. Perhaps you should either change it to be a book on
learning Python for experienced programmers (this seems the best
option) or start again and take a different approach.

With what you have written so far your audience seems to be youself
(or someone in your position).


 Without reference to an OS you can't address any of the issues that a beginner
 has to grapple with, including most importantly tool usage, without which it's
 not even possible to get started, but also, very importantly, a file system.

There's a difference between referring to an OS and tieing it in
throughout the text which is what I think tm was getting at.


 Learning programming without tools and without using files (or only using the
 common denominator for file systems in OSes X, Y and Z) is sort of vacuous...

OSes generally support concepts of files and editors. If you cannot
teach the general concepts make it clear that you are teaching
Activestate Python under Windows.

 In addition there's the motivational factor.

 Doing only academic examples, utilizing only a language's more or less 
 portable
 functionality, is very de-motivational, while the opposite is motivational.

The graphics and examples are fun. Be clear about what you are
teaching, though, and who your intended audience is.

...

  C++ was way too complex for the novice, JScript and C# suffered from too
  fast-changing specifications and runtime environment, Java, well, nothing
  particularly wrong but it's sort of too large and unwieldy and inefficient.

Agreed. Python is a good introductory language.

  I don't know whether this will ever become an actual book. I hope so!

It's a good start.


  But since I don't know much Python -- I'm *learning* Python as I write

This and that you are an experienced programmer show in what you have
written. If you don't recast the material for a beginner now it will
only get harder to change the approach later.

...

 Yes, it would be silly to write a book or whatever about Python. This text is
 primarily about programming, at the novice level, not about the Python 
 

Re: Annoying octal notation

[James Harris]

On 3 Sep, 14:26, Albert van der Horst alb...@spenarnc.xs4all.nl
wrote:
 In article 6031ba08-08c8-416b-91db-ce8ff57ae...@w6g2000yqw.googlegroups.com,
 James Harris  james.harri...@googlemail.com wrote:
 SNIP



 So you are saying that Smalltalk has base in decimalrnumber where
 r is presumably for radix? That's maybe best of all. It preserves the
 syntactic requirement of starting a number with a digit and seems to
 have greatest flexibility. Not sure how good it looks but it's
 certainly not bad.

   0xff  0x0e | 0b1101
   16rff  16r0e | 2r1101

 Hmm. Maybe a symbol would be better than a letter.

 Like 0#ff  16#ff ?

Yes, that looks better.

 That is ALGOL68. It is incredible how many of it has become
 vindicated over time. (Yes, nineteen hundred sixty eight was
 the year that language was conceived.)

Yes, and its predecessor Algol 60 was a masterful advance in
programming languages. It set up standards we still use today.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Annoying octal notation

[James Harris]

On 3 Sep, 15:35, Grant Edwards inva...@invalid wrote:

...

  Obviously I can't speak for Ken Thompson's motivation in creating this
  feature, but I'm pretty sure it wasn't to save typing or space on
  punchcards. Even in 1969, hex was more common than octal, and yet hex
  values are written with 0x. My guess is that he wanted all numbers to
  start with a digit, to simplify parsing, and beyond that, it was just his
  programming style -- why call the copy command `copy` when you could call
  it `cp`? (Thompson was the co-inventor of Unix.)

 Maybe it was because they were working on minicomputers, not mainframes,
 so there was less processing power and storage available.

  Not just any minicomputers: PDP11. Octal notation is friendly with
  the PDP11 instruction set.

 Indeed.  Octal was the way that all of the DEC PDP-11 sw tools
 displayed machine code/data.

True. Octal was default in Macro-11 and what surprises me is that when
I used it it seemed natural to think in octal (or, preferably, hex)
rather than decimal.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Annoying octal notation

[James Harris]

On 3 Sep, 15:54, Albert van der Horst alb...@spenarnc.xs4all.nl
wrote:
 In article mailman.346.1251135629.2854.python-l...@python.org,
 Derek Martin  c...@pizzashack.org wrote:





 --W1uEbMXJ1Mj4g6TI
 Content-Type: text/plain; charset=iso-8859-1
 Content-Disposition: inline

 On Mon, Aug 24, 2009 at 05:03:28PM +, Steven D'Aprano wrote:
  On Mon, 24 Aug 2009 11:21:46 -0500, Derek Martin wrote:
   since the old syntax is prevalent both within and without the
   Python community, making the change is, was, and always will be a
   bad idea.

  Octal syntax isn't prevalent *at all*, except in a small number of
  niche areas.

 Steven, don't be obtuse.  Where octal is used in programming, the
 leading zero is prevalent.

 That is not the point. Octal is not prevalent. Leading zero's have
 a mathematical sound meaning. The convention is changed because
 every new user to Python will fall once into this trap.
 For a person not familiar with C or the like this will be a
 hair pulling, nail byting, head banging situation.
 A mathematician might even think he is gone mad.

 Regarding you, you will probably have noticed by now that it is
 going to change, so you will not pull your hair, byte your nails

The first time you wrote, byte your nails, I thought you meant it as
a pun. But since you've mentioned it twice

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Annoying octal notation

[James Harris]

On 27 Aug, 18:31, Ethan Furman et...@stoneleaf.us wrote:

 Steven D'Aprano wrote:

  A mistake is still a mistake even if it shared with others.

  Treating its with a lead zero as octal was a design error when it was
  first thought up

 [snippage]

 I have to disagree with you on this one.  The computing world was vastly
 different when that design decision was made.  Space was at a premium,
 programmers were not touch-typists, every character had to count, and
 why in the world would somebody who had to use papertape or punch cards
 add a lead zero without a *real* good reason?  I submit that that real
 good reason was to specify an octal literal, and not a decimal literal.

Nice idea. Characters were expensive but not that expensive - even
then. One extra character to make the octal prefix 0t or 0q or
something could have been used. Check out the History heading at

  http://sundry.wikispaces.com/octal-zero-prefix

Note how B migrated away from both BCPL's octal and its hex notation.

  #octal and #xhexadecimal in BCPL became
  0octal and 0xhexadecimal in B

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Numeric literals in other than base 10 - was Annoying octal notation

[James Harris]

On 25 Aug, 01:25, Steven D'Aprano st...@remove-this-
cybersource.com.au wrote:

 On Mon, 24 Aug 2009 16:23:06 -0700, James Harris wrote:

  Sure but while I wouldn't normally want to type something as obscure as
  32rst into a file of data I might want to type 0xff00 or similar. That
  is far clearer than 65280 in some cases.

  My point was that int('ff00', 16) is OK for the programmer but cannot be
  used generally as it includes a function call.

 No, it's the other way around. If you have *data*, whether entered at run
 time by the user or read from a file, you can easily pass it to a
 function to convert to an int. (In fact you have to do this anyway,
 because the data will be a string and you need an int.)

 If you want your data file to have values entered in hex, or oct, or even

You are looking at this the other way round from me and I'll explain
why I think this is important. I'm not talking about the *programmer*
prescribing how the user should enter the data but the *user* being
able to choose to prepare the data in a certain format - the format
which makes sense for the data being entered. Your way only works if
those fields in every row have the same format. That may be the case
sometimes but is not general.


 unary (1=one, 11=two, 111=three, =four...) you can. There's no need
 to have the user enter int('ff00', 16) to get hex, just have them enter
 ff00.

 But when writing *code*, you want syntax which will accept integers in
 the most common bases (decimal, a distant second hex, an even more
 distant third octal, and way out on the horizon binary) without the
 runtime cost of a function call.

Be careful! When designing a language feature don't be too ready to
*tell* the users of that language what restrictions they want. Just
because the designer wants features with a certain frequency does not
mean that all users will have the same priorities. (If it is
impossible or unreasonable to support a feature then by all means
don't support it. But don't decide up-front before examining the
options that something in not personally appealing.)

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Numeric literals in other than base 10 - was Annoying octal notation

[James Harris]

On 24 Aug, 03:49, Dennis Lee Bieber wlfr...@ix.netcom.com wrote:

...

  Here's another suggested number literal format. First, keep the
  familar 0x and 0b of C and others and to add 0t for octal. (T is the
 thirdletter of octal as X is thethirdletter of hex.) The numbers
  above would be

         The thing is -- x and hex have similar pronunciations: (h)ecks;
 the name of the radix is its own reminder for the character to use
 without thinking such conventions as thirdletter of the radix name.

         But t (tee) has no pronunciation resemblance to oct (o'kt)
 whereas the unlovely o at least if taken as a short vowel sound is
 similar to the o of oct given the short stop between it and the
 ct.

    0b1011, 0t7621, 0xc26b

         And b for binary breaks the thirdletter of radix name
 convention... You should be using n for that (and c for decimal G)

I wasn't proposing a convention of using the third character of the
base name. I was saying that t is not too unreasonable given that we
use x for hex (rather than h).


         Or we use b, o, d, h (as the HP calculator) and drop the x
 version.



  where the three characters 0.( begin the sequence.

         Ugly...

  Comments? Improvements?

         Retaining which ever character is finally decided, I'd make all
 radix specified literals follow a quoted format:

         digitsradix

         01110b
         123d (of course, just 123 would be allowed for simplicity)
         7Cx
         327o

The quoting is good. For hex, decimal, octal and binary, however, I
can't see a good reason to change away from the conventional prefix
form. And, in general, it's easier for a human to parse if the base is
specified first.


         Probably wouldn't need that much change to the parser as it would,
 left to right, see a string, and then when the string is not followed by
 one white space character, find a radix marker -- the parser would then
 parse the string using the specified radix, and emit the appropriate
 numeric value instead of a string value.

Maybe. I think, though, that having the base as prefix would make the
parser's job easier as well as the job of humans. It's easier if we
know what we are parsing before we parse it rather than afterwards.

        It only adds one extra
 character (instead of leading 0r, one has r), and the leading  serves
 as a warning to a reader that this is not directly readable as a number.

         The alternative syntax of radixdigits is the same length, but adds
 to the parsing as it initially looks like a name entity, then hits the
 quote, and has to back up to interpret the result as a radix marker.

True. The beginning of a number should be numeric. Using your scheme,
though, instead of radixdigits you could have 0radixdigits.

 0r
 format starts as a number, hits a radix marker while the
 conversion/accumulator is still a 0 value (0 is 0 in all radix) and
 switches the converter to accept the digits in the specified radix.

Sounds like you are suggesting 0radixdigits but I'm not sure.


         Whereas all prefix versions that don't start with a 0r tend to
 require more complex parsing:

         0.(

 starts out looking like a number (the 0)... a floating point number (the
 .)... a function/method call on a floating point 0... WAIT? floating
 point numbers aren't callables (yet! I'm sure someone is going to show a
 way to define a variable bound to a number as a callable, though not to
 a literal number)... throw away this parse tree branch, back up and
 reparse as special numeric radix prefix...

You've laid it on thick but I agree in principle. What about
radixdigits where radix is numeric: So 21101 or 3122? (Not to
replace 0b1101 etc but to supplement it for arbitrary bases.)


         Of course, one still has to consider what will be used for \
 character encoding... \x0F vs \013 vs \b000?

The plans I had did not allow for the suggestions above so I have no
comments on character encoding yet but it's good that you mentioned
it.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Numeric literals in other than base 10 - was Annoying octal notation

[James Harris]

On 24 Aug, 02:19, Max Erickson maxerick...@gmail.com wrote:

...

  It can be assumed however that .9. isn't in binary?

  That's a neat idea. But an even simpler scheme might be:

  .octal.100
  .decimal.100
  .hex.100
  .binary.100
  .trinary.100

  until it gets to this anyway:

  .thiryseximal.100

 At some point, abandoning direct support for literals and just
 having a function that can handle different bases starts to make a
 lot of sense to me:

  int('100', 8)
 64
  int('100', 10)
 100
  int('100', 16)
 256
  int('100', 2)
 4
  int('100', 3)
 9
  int('100', 36)
 1296

This is fine typed into the language directly but couldn't be entered
by the user or read-in from or written to a file.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Numeric literals in other than base 10 - was Annoying octal notation

[James Harris]

On 24 Aug, 09:05, Erik Max Francis m...@alcyone.com wrote:

...

  Here's another suggested number literal format. First, keep the
  familar 0x and 0b of C and others and to add 0t for octal. (T is the
  third letter of octal as X is the third letter of hex.) The numbers
  above would be

    0b1011, 0t7621, 0xc26b

  Second, allow an arbitrary number base by putting base and number in
  quotes after a zero as in

    02:1011, 08:7621, 016:c26b

  Why not just put the base first, followed by the value in quotes:

      21011, 87621, 16c26b

 It's always a bit impressive how syntax suggestions get more and more
 involved and, if you'll forgive me for saying, ridiculous as the
 conversation continues.  This is starting to get truly nutty.

Why do you say that here? MRAB's suggestion is one of the clearest
there has been. And it incorporates the other requirements: starts
with a digit, allows an appropriate alphabet, has no issues with
spacing digit groups, shows clearly where the number ends and could
take an exponent suffix.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Numeric literals in other than base 10 - was Annoying octal notation

[James Harris]

On 24 Aug, 09:30, Erik Max Francis m...@alcyone.com wrote:
 James Harris wrote:
  On 24 Aug, 09:05, Erik Max Francis m...@alcyone.com wrote:
  Here's another suggested number literal format. First, keep the
  familar 0x and 0b of C and others and to add 0t for octal. (T is the
  third letter of octal as X is the third letter of hex.) The numbers
  above would be
    0b1011, 0t7621, 0xc26b
  Second, allow an arbitrary number base by putting base and number in
  quotes after a zero as in
    02:1011, 08:7621, 016:c26b
  Why not just put the base first, followed by the value in quotes:
      21011, 87621, 16c26b
  It's always a bit impressive how syntax suggestions get more and more
  involved and, if you'll forgive me for saying, ridiculous as the
  conversation continues.  This is starting to get truly nutty.

  Why do you say that here? MRAB's suggestion is one of the clearest
  there has been. And it incorporates the other requirements: starts
  with a digit, allows an appropriate alphabet, has no issues with
  spacing digit groups, shows clearly where the number ends and could
  take an exponent suffix.

 In your opinion.  Obviously not in others.  Which is pretty obviously
 what I meant, so the rhetorical question is a bit weird here.

Don't get defensive Yes, in my opinion, if you like, but you can't
say obviously not in others as no one else but you has commented on
MRAB's suggestion.

Also, when you say This is starting to get truly nutty would you
accept that that's in your opinion?

 There's a reason that languages designed by committee end up horrific
 nightmares.

True but I would suggest that mistakes are also made by designers who
do not seek the opinions of others. There's a balance to be struck
between a committee and an ivory tower.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Numeric literals in other than base 10 - was Annoying octal notation

[James Harris]

On 24 Aug, 14:05, Mel mwil...@the-wire.com wrote:
 James Harris wrote:
  On 24 Aug, 02:19, Max Erickson maxerick...@gmail.com wrote:

 [ ... ]
   int('100', 3)
  9
   int('100', 36)
  1296

  This is fine typed into the language directly but couldn't be entered
  by the user or read-in from or written to a file.

 That's rather beside the point.  Literals don't essentially come from files
 or user input.  Essentially literals are a subset of expressions, just like
 function calls are, and they have to be evaluated by Python to yield a
 value.  I'm not averse to 32'rst', but we already have

...

  int ('rst', 32)

 28573

Sure but while I wouldn't normally want to type something as obscure
as 32rst into a file of data I might want to type 0xff00 or similar.
That is far clearer than 65280 in some cases.

My point was that int('ff00', 16) is OK for the programmer but cannot
be used generally as it includes a function call.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Annoying octal notation

[James Harris]

On 21 Aug, 00:59, James Harris james.harri...@googlemail.com wrote:

...

  Is there some magic to make the 2.x CPython interpreter to ignore the
  annoying octal notation?
  I'd really like 012 to be 12 and not 10.

 This is (IMHO) a sad hangover from C (which took it from B ...

This seemed worth writing up so I've taken the snipped comments and
posted them at

  http://sundry.wikispaces.com/octal-zero-prefix

The idea is that the page can be pointed to any time the issue comes
up again.

I've also fleshed the comments out a bit.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Numeric literals in other than base 10 - was Annoying octal notation

[James Harris]

On 23 Aug, 04:38, c...@tiac.net (Richard Harter) wrote:
 On Sat, 22 Aug 2009 14:54:41 -0700 (PDT), James Harris





 james.harri...@googlemail.com wrote:
 On 22 Aug, 10:27, David 71da...@libero.it wrote:

 ... (snipped a discussion on languages and other systems interpreting
 numbers with a leading zero as octal)

   Either hexadecimal should have been 0h or octal should
   have been 0t :=3D)

  I have seen the use of Q/q instead in order to make it clearer. I still
  prefer Smalltalk's 16rFF and 8r377.

  Two interesting options. In a project I have on I have also considered
  using 0q as indicating octal. I maybe saw it used once somewhere else
  but I have no idea where. 0t was a second choice and 0c third choice
  (the other letters of oct). 0o should NOT be used for obvious reasons.

  So you are saying that Smalltalk has base in decimalrnumber where
  r is presumably for radix? That's maybe best of all. It preserves the
  syntactic requirement of starting a number with a digit and seems to
  have greatest flexibility. Not sure how good it looks but it's
  certainly not bad.

 I opine that a letter is better; special characters are a
 valuable piece of real estate.

Very very true.

  However for floating point you
 need at least three letters because a floating point number has
 three parts: the fixed point point, the exponent base, and the
 exponent.  Now we can represent the radices of the individual
 parts with the 'r'scheme, e.g., 2r101001, but we need separate
 letters to designate the exponent base and the exponent.  B and E
 are the obvious choices, though we want to be careful about a
 confusion with 'b' in hex.  For example, using 'R',

 3R20.1B2E16Rac

Ooh err!


 is 20.1 in trinary (6 1/3) times 2**172 (hex ac).

 I grant that this example looks a bit gobbledegookish,

You think? :-)

 but normal
 usage would be much simpler.  The notation doesn't handle
 balanced trinary; however I opine that balanced trinary requires
 special notation.

When the programmer needs to construct such values how about allowing
him or her to specify something like

  (20.1 in base 3) times 2 to the power of 0xac

Leaving out how to specify (20.1 in base 3) for now this could be

  (20.1 in base 3) * 2 ** 0xac

The compiler could convert this to a constant.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Numeric literals in other than base 10 - was Annoying octal notation

[James Harris]

On 23 Aug, 00:16, Mel mwil...@the-wire.com wrote:
 James Harris wrote:
  I have no idea why Ada which uses the # also apparently uses it to end
  a number

    2#1011#, 8#7621#, 16#c26b#

 Interesting.  They do it because of this example from
 http://archive.adaic.com/standards/83rat/html/ratl-02-01.html#2.1:

Thanks for providing an explanation.


 2#1#E8                    -- an integer literal of value 256

 where the E prefixes a power-of-2 exponent, and can't be taken as a digit of
 the radix.  That is to say

 16#1#E2

 would also equal 256, since it's 1*16**2 .

Here's another suggested number literal format. First, keep the
familar 0x and 0b of C and others and to add 0t for octal. (T is the
third letter of octal as X is the third letter of hex.) The numbers
above would be

  0b1011, 0t7621, 0xc26b

Second, allow an arbitrary number base by putting base and number in
quotes after a zero as in

  02:1011, 08:7621, 016:c26b

This would work for arbitrary bases and allows an exponent to be
tagged on the end. It only depends on zero followed by a quote mark
not being used elsewhere. Finally, although it uses a colon it doesn't
take it away from being used elsewhere in the language.

Another option:

  0.(2:1011), 0.(8:7621), 0.(16:c26b)

where the three characters 0.( begin the sequence.

Comments? Improvements?

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Numeric literals in other than base 10 - was Annoying octal notation

[James Harris]

On 23 Aug, 21:55, James Harris james.harri...@googlemail.com wrote:

...

   However for floating point you
  need at least three letters because a floating point number has
  three parts: the fixed point point, the exponent base, and the
  exponent.  Now we can represent the radices of the individual
  parts with the 'r'scheme, e.g., 2r101001, but we need separate
  letters to designate the exponent base and the exponent.  B and E
  are the obvious choices, though we want to be careful about a
  confusion with 'b' in hex.  For example, using 'R',

  3R20.1B2E16Rac

 Ooh err!

  is 20.1 in trinary (6 1/3) times 2**172 (hex ac).

  I grant that this example looks a bit gobbledegookish,

 You think? :-)

  but normal
  usage would be much simpler.  The notation doesn't handle
  balanced trinary; however I opine that balanced trinary requires
  special notation.

 When the programmer needs to construct such values how about allowing
 him or her to specify something like

   (20.1 in base 3) times 2 to the power of 0xac

 Leaving out how to specify (20.1 in base 3) for now this could be

   (20.1 in base 3) * 2 ** 0xac

Using the suggestion from another post would convert this to

  0.(3:20.1) * 2 ** 0xac


 The compiler could convert this to a constant.

-- 
http://mail.python.org/mailman/listinfo/python-list

Numeric literals in other than base 10 - was Annoying octal notation

[James Harris]

On 22 Aug, 10:27, David 71da...@libero.it wrote:

... (snipped a discussion on languages and other systems interpreting
numbers with a leading zero as octal)

  Either hexadecimal should have been 0h or octal should
  have been 0t :=)


 I have seen the use of Q/q instead in order to make it clearer. I still
 prefer Smalltalk's 16rFF and 8r377.


 Two interesting options. In a project I have on I have also considered
 using 0q as indicating octal. I maybe saw it used once somewhere else
 but I have no idea where. 0t was a second choice and 0c third choice
 (the other letters of oct). 0o should NOT be used for obvious reasons.

 So you are saying that Smalltalk has base in decimalrnumber where
 r is presumably for radix? That's maybe best of all. It preserves the
 syntactic requirement of starting a number with a digit and seems to
 have greatest flexibility. Not sure how good it looks but it's
 certainly not bad.


    0xff  0x0e | 0b1101
    16rff  16r0e | 2r1101

  Hmm. Maybe a symbol would be better than a letter.

...

 Or Ada's16#FF#, 8#377#...

 I forget if DEC/VMS FORTRAN or Xerox Sigma FORTRAN used x'FF' or
'FF'x, and o'377' or '377'o

...


 What about 2_1011, 8_7621, 16_c26h or 2;1011, 8;7621, 16;c26h ?

They look good - which is important. The trouble (for me) is that I
want the notation for a new programming language and already use these
characters. I have underscore as an optional separator for groups of
digits - 123000 and 123_000 mean the same. The semicolon terminates a
statement. Based on your second idea, though, maybe a colon could be
used instead as in

  2:1011, 8:7621, 16:c26b

I don't (yet) use it as a range operator.

I could also use a hash sign as although I allow hash to begin
comments it cannot be preceded by anything other than whitespace so
these would be usable

  2#1011, 8#7621, 16#c26b

I have no idea why Ada which uses the # also apparently uses it to end
a number

  2#1011#, 8#7621#, 16#c26b#

Copying this post also to comp.lang.misc. Folks there may either be
interested in the discussion or have comments to add.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Annoying octal notation

[James Harris]

On 21 Aug, 20:48, Derek Martin c...@pizzashack.org wrote:

...

 James Harris wrote:
  It maybe made sense once but this relic of the past should have been
  consigned to the waste bin of history long ago.

 Sigh.  Nonsense.  I use octal notation *every day*, for two extremely
 prevalent purposes: file creation umask, and Unix file permissions
 (i.e. the chmod() function/command).  

You misunderstand. I was saying that taking a leading zero as
indicating octal is archaic. Octal itself is fine where appropriate.

The chmod command doesn't require a leading zero.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Annoying octal notation

[James Harris]

On 21 Aug, 22:18, MRAB pyt...@mrabarnett.plus.com wrote:
 Piet van Oostrum wrote:
  Derek Martin c...@pizzashack.org (DM) wrote:

  DM I fail to see how 0O012, or even 0o012 is more intelligible than 012.
  DM The latter reads like a typo, and the former is virtually
  DM indistinguishable from 00012, O0012, or many other combinations that
  DM someone might accidentally type (or intentionally type, having to do
  DM this in dozens of other programming languages).  

  You're right. Either hexadecimal should have been 0h or octal should
  have been 0t :=)

 I have seen the use of Q/q instead in order to make it clearer. I still
 prefer Smalltalk's 16rFF and 8r377.

Two interesting options. In a project I have on I have also considered
using 0q as indicating octal. I maybe saw it used once somewhere else
but I have no idea where. 0t was a second choice and 0c third choice
(the other letters of oct). 0o should NOT be used for obvious reasons.

So you are saying that Smalltalk has base in decimalrnumber where
r is presumably for radix? That's maybe best of all. It preserves the
syntactic requirement of starting a number with a digit and seems to
have greatest flexibility. Not sure how good it looks but it's
certainly not bad.

  0xff  0x0e | 0b1101
  16rff  16r0e | 2r1101

Hmm. Maybe a symbol would be better than a letter.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Identifying a class type - bad practice?

[James Harris]

On 18 Aug, 21:50, Chris Rebert c...@rebertia.com wrote:
 On Tue, Aug 18, 2009 at 10:09 AM, James

 Harrisjames.harri...@googlemail.com wrote:
  I am writing some code to form a tree of nodes of different types. The
  idea is to define one class per node type such as

  class node_type_1(node):
   specific properties by name including other node types
  class node_type_2(node):
   specific properties by name including other node types
  etc

  (Class node would hold any common properties).

  When walking the tree I need to know what type of node I'm dealing
  with so polymorphism isn't generally useful. The action to be taken
  depends on the node type.

 I'm sure it relates to the exact type of tree you're walking and the
 calculation you're doing on it, but what is the reason why your code,
 which in the abstract sounds like it will vaguely resemble this:

 def walk_tree(tree):
     if isinstance(tree, node_type_1):
         #code
         walk_tree(subtree)
     elif isinstance(tree, node_type_2):
         #code
         walk_tree(subtree)
     #etc...

 can't be written instead as:

 class node_type_1:
     def walk_tree(self):
         #code
         self.subtree.walk()

 class node_type_2:
     def walk_tree(self):
         #code
         self.subtree.walk()

 #etc

Interesting idea. This may be a better and a more OO solution than
what I had in mind. I'm not sure if I can use this but I'll see how it
fits in as the work progresses.

The tree is for a compiler. Initially the tree is for parsing of
source code. Then it will need to be processed and changed as further
compiler phases are written. I don't know yet whether it will be
easier to modify the tree or to create a new one for each phase.

So I guess whether I use the idea depends on the commonality of
operations.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Identifying a class type - bad practice?

[James Harris]

I am writing some code to form a tree of nodes of different types. The
idea is to define one class per node type such as

class node_type_1(node):
  specific properties by name including other node types
class node_type_2(node):
  specific properties by name including other node types
etc

(Class node would hold any common properties).

When walking the tree I need to know what type of node I'm dealing
with so polymorphism isn't generally useful. The action to be taken
depends on the node type. Two options appear to be useful: __class__
and isinstance. I know the latter will match the instance against any
superclass and the former will match one class only.

My question is: is this the Pythonic way to deal with such a tree? Is
there a better way? In C I would use structs where one field was a tag
indicating the kind of struct.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exhaustive Unit Testing

[James Harris]

On 27 Nov, 16:32, Emanuele D'Arrigo [EMAIL PROTECTED] wrote:
 On Nov 27, 5:00 am, Steven D'Aprano

 [EMAIL PROTECTED] wrote:
  Refactor until your code is simple enough to unit-test effectively, then
  unit-test effectively.

 Ok, I've taken this wise suggestion on board and of course I found
 immediately ways to improve the method. -However- this generates
 another issue. I can fragment the code of the original method into one
 public method and a few private support methods. But this doesn't
 reduce the complexity of the testing because the number and complexity
 of the possible path stays more or less the same. The solution to this
 would be to test the individual methods separately, but is the only
 way to test private methods in python to make them (temporarily) non
 private? I guess ultimately this would only require the removal of the
 appropriate double-underscores followed by method testing and then
 adding the double-underscores back in place. There is no cleaner
 way, is there?

Difficult to say without seeing the code. You could post it, perhaps.
On the other hand a general recommendation from Programming Pearls
(Jon Bentley) is to convert code to data structures. Maybe you could
convert some of the code to decision tables or similar.

James
--
http://mail.python.org/mailman/listinfo/python-list

Re: Getting fractional part from a float without using string operations

[James Harris]

On 20 Nov, 06:01, srinivasan srinivas [EMAIL PROTECTED] wrote:
 Yes it works for most of the cases.  But it doesn't for the following case:

  str(abs(int(1234567.89)-1234567.89))

 '0.88999898'

Well, that is 0.89 or about as near to it as the calculation can
represent. Like other numbers 0.89 evidently cannot be exactly
represented as a float in binary. The nearest number above it is

 0.89
0.89001


This is not a Python issue but a result of storing numbers in floating
binary form.

BTW, please post comments below existing ones rather than above them.
It is more familiar on Usenet and, as a consequence, easier to read.

James
--
http://mail.python.org/mailman/listinfo/python-list

Re: external program crashes when run through subprocess.popen on XP

[James Harris]

On 17 Nov, 19:15, ckkart [EMAIL PROTECTED] wrote:
 Hi,

 on XP when starting a certain external program (plain C calculation
 program which communicates via stdout/fs) from python 2.5 using
 subprocess.Popen the external program crashes. It does not if started
 directly from the XP command prompt. This is not a purely python
 problems since the crash occurs as well if started e.g. from a msys
 bash shell. The only things I find worth mentioning from the windows
 debugging message is that FileVer and ProdVer of msvrct.dll differ, I
 am no windows expert though.

 Any ideas?

If it helps there is some finished code at

  http://codewiki.wikispaces.com/tester.py

which used subprocess.Popen and works well under Windows and Linux.
Search for Popen to locate the specific call. Note that the code works
whether the shell argument is true or false.

James
--
http://mail.python.org/mailman/listinfo/python-list

Re: best python unit testing framwork

[James Harris]

On 15 Nov, 01:02, Brendan Miller [EMAIL PROTECTED] wrote:
 On Thu, Nov 13, 2008 at 3:54 AM, James Harris



 [EMAIL PROTECTED] wrote:
  On 11 Nov, 22:59, Brendan Miller [EMAIL PROTECTED] wrote:
  What would heavy python unit testers say is the best framework?

  I've seen a few mentions that maybe the built in unittest framework
  isn't that great. I've heard a couple of good things about py.test and
  nose. Are there other options? Is there any kind of concensus about
  the best, or at least how they stack up to each other?

  You don't mention what you want from testing so it's hard to say which
  is best as it depends on one's point of view.

  For example, I had a requirement to test more than just Python
  programs. I wanted to test code written in any language. None of the
  frameworks I found supported this so I wrote my own tester. It
  interacts with programs via file streams - principally stdin, stdout
  and stderr though others can be added as needed.

 I was speaking to unit tests (tests of individual classes and
 functions in isolation of the rest of the program) and a test driven
 development approach. I find those to be much more useful, and good
 for pinpointing bugs and guiding development. In contrast regression
 tests tend to be slow, and are the programmatic equivalent of kicking
 the tires and seeing if they fall off.

I agree with your points but would add

1. We should write smaller modules than we generally do. It promotes
clear thinking and encourages the production of reusable code. In
Python it is easy to add a __main__ section to interface to the
classes or functions defined in the code. Such code can then be tested
as mentioned. The functions or classes can still be imported into and
used in other code as needed.

2. The example given wasn't intended to be the only way forward but to
highlight that what makes a test package good depends on your
requirements (which I don't think you mentioned). In my case the
desire to test arbitrary languages was a killer blow to various Python-
only frameworks which I wasn't too impressed with anyway. (At least I
saw none that looked worth investing the time to learn.) Similarly an
option for you is to write your own tester. It may be a better fit
than someone else's package and not too hard to do. On the other hand
a pre-written package may be better - depending on what you are
looking for.


 I've also seen regression tests lead to a sweep the bugs under the
 rug mentality where developers will code to prevent errors from
 crashing the regression test, e.g. by catching and swallowing all
 exceptions without fixing the underlying problem. It's easy to fool
 regression tests since what it does works at such a high level that
 most aspects of program correctness can't be directly checked. This is
 very frustrating to me because it actually leads to lower code
 quality.


If you have dodgy developers then much is wrong, not just testing. If
the team works well, IMHO, production of test cases should be a
welcome part of the development process. When is it better to find
bugs: before or after shipping to a client? Test cases can be put
together in various ways. For example, if one has the code and chooses
to work this way white-box testing can lead to generation of test
cases. On the other hand if an organisation prefers it can produce
test cases from the spec - or they can do both. It's their choice but
in any case the developers and testers should work together.


  One nice by-product is that test code does not bloat-out the original
  source which remains unchanged.

 That's the main reason most people don't write unit tests. It forces
 them to properly decouple their code so that parts can be used
 independently of one another. Adding such things to messy ball of mud
 code after the fact is an enourmous pain in the butt. Thankfully,
 since python is duck typed and doesn't require lots of boilerplate
 writing interfaces and abstract factories (since the class object
 itself acts as an abstract factory), writing properly decoupled code
 in the first place doesn't look nearly as hard as in C++ or Java.

You could still import one Python module into another to run test
cases such as

-
import test_support
import program_to_be_tested

tests of program_to_be_tested using test_support
-

where the tests can interact directly with the components of the code
under test. The test support code can handle things such as collating
results and reporting as well as providing code to directly support
certain tests, handle exceptions etc.

I wouldn't be surprised if good testing code were in many cases to be
significantly longer than the code being tested. Keeping the tests in
a separate file keeps the code being tested shorter which can only
improve clarity. It also allows either the developer or someone else
to write the tests. Putting code and testing code in the same file
almost requires one person to write both parts.

A final thought: if you

Re: best python unit testing framwork

[James Harris]

On 11 Nov, 22:59, Brendan Miller [EMAIL PROTECTED] wrote:
 What would heavy python unit testers say is the best framework?

 I've seen a few mentions that maybe the built in unittest framework
 isn't that great. I've heard a couple of good things about py.test and
 nose. Are there other options? Is there any kind of concensus about
 the best, or at least how they stack up to each other?

You don't mention what you want from testing so it's hard to say which
is best as it depends on one's point of view.

For example, I had a requirement to test more than just Python
programs. I wanted to test code written in any language. None of the
frameworks I found supported this so I wrote my own tester. It
interacts with programs via file streams - principally stdin, stdout
and stderr though others can be added as needed.

One nice by-product is that test code does not bloat-out the original
source which remains unchanged. And test cases can be written before,
alongside or after the code as desired.

So, YMMV.
--
James
--
http://mail.python.org/mailman/listinfo/python-list

Re: What is the best Python GUI API?

[James Harris]

On 13 Nov, 18:59, Stef Mientki [EMAIL PROTECTED] wrote:
 Abah Joseph wrote:
   What is the best Python GUI API? I am planning to start my first GUI
  application and I need something easy and cross platform. Qt
  applications look beautiful but I hate the license. What do you advice?

 I agree about the Qt-license,
 and I'm now a happy wxPython user.

I too have had good results with wxwidgets when developing a GUI. The
cross-platform native look and feel was a major benefit from my point
of view allowing screens to look native under different OSs with no
code changes.

--
James
--
http://mail.python.org/mailman/listinfo/python-list

Re: etymology of list comprehension?

[James Harris]

On 6 Nov, 22:13, [EMAIL PROTECTED] wrote:
 Chris Rebert [EMAIL PROTECTED] wrote:
  the term
  comprehension for the concept was first used in the NPL programming
  language (Wikipedia again).

 Ah, thanks... and does comprehension have any special computer
 science meaning?

Good question. It seems that comprehension in this case relates to
comprehensiveness - i.e. a note of how complete a set is. From the
list at

   http://www.onelook.com/?w=comprehensionls=a

check out in particular Merriam-Webster meanings 2a and 2b, and
Wikipedia for comprehension (logic).

--
HTH,
James
--
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing a file with iterators

[James Harris]

On 17 Oct, 16:42, Luis Zarrabeitia [EMAIL PROTECTED] wrote:
 I need to parse a file, text file. The format is something like that:

 TYPE1 metadata
 data line 1
 data line 2
 ...
 data line N
 TYPE2 metadata
 data line 1
 ...
 TYPE3 metadata
 ...

 And so on. The type and metadata determine how to parse the following data
 lines. When the parser fails to parse one of the lines, the next parser is
 chosen (or if there is no 'TYPE metadata' line there, an exception is thrown).

 This doesn't work:

 ===
 for line in input:
 parser = parser_from_string(line)
 parser(input)
 ===

 because when the parser iterates over the input, it can't know that it 
 finished
 processing the section until it reads the next TYPE line (actually, until it
 reads the first line that it cannot parse, which if everything went well, 
 should
 be the 'TYPE'), but once it reads it, it is no longer available to the outer
 loop. I wouldn't like to leak the internals of the parsers to the outside.

 What could I do?
 (to the curious: the format is a dialect of the E00 used in GIS)

The main issue seems to be that you need to keep the 'current' line
data when a parser has decided it doesn't understand it so it can
still be used to select the next parser. The for loop in your example
uses the next() method which only returns the next and never the
current line. There are two easy options though:

1. Wrap the input file with your own object.
2. Use the linecache module and maintain a line number.

  http://blog.doughellmann.com/2007/04/pymotw-linecache.html

--
HTH,
James
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python program as daemon?

[James Harris]

On 25 Jul, 20:05, sturlamolden [EMAIL PROTECTED] wrote:
 On Jul 25, 8:37 pm, Johny [EMAIL PROTECTED] wrote:

  Is it possible to run a Python program as daemon?
  Thanks

 Here is an example on how to run a Python script as a Unix daemon:

 http://svn.plone.org/svn/collective/bda.daemon/trunk/bda/daemon/daemo...

 Basically it forks twice and redirects open file descriptors to /dev/
 null.

 On Windows, 'daemons' are called services. You can write Windows
 services in Python using the Pywin32 extension. See Mark Hammond's
 book for an explanation.

Also, on Windows, you can get good results using srvany

  http://support.microsoft.com/kb/137890

I've set this up to run Pyhton for a couple of scripts.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Running commands on cisco routers using python

[James Harris]

On 19 May, 16:18, SPJ [EMAIL PROTECTED] wrote:
 Is it possible to run specific commands on cisco router using Python?
 I have to run command show access-list on few hundred cisco routers and get 
 the dump into a file. Please let me know if it is feasible and the best way 
 to achieve this.

Can you access the routers with telnet or do you need to use ssh? I've
written loads of Python to access Cisco routers using a wrapper around
the standard telnetlib. Telnetlib supplies an excellent expect
function which makes this work.

It is telnet, though. If I had to use ssh I would probably add it
under the same wrapper so that higher level code could work unchanged.
There may be a way to route the telnet interractions through openssh
or similar externally but I haven't tried that.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Portable general timestamp format, not 2038-limited

[James Harris]

On 5 Jul, 02:53, greg [EMAIL PROTECTED] wrote:
 James Harris wrote:
  With that the time would range to +/- 9000
  quintillion years (18 digits)

 Use the Big Bang as the epoch, and you won't have
 to worry about negative timestamps.

Good idea if only they didn't keep shifting the femtosecond on which
it happened.. :-)


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Portable general timestamp format, not 2038-limited

[James Harris]

On 5 Jul, 08:46, Dennis Lee Bieber [EMAIL PROTECTED] wrote:
 On Wed, 04 Jul 2007 22:12:46 -0400, Roy Smith [EMAIL PROTECTED] declaimed
 the following in comp.lang.python:

  Astronomers use Julian Date (http://en.wikipedia.org/wiki/Julian_date) for
  calculations like this.  It's a widely used format and highly portable.  
  I'm sure there are libraries to deal with it in all the languages you
  mention (and more).  Ask on sci.astro for more information.

 playing devils advocate But do you also need to account for
 Besselian or Julian centuries (Astronomy used to use B1900 as a
 computational epoch, but now uses J2000. A Julian century is 36525 days,
 Besselian century was 36524.22 days.

Whew! It was for reasons such as this that I suggested treating a day
(i.e. a /nominal/ 24-hour period) as the primary unit. The Gregorian
switch to Julian, for example, missed out a bunch of days to adjust
the calendars of Christendom but they had to be whole numbers of days.
In terms of real people (about the level I need) once a dividing line
has been chosen between one day and the next it becomes a reference
point.

Incidentally I have chosen to store /average/ values in the
application so if the sample period is 10 seconds and the count
increases by 45 I will store 4.5. This is plottable directly and I
could even allow an 11 second sample when a leap second is added (if I
needed that detail).

Is your Julian century a bit long, on average, 2000, 2400, 2800 etc
having 28 days in Feb?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Portable general timestamp format, not 2038-limited

[James Harris]

On 1 Jul, 15:11, Peter J. Holzer [EMAIL PROTECTED] wrote:
...
 Stick to unix timestamps but store them as a double precision floating
 point number. The 53 bit mantissa gives you currently a resolution of
 about 200 ns, slowly deteriorating (you will hit ms resolution in about
 280,000 years, if I haven't miscalculated). Any language and database
 should be able to handle double-precision FP numbers, so that's as
 portable as it gets and conversion from/to system time should be
 trivial.

 If you need to represent milliseconds exactly, you can just multiply the
 timestamp with 1000 (and get java timestamps).

Interesting option. I think my choice is between separate day and sub-
day 32-bit unsigned integers, text, and this 64-bit float option.

I'm not clear, though. Did you mean to store double precision numbers
where the seconds are the units (I assume this) or where the days are
the units? And what do you think of the other option?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Portable general timestamp format, not 2038-limited

[James Harris]

On 3 Jul, 06:12, Scott David Daniels [EMAIL PROTECTED] wrote:
...
 Inspired format:
  Days since a some standard date (the TAI date may be a good such
 date) expressed as fixed point 64-bit (32-bit day part, 32-bit
 day-fraction part) or floating point (using Intel's double-precision,
 for example, gives you 26 bits each for day and day-fraction, though
 the binary point moves for particular stamps).

This is close to or the same as my original suggestion. The break
between days and sub-days seems to make more sense than breaking the
fractional part elsewhere. It also gives a convenient point to hang
datestamps on rather than just timestamps.

FWIW I wonder if a 64-bit version of the above would cope with all
practical time needs. With that the time would range to +/- 9000
quintillion years (18 digits) and there would be over 200 trillion
ticks per second or 200 in a picosecond making, I think, each tick 5
femtoseconds.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Portable general timestamp format, not 2038-limited

[James Harris]

On 4 Jul, 22:18, Peter J. Holzer [EMAIL PROTECTED] wrote:
...
 But it really doesn't matter much. If you ignore leap seconds, using
 days instead of seconds is just a constant factor (in fact, the unix
 timestamp ignores leap seconds, too, so it's always a constant factor).
 You can't represent a second exactly if the unit is one day (1/86400 is
 not a multiple of a power of two), but that probably doesn't matter.

Sure. However, the proposal was to define ticks as 25 microseconds.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Portable general timestamp format, not 2038-limited

[James Harris]

On 25 Jun, 02:14, [EMAIL PROTECTED] (Robert Maas, see http://tinyurl.com/uh3t)
wrote:
  From:  James Harris [EMAIL PROTECTED]
  I have a requirement to store timestamps in a database. ...
  1) subsecond resolution - milliseconds or, preferably, more detailed

 How do you plan to deal with leap seconds?
 - Stick to astronomical time, which is absolutely consistent but
which drifts from legal time?
 - Stick to legal time (UTC), which stalls by one second from time
to time, causing time-difference calculations to be incorrect by
varying numbers of seconds?
 Only after you make *that* crucial decision, will it be reasonable
 to consider milliseconds or other sub-second resolution.

Not a problem for me. I will be taking samples and storing either
point samples or averages depending on the value being sampled. Pseudo-
GMT will be good enough. Astronimical time will be no good as the time
is to relate to the time of day the samples were taken. I think I can
just use the time as returned by the language I am using (which
presumably will get it from a C system call or similar). If one sample
over midnight when a leap second adjustment happens turns out to be
slightly incorrect it won't skew the results significantly. I could
sanity check the time, though. Hmmm.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Portable general timestamp format, not 2038-limited

[James Harris]

On 22 Jun, 23:49, Roger Miller [EMAIL PROTECTED] wrote:
...
 My rule of thumb in situations like this is When in doubt store it as
 text.  The one format I am pretty sure we will still be able to deal
 with in 2039.

Interesting. I hadn't thought about using text. It would add to the
storage a bit as each record is otherwise quite short. But this sounds
like a good option and may help - at least while debugging - to see
the raw date and time as digits. I will consider using this, perhaps
as mmddhhmmssttt.

-- 
http://mail.python.org/mailman/listinfo/python-list

Portable general timestamp format, not 2038-limited

[James Harris]

I have a requirement to store timestamps in a database. Simple enough
you might think but finding a suitably general format is not easy. The
specifics are

1) subsecond resolution - milliseconds or, preferably, more detailed
2) not bounded by Unix timestamp 2038 limit
3) readable in Java
4) writable portably in Perl which seems to mean that 64-bit values
are out
5) readable and writable in Python
6) storable in a free database - Postgresql/MySQL

The formats provided by the two database systems are such as 8-byte or
12-byte values which, even if I could get Perl to work with I guess it
would be messy. Keeping to 32-bit values should give me portability
and be easy enough to work with without obscuring the program logic.
Since 32 bits of microseconds is less than 50 days I have to store two
32-bit values. How to split them? The option I favour at the moment is
to split days and parts of days like this:

a) store, as a 32-bit number, days since a virtual year zero (there is
no year zero in common era time http://en.wikipedia.org/wiki/
Common_Era). This allows over five million years plus and minus.
Still not completely general, I know.
b) store parts of days as another 32-bit value. Its range would have
to go to 86401 seconds - the number of seconds in a leap day. This
means each 'tick' would be around 21 microseconds. For regularity I
could make the ticks 25 microseconds so there would be 40,000 in a
second and 3,456,000,000 in a day; and, finally, the counter could
tick about 5 hours into the next day if not caught.

Any thoughts on a better way to do this? (Please reply-all. Thanks).

--
James

-- 
http://mail.python.org/mailman/listinfo/python-list

Python music interfaces

[James Harris]

Hi,

I am wanting to develop some software that

a) Reads existing files of music of a few well known types
b) Displays the music in traditional notation - i.e. on a stave
c) Displays the same music in my own notation that I am playing with
d) Allows both to be printed

I guess I can use Python with Tk for the display part but there seems to 
be a plethora/cornucopia of Python music interfaces. (For example, 
http://wiki.python.org/moin/PythonInMusic.) I'd appreciate if you 
could recommend which would be good for me to look at in order to do the 
above.

--
TIA,
James


-- 
http://mail.python.org/mailman/listinfo/python-list