Re: low-level csv
The few times I looked at CSV module it never looked useful to me. I seem to use shlex for everything. For example: def csv_split (s): lex = shlex.shlex(s, posix=True) lex.whitespace_split = True lex.whitespace = ',' return list(lex) will split on commas while honoring quoted values (and stripping out the quotes) Dave On 2019/11/17 08:18, Antoon Pardon wrote: This is python 2.6->2.7 and 3.5->3.7 I need to convert a string that is a csv line to a list and vice versa. I thought to find functions doing this in the csv module but that doesn't seem to be the case. I can of course write special classes that would allow for this using a csv.reader and csv.writer but that would be some convoluted code. Anyone some suggestions? -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
Rudy,
OK I'm old enough cancer to know what X10 is. :-) I wouldn't think this
is too difficult to do as you're only sending. For this application IMO
there is too much going on inside python that can interrupt the cycle of
modulation. (Intra and Extra python; garbage collection, other
processes, etc)
I would drop back to a simple program in C that you execute from python
to do the actual sending putting your 'message' on the command line to
that program (or use a tmpfile). I think your solution will be to
execute this program from python.
Let me stress, my advise is doing this *reliably*. In that regard you
might need look at locking your C process to 1 CPU and giving it highest
priority. (man nice)
Once you have this working reliably, you could then look to convert it
to a python c-module to more tightly integrate it.
Dave
On 2019/11/14 15:00, R.Wieser wrote:
Dave,
Can you expand on what you are trying to accomplish with this?
There is a small 433 MHz rf transmitter connected to the pin, and when I
send the right pattern a wireless wall-wart will respond and switch a lamp
on or off. Its just ment as an example of a real-world application of
Python, nothing serious.
Ofcourse, by doing so I'm learning about how to use python (and the Pi)
myself too. :-)
It seems a small C program or library you interface python too is a better
solution.
:-) I already wrote that program in C{something} (I'm not sure what my Pi
offers by default) a while ago, but "ported" it to python. For the above
mentioned "example" reason.
... Which is also why I didn't even try to just shell to that program, or
try to interface with a C{something} library.
Though doing such interfacing is in the pipeline (I've already
found-and-stored some documentation about it).
Regards,
Rudy Wieser
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
Can you expand on what you are trying to accomplish with this? It seems a small C program or library you interface python too is a better solution. With that said, as others mentioned you might need a real time OS or micro controller if this needs to be dead on timing. Dave On 2019/11/13 09:02, R.Wieser wrote: Hello all, I'm writing some code to toggle a pin on a Raspberry Pi, and would like to have that happen at (multiples of) 300 uSec increments. I tried time.sleep(), but that one disregards the time after the last one ends and the new one is started. In other words, all the time spend in code (or being interrupted by another thread!) between the end of the previous and the start of the current sleep throws the whole repetitive timing off. So, I'm looking for a method that will allow me to wait for a "last time plus increment". Is there one with the properties of sleep() (not just burning processor cycles way, blocking all threads), but referencing a previous time. Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Thesaurus and ThesaurusCfg - recursive mapping and cfg file data types
This announcement is for a pre-release that I would like people to
comment on structure, naming, etc. (Code review maybe not yet. :-)
Before you say "It's all been done before." I suggest you take a closer
look and I think you may conclude that what I've revised over 7 years is
now interesting.
---
Thesaurus is a mapping data type with key recursion and attribute
aliasing. It is a subclass of dict() and compatible as a dictionary
replacement baring where key path recursion may take place.
ThesaurusExtended is a subclass of Thesaurus providing additional
usability methods.
ThesaurusCfg is a subclass of ThesaurusExtended providing a
configuration file parser and per key data coercion methods.
The Thesaurus family works with Python 2.6+ to 3.8+.
A simple example of ThesaurusCfg
--
cfgs = '''
prog.version(static_int)= 123
opt.verbose (str_to_bool) = yes
hi = Hello
'''
from thesauruscfg import thescfg
cfg = thescfg()
>>> cfg.parse(cfgs)
{'prog': {'version': 123}, 'opt': {'verbose': True}, 'hi': 'Hello'}
>>> cfg.opt.verbose
True
import json
>>> print(json.dumps(cfg, indent=4, separators=(',', ': ')))
{
"prog": {
"version": 123
},
"opt": {
"verbose": true
},
"hi": "Hello"
}
browse:
https://git.cinege.com/thesaurus/
or
git clone https://git.cinege.com/thesaurus/
---
Dave
--
https://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursively when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in python. I've re-written some
exiting programs using Thesaurus, and often realized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
#!/usr/bin/env python
thesaurus.py 2012-09-13
Copyright (c) 2012 Dave Cinege
Licence: python, Copyright notice may not be altered.
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursivly when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in python. I've re-written some
exiting programs using Thesaurus, and often relized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
By example:
---
#!/usr/bin/env python
import thesaurus
thes = thesaurus.Thesaurus
# I like to create a master global object called 'g'.
# No more 'global' statements for me.
g = thes()
g.prog = thes()
g['prog']['VERSION'] = '1.0' # I can do this like a normal nested dictionary
g.prog.VERSION = '1.0' # But isn't this so much cleaner?
g.prog.NAME = 'Thesaurus'
class TestClass:
no = 'Class'
way = 'this'
def main ():
L = thes() # Some local varibles
L.tc = TestClass() # We can also recurse to class attributes
L.l = list() # And recurse to indices too!
L.l.append('Some')
L.l.append('objects')
g.L = L # Easily make these locals global
# Here's the real magic. Creating output without a fight.
print '''
When programing python using %(prog.NAME)s, it is very
easy to access your %(L.l.1)s.
%(L.l.0)s people might say %(prog.NAME)s has no %(L.tc.no)s.
'''.replace('\t','')[1:-1] % g
del g.L # Clean locals up out of global space
# If I was using a storage class, I must use hasattr() or an ugly eval.
# But we can still use a str for the key name and 'in' like on any
# regular dictionary.
if 'VERSION' in g.prog:
print 'But I challenge them to write code %(tc.way)s clean without it!' % L
if __name__ == '__main__':
main()
---
__VERSION__ = 20120913
class Thesaurus (dict):
def __getattr__(self, name):
return self.__getitem__(name)
def __setattr__(self, name, value):
return dict.__setitem__(self, name, value)
def __delattr__(self, name):
return dict.__delitem__(self, name)
def __getitem__(self, key):
if '.' not in key:
return dict.__getitem__(self, key)
l = key.split('.')
if isinstance(l[0], (dict, Thesaurus)):
a = self.data
else:
a = self
for i in range(len(l)): # Walk keys recursivly
try:
a = a[l[i]] # Attempt key
except:
try:
a = getattr(a, l[i]) # Attempt attr
except:
try:
a = a[int(l[i])] # Attempt indice
except:
raise KeyError(key + ' [%s]' % i)
return a
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wednesday 12 December 2012 05:25:11 D'Arcy J.M. Cain wrote: As a 16yr OSS vet I know that for every 1 person that that actually creates something there will always be 2000 people to bitch about it. My skin isn't thin, I just don't give a shit to listen to anyone one that doesn't get it. The point to Thesaurus for those that want to pay attention: The concept in these ~25 lines of code have changed the way I program Python and reduced existing functionally identical code up to 30%...and I like the code better. If that doesn't peak your interest, then move on...nothing here for you to see. If you feel it needs to be expanded/corrected, do it and share it. If you can do it better, re-implement it. That's why I sent it to the mailing list. On Wed, 12 Dec 2012 17:44:24 +1100 Chris Angelico ros...@gmail.com wrote: On Wed, Dec 12, 2012 at 5:34 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: and you consider this not productive, not worth my time. Code review with you must be *all* sorts of fun. He never asked for code review :) I think by posting it he sort of did. He should probably grow a thicker skin before he does so again though. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tuesday 11 December 2012 01:41:38 Ian Kelly wrote: I have a few critiques on the code. First, you might want to use __getattribute__ instead of __getattr__. Otherwise you'll end up File /home/dcinege/src/thesaurus/thesaurus.py, line 84, in __getattribute__ return self.__getitem__(name) RuntimeError: maximum recursion depth exceeded while calling a Python object This takes me into the same hell as when I was trying to implement this as a class. Someone else would have to take the time to do this. __getattr__ doing what I expect it to do, for what i do. Dave -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wednesday 12 December 2012 15:42:36 Ian Kelly wrote:
def __getattribute__(self, name):
if name.startswith('__') and name.endswith('__'):
return super(Thesaurus, self).__getattribute__(name)
return self.__getitem__(name)
Ian,
Tested, and works as you advertised.
Isn't super() depreciated? I've replaced it with this:
-return super(Thesaurus, self).__getattribute__(name)
+return dict.__getattribute__(self, name)
Aside from a more palatable result in the python shell for otherwise bad
code...does this get me anything else? Is it really worth the performance hit
of 2 string comparisons for every getattribute call?
I also just reviewed all your original comments again:
In general: The recursion code is nothing special to me and I finally settled
on the nested try's as something simple that 'just works good for now'.
I think the concept of what I'm doing here is the big idea.
Should the idea of implementing what Thesaurus does in mainline python ever
happen, those 10 lines of code will likely spark a 3 month jihad about how to
properly do in python which up until now hasn't been something you do in
python.
To me for i in range(len(l)) seems like simpler, faster, tighter code for this
now. It's duly noted that enumerate() is more python and I'm an old fart that
still thinks too much in state machine. I've add except Exception per your
advise.
What I'd really like to hear is that someone reading was curious enough to
convert some existing code and their joy or displeasure with the experience.
Dave
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Wednesday 12 December 2012 20:14:08 Chris Angelico wrote: Ok enough already, I'll use the frigging thing! Be happy I'm at least still not coding for python 1.5. To add to this: Using enumerate gives the possibility (don't know if any current interpreter takes advantage or not, but a future one certainly could) that the enumerate() call could be optimized out. Yes, it's theoretically possible that someone could redefine enumerate, which would be broken by such an optimization. But all it'd take is some kind of directive-based optimization and voila, safe performance improvements. -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes - v20121212
Version 20121212
#!/usr/bin/env python
thesaurus.py 2012-12-12
Copyright (c) 2012 Dave Cinege
Licence: PSF Licence, Copyright notice may not be altered.
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursively when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in Python. I've re-written some
exiting programs using Thesaurus, and often realized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
By example:
---
#!/usr/bin/env python
import thesaurus
thes = thesaurus.Thesaurus
# I like to create a master global object called 'g'.
# No more 'global' statements for me.
g = thes()
g.prog = thes()
g['prog']['VERSION'] = '1.0' # I can do this like a normal nested dictionary
g.prog.VERSION = '1.0' # But isn't this so much cleaner?
g.prog.NAME = 'Thesaurus'
class TestClass:
no = 'Class'
way = 'this'
def main ():
L = thes() # Some local varibles
L.tc = TestClass() # We can also recurse to class attributes
L.l = list() # And recurse to indices too!
L.l.append('Some')
L.l.append('objects')
g.L = L # Easily make these locals global
# Here's the real magic. Creating output without a fight.
print '''
When programing python using %(prog.NAME)s, it is very
easy to access your %(L.l.1)s.
%(L.l.0)s people might say %(prog.NAME)s has no %(L.tc.no)s.
'''.replace('\t','')[1:-1] % g
del g.L # Clean locals up out of global space
# If I was using a storage class, I must use hasattr() or an ugly eval.
# But we can still use a str for the key name and 'in' like on any
# regular dictionary.
if 'VERSION' in g.prog:
print 'But I challenge them to write code %(tc.way)s clean without it!' % L
if __name__ == '__main__':
main()
---
__VERSION__ = 20121212
class Thesaurus (dict):
def __getattribute__(self, name):
if name.startswith('__') and name.endswith('__'):
return dict.__getattribute__(self, name)
return self.__getitem__(name)
def __setattr__(self, name, value):
return dict.__setitem__(self, name, value)
def __delattr__(self, name):
return dict.__delitem__(self, name)
def __getitem__(self, key):
if '.' not in key:
return dict.__getitem__(self, key)
a = self
l = key.split('.')
for i,v in enumerate(l): # Walk keys recursivly
try:
a = a[v] # Attempt key
except Exception:
try:
a = getattr(a, v) # Attempt attr
except Exception:
try:
a = a[int(v)] # Attempt indice
except Exception:
raise KeyError('%(v)s (%(key)s[%(i)s])' % locals() )
return a
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Monday 10 December 2012 23:08:24 Jason Friedman wrote: 2) Posting your code at ActiveState.com. If someone wants to please do. I'm back to being completely overloaded with normal work now. The v20121212 release based on the last few days comments is as far as I will go with this now. Dave -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tuesday 11 December 2012 01:41:38 Ian Kelly wrote: running into bugs like this: thes = Thesaurus() thes.update = 'now' thes.update built-in method update of Thesaurus object at 0x01DB30C8 I've noticed this but it's mostly pointless, as meaningful code does work. (Also you stepped on the existing 'update()' dictionary method.) import thesaurus t = thesaurus.Thesaurus() t.s = 'now' t.s + ' this' 'now this' Works perfect. However I'll take a look at __getattribute__ as from what you say it's more proper. Second, in __getitem__ you start a loop with for i in range(len(l)):, and then you use i as an index into l several times. It would be cleaner and more Pythonic to do for i, part in enumerate(l):, and then you can replace every occurrence of l[i] My python is still 'old school' due to being stuck on old versions for in production embedded system python applications. It's not clear to me what the isinstance call here is meant to be testing for. It's used to determine if it's the root instance of the recursive string because self.data, not self must be used to access that. Can you offer a better way? The prior statements require key to be a string. If key is a string, then by construction l[0] is also a string. So it seems to me that the isinstance check here will always be False. OK, try and remove it and then get back to me. :-) In any case, the key splitting here seems to be done primarily to support the use of formatting placeholders like %(L.l.1)s in the examples. I want to point out that this use case is already well supported (I might even say better supported since it cleanly distinguishes index elements from attributes with syntax) by the Thesaurus recursion works with modules like Template, in addition to allowing easy hierarchical organization of (global) data. It's not about string formatting. It's about organizing data and recursive retrieval of that data. Lastly, you have several bare except clauses in the code. Bare Not going to get into religion. There is no 'correct' way to do this code at all because it's not established normal python. I left it simple and it suits my needs. I've converted several production commercial utilities and applications to use Thesaurus, and have seen nothing but benefit. If anyone needs more to this, have at it, I'm busy Dave -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tuesday 11 December 2012 03:12:19 Steven D'Aprano wrote:
Is this intended as a ready-for-production class?
For me, yes. In production code.
py d = Thesaurus()
py d['spam'] = {}
Maybe because spam is type dict instead of type thes???
import thesaurus
thes = thesaurus.Thesaurus
t = thes()
t.spam = thes()
t.spam.ham = 'cheese'
print t.spam.ham
cheese
print t['spam'].ham
cheese
print t['spam']['ham']
cheese
'%(spam.ham)s' % t
'cheese'
Works for me!
Remainder of your post, not productive, not worth my time.
Dave
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tuesday 11 December 2012 16:53:12 Ian Kelly wrote:
Just out of curiosity, how old are we talking? enumerate was added in
Python 2.3, which is nearly 10 years old. Prior to 2.2 I don't think
it was even possible to subclass dict, which would make your Thesaurus
implementation unusable, so are these systems running Python 2.2?
I'm finally beyond 2.2 and getting rid of 2.4 soon. Just started using 2.6 5
months ago.
Thesaurus initially came about from me doing this:
class Global:
pass
g = Global()
As a way to organize/consolidate global vars and eliminate the global
statement.
After a brain fart one day I expanded this to some simple recursion and felt I
was onto something as my entire life changed with how easy it now was to build
output strings.
As you noted it was not possible to subclass dict, so I first tried with a
class, and you run into recursion hell with __setatrib__ to which i think
there is no fix.
I then made a UserDict version. Then when I moved mostly to 2.6 I could do a
proper dict subclass.
So I believe you're actually correct here
if isinstance(l[0], (dict, Thesaurus)):
a = self.data
Looks like an artifact from my UserDict version and was needed. :-(
class UserDict:
def __init__(self, dict=None, **kwargs):
self.data = {}
Thanks for this. You'll see from the version number I wrote this 3 months ago
so it's not 100% fresh in my mind. I'm releasing it now because a python coder
I contracted to pick up some slack for me saw this and went ape at how much he
liked it...and that prompted me to finally get it out into the wild.
As for in depth discussion and enhancement to this, I lack the time.
Dave
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
On Tuesday 11 December 2012 17:39:12 Dave Cinege wrote: My memory is getting jogged more why did some things: raise KeyError(key + ' [%s]' % i) I did this to specificly give you the indice that failed recursion but provide the entire key name as it was provided to __getitem__ So if: g.cfg.host.cpu fails recursion on 'host' you will see: g.cfg.host.cpu [2] I know my code sent g.cfg.host.cpu. I know host failed. if It was g.cfg.host.host, I'd know which host failed. Makes sense to me. Works for me. Sure there are other ways to do it. -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: Thesaurus - a recursive dictionary subclass using attributes
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursively when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in python. I've re-written some
exiting programs using Thesaurus, and often realized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
#!/usr/bin/env python
thesaurus.py 2012-09-13
Copyright (c) 2012 Dave Cinege
Licence: python, Copyright notice may not be altered.
Thesaurus: A different way to call a dictionary.
Thesaurus is a new a dictionary subclass which allows calling keys as
if they are class attributes and will search through nested objects
recursivly when __getitem__ is called.
You will notice that the code is disgusting simple. However I have found that
this has completely changed the way I program in python. I've re-written some
exiting programs using Thesaurus, and often relized 15-30% code reduction.
Additionally I find the new code much easier to read.
If you find yourself programing with nested dictionaries often, fighting to
generate output or command lines for external programs, or wish you had
a dictionary that could act (sort of) like a class, Thesaurus may be for you.
By example:
---
#!/usr/bin/env python
import thesaurus
thes = thesaurus.Thesaurus
# I like to create a master global object called 'g'.
# No more 'global' statements for me.
g = thes()
g.prog = thes()
g['prog']['VERSION'] = '1.0' # I can do this like a normal nested dictionary
g.prog.VERSION = '1.0' # But isn't this so much cleaner?
g.prog.NAME = 'Thesaurus'
class TestClass:
no = 'Class'
way = 'this'
def main ():
L = thes() # Some local varibles
L.tc = TestClass() # We can also recurse to class attributes
L.l = list() # And recurse to indices too!
L.l.append('Some')
L.l.append('objects')
g.L = L # Easily make these locals global
# Here's the real magic. Creating output without a fight.
print '''
When programing python using %(prog.NAME)s, it is very
easy to access your %(L.l.1)s.
%(L.l.0)s people might say %(prog.NAME)s has no %(L.tc.no)s.
'''.replace('\t','')[1:-1] % g
del g.L # Clean locals up out of global space
# If I was using a storage class, I must use hasattr() or an ugly eval.
# But we can still use a str for the key name and 'in' like on any
# regular dictionary.
if 'VERSION' in g.prog:
print 'But I challenge them to write code %(tc.way)s clean without it!' % L
if __name__ == '__main__':
main()
---
__VERSION__ = 20120913
class Thesaurus (dict):
def __getattr__(self, name):
return self.__getitem__(name)
def __setattr__(self, name, value):
return dict.__setitem__(self, name, value)
def __delattr__(self, name):
return dict.__delitem__(self, name)
def __getitem__(self, key):
if '.' not in key:
return dict.__getitem__(self, key)
l = key.split('.')
if isinstance(l[0], (dict, Thesaurus)):
a = self.data
else:
a = self
for i in range(len(l)): # Walk keys recursivly
try:
a = a[l[i]] # Attempt key
except:
try:
a = getattr(a, l[i]) # Attempt attr
except:
try:
a = a[int(l[i])] # Attempt indice
except:
raise KeyError(key + ' [%s]' % i)
return a
--
http://mail.python.org/mailman/listinfo/python-list
