ANNOUNCE: Thesaurus and ThesaurusCfg - recursive mapping and cfg file data types
This announcement is for a pre-release that I would like people to
comment on structure, naming, etc. (Code review maybe not yet. :-)
Before you say "It's all been done before." I suggest you take a closer
look and I think you may conclude that what I've revised over 7 years is
now interesting.
---
Thesaurus is a mapping data type with key recursion and attribute
aliasing. It is a subclass of dict() and compatible as a dictionary
replacement baring where key path recursion may take place.
ThesaurusExtended is a subclass of Thesaurus providing additional
usability methods.
ThesaurusCfg is a subclass of ThesaurusExtended providing a
configuration file parser and per key data coercion methods.
The Thesaurus family works with Python 2.6+ to 3.8+.
A simple example of ThesaurusCfg
--
cfgs = '''
prog.version(static_int)= 123
opt.verbose (str_to_bool) = yes
hi = Hello
'''
from thesauruscfg import thescfg
cfg = thescfg()
>>> cfg.parse(cfgs)
{'prog': {'version': 123}, 'opt': {'verbose': True}, 'hi': 'Hello'}
>>> cfg.opt.verbose
True
import json
>>> print(json.dumps(cfg, indent=4, separators=(',', ': ')))
{
"prog": {
"version": 123
},
"opt": {
"verbose": true
},
"hi": "Hello"
}
browse:
https://git.cinege.com/thesaurus/
or
git clone https://git.cinege.com/thesaurus/
---
Dave
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
On 13.11.2019 21:20, R.Wieser wrote: 300us is getting on towards realtime. Not really. Translated to a frequency (toggeling the pin) it would be just 1.6 KHz. Thats rather slow for an ARM machine running on 1.4 Ghz (about a million times as fast). It *is* real-time... Real-time is not about speed, but about guaranteed timing. Not possible: the 300 uSec is the basic timing, but the pin isn't always changed on every tick of it (the result is a code train). With Micropython I did successfully use timer and DMA to accomplish something like that (in the 100ns range, though). With a mainloop and delays you can't guarantee to meet your requirements, at least not on a multitasking non-realtime system. Regards, Dietmar -- https://mail.python.org/mailman/listinfo/python-list
Re: Logistic Regression Define X and Y for Prediction
> > > When I define the X and Y values for prediction in the train and test > data, should I capture all the columns that has been "OneHotEncoded" (that > is all columns with 0 and 1) for the X and Y values??? > You might have better luck asking on Stackoverflow, per the Pandas instructions: https://pandas.pydata.org/community.html. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
On 13.11.2019 23:20, Dennis Lee Bieber wrote: For Windows it may require coding a busy-wait sleep function using the high-performance counter and computing a counter value (modulo?) on which to exit the loop. time.perf_counter() is using this on Windows. I'm just worried about floating point limitations on older Python versions. From Python 3.7 onwards, there's time.perf_counter_ns(), but even then it would be nice if it was just possible to reset the perf_counter. Regards, Dietmar -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
Luciano, > Mr. Wieser, I haven't seen you mention which library you're using > to write to GPIO with Python. To be honest, I took a "blink.py" example as a base, and just went with it. So, I had to look it up. Its the first one you mentioned, RPi.GPIO. Thanks for telling me about WiringPi's better performance. I like having multiple options available, and most likely try them al out. > RPi.GPIO has dedicated PWM functions, which may or may not solve > your problem. Probably not, as the clock signal is only used to keep the software-controlled pin-switching synchronised. Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
On 11/13/2019 9:02 AM, R.Wieser wrote: I'm writing some code to toggle a pin on a Raspberry Pi, and would like to have that happen at (multiples of) 300 uSec increments. If you were looking at increments of multiple milleseconds, such as 1/60 == 16.6..., I would suggest using tkinter's root.after, with a minimum non-zero delay of 1 millesecond. I believe one can do better with async as its default loop runs a bit faster and there is a C-coded replacement available, which should be faster yet. It has delay-until(time) as well as delay-for(interval). -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
Dietmar, > Actually, with such requirements you're usually better off with a > microcontroller. Probably, yes. /Anything/ is better than a multi process mini 'puter for tasks like these. Doesn't mean I'm not going to try though. But honestly, generating a "modulated" 1.6 KHz square-wave signal on a 1.4 GHz ARM processor is going towards the limits of what it can do ?Cricky ... > Maybe the PyBoard with Micropython is the right tool for you. Currently I just want to know how the Pi and Python on it handles itself. But thanks for the suggestion nonetheless. Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
David, > The general simplified idea was run it whenever the time is 0 modulo > your interval. So you ask how long until that next time. Ah, using the perf_counter() as its own reference (no "last time fired" storage needed). Thanks for the explanation. Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
Rob, > 300us is getting on towards realtime. Not really. Translated to a frequency (toggeling the pin) it would be just 1.6 KHz. Thats rather slow for an ARM machine running on 1.4 Ghz (about a million times as fast). > I've never tried it that fast. I've already got it running it using the standard sleep() command, but I would like to make the used delay value independant of the Pi board it runs on (had to tweak it when I went from a C++ program on a 3B to a python program on a 3B+).And possibly get a bit more stability of the output signal. > That said, I'll throw out a different option than the threading based ones > people have suggested. Trap SIGALRM with signal.signal and use > signal.setitimer to create an interval timer. (googeling it) Yup, looks like I have to try that out. Thanks. And that automaticaly repeating signal is pretty-much the best solution I can think of. > Also, does the rPi have any PWM or counter pins that you can just set and > forget, rather than trying to keep it going yourself? Not possible: the 300 uSec is the basic timing, but the pin isn't always changed on every tick of it (the result is a code train). Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
Mr. Wieser, I haven't seen you mention which library you're using to write to GPIO with Python. I know of two options: 1) RPi.GPIO: https://sourceforge.net/projects/raspberry-gpio-python/ -- the more popular option 2) WiringPy: https://github.com/WiringPi/WiringPi-Python -- an alternative with better performance See (7 year old) benchmarks here: https://codeandlife.com/2012/07/03/benchmarking-raspberry-pi-gpio-speed/ WiringPy has it's own delay function, which may have better performance than using any Python alternative. See an example here: https://github.com/WiringPi/WiringPi-Python/blob/master/examples/softpwm.py RPi.GPIO has dedicated PWM functions, which may or may not solve your problem. See docs here: https://sourceforge.net/p/raspberry-gpio-python/wiki/PWM/ Cheers, Luciano On Wed, Nov 13, 2019 at 3:27 PM R.Wieser wrote: > > Dennis, > > > So... you need to adjust for the time spent in processing between > > calls to sleep(). > > That was my first thought too, but any code calculating that adjustment > before ultimatily calling sleep itself can also be interrupted.With that > in mind I was aiming/hoping for something with a /very/ short path between > the calculation and executing the sleep (as in: none at all, done by the > system itself). > > > The creep could be mitigated some by having the handler's first > > action being to start the next timer instance, and then doing "stuff". > > Yup. But the cost of using that command is generating threads - which some > search results warned against (not sure why though). > > The best solution I can think of would be a build-in (hardware?) timer which > would generate "ticks" until its stopped. > > Regards, > Rudy Wieser > > > -- > https://mail.python.org/mailman/listinfo/python-list -- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg -- https://mail.python.org/mailman/listinfo/python-list
RE: How to delay until a next increment of time occurs ?
I just used .perf_counter() as it's "a clock with the highest available resolution to measure a short duration" The general simplified idea was run it whenever the time is 0 modulo your interval. So you ask how long until that next time. Run whenever the time is 0 modulo .0003 Current time is 33.0143379 So I'm 0.0002379 into the cycle (33.0143379 % .0003) And I've got .621 to sleep until the next mark (.0003 - .0002379) -Original Message- From: Python-list On Behalf Of R.Wieser Sent: Wednesday, November 13, 2019 1:00 PM To: python-list@python.org Subject: Re: How to delay until a next increment of time occurs ? David, > timeInterval = 0.0003 > time.sleep(timeInterval - (time.perf_counter() % timeInterval)) Possibly: any reason for the performance counter modulo the interval ? Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
On 13.11.2019 19:21, R.Wieser wrote: Yup. But the cost of using that command is generating threads - which some search results warned against (not sure why though). I'm currently looking for a way to have short breaks (in the range of 10us to some ms) on Windows and time.sleep() always seems to be too long, probably due to thread switching. From looking at the source code, it could behave better with Linux, but anyway such solutions will always create a lot of Jitter. The best solution I can think of would be a build-in (hardware?) timer which would generate "ticks" until its stopped. Actually, with such requirements you're usually better off with a microcontroller. These have timers and interrupts available. Maybe the PyBoard with Micropython is the right tool for you. I'm using it for some measurement and control applications and it's really great. Regards, Dietmar -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
On 11/13/19 6:02 AM, R.Wieser wrote: Hello all, I'm writing some code to toggle a pin on a Raspberry Pi, and would like to have that happen at (multiples of) 300 uSec increments. I tried time.sleep(), but that one disregards the time after the last one ends and the new one is started. In other words, all the time spend in code (or being interrupted by another thread!) between the end of the previous and the start of the current sleep throws the whole repetitive timing off. So, I'm looking for a method that will allow me to wait for a "last time plus increment". Is there one with the properties of sleep() (not just burning processor cycles way, blocking all threads), but referencing a previous time. Regards, Rudy Wieser 300us is getting on towards realtime. Depending on how rigorous you need to be, Python may not be the right tool for the job; I've never tried it that fast. That said, I'll throw out a different option than the threading based ones people have suggested. Trap SIGALRM with signal.signal and use signal.setitimer to create an interval timer. I can't tell you offhand that signals will give you better realtime performance than threads or vice versa. There's also async in the mix, which I still have no idea how to use. But this way if you strike out on one approach you've got some others to consider. Also, does the rPi have any PWM or counter pins that you can just set and forget, rather than trying to keep it going yourself? -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
Dennis, > So... you need to adjust for the time spent in processing between > calls to sleep(). That was my first thought too, but any code calculating that adjustment before ultimatily calling sleep itself can also be interrupted.With that in mind I was aiming/hoping for something with a /very/ short path between the calculation and executing the sleep (as in: none at all, done by the system itself). > The creep could be mitigated some by having the handler's first > action being to start the next timer instance, and then doing "stuff". Yup. But the cost of using that command is generating threads - which some search results warned against (not sure why though). The best solution I can think of would be a build-in (hardware?) timer which would generate "ticks" until its stopped. Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
David, > timeInterval = 0.0003 > time.sleep(timeInterval - (time.perf_counter() % timeInterval)) Possibly: any reason for the performance counter modulo the interval ? Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list
RE: How to delay until a next increment of time occurs ?
Maybe something along the lines of this? timeInterval = 0.0003 time.sleep(timeInterval - (time.perf_counter() % timeInterval)) -Original Message- From: Python-list On Behalf Of R.Wieser Sent: Wednesday, November 13, 2019 11:12 AM To: python-list@python.org Subject: Re: How to delay until a next increment of time occurs ? Skip, > Take a look at threading.Timer. Thanks. It seems to solve the problem by calling it at the start of the executed code (instead of the end), but costs thread usage ... Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
Skip, > Take a look at threading.Timer. Thanks. It seems to solve the problem by calling it at the start of the executed code (instead of the end), but costs thread usage ... Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list
Re: How to delay until a next increment of time occurs ?
> So, I'm looking for a method that will allow me to wait for a "last time > plus increment". Is there one with the properties of sleep() (not just > burning processor cycles way, blocking all threads), but referencing a > previous time. Take a look at threading.Timer. Skip -- https://mail.python.org/mailman/listinfo/python-list
How to delay until a next increment of time occurs ?
Hello all, I'm writing some code to toggle a pin on a Raspberry Pi, and would like to have that happen at (multiples of) 300 uSec increments. I tried time.sleep(), but that one disregards the time after the last one ends and the new one is started. In other words, all the time spend in code (or being interrupted by another thread!) between the end of the previous and the start of the current sleep throws the whole repetitive timing off. So, I'm looking for a method that will allow me to wait for a "last time plus increment". Is there one with the properties of sleep() (not just burning processor cycles way, blocking all threads), but referencing a previous time. Regards, Rudy Wieser -- https://mail.python.org/mailman/listinfo/python-list
