Re: How/where to store calibration values - written by program A, read by program B

[Thomas Passin via Python-list]

On 12/5/2023 11:50 AM, MRAB via Python-list wrote:

On 2023-12-05 14:37, Chris Green via Python-list wrote:

Is there a neat, pythonic way to store values which are 'sometimes'
changed?

My particular case at the moment is calibration values for ADC inputs
which are set by running a calibration program and used by lots of
programs which display the values or do calculations with them.

 From the program readability point of view it would be good to have a
Python module with the values in it but using a Python program to
write/update a Python module sounds a bit odd somehow.

I could simply write the values to a file (or a database) and I
suspect that this may be the best answer but it does make retrieving
the values different from getting all other (nearly) constant values.

Are there any Python modules aimed specifically at this sort of
requirement?

Some kind of key/value store sounds like the correct solution. I 
wouldn't go as far a database - that's overkill for a few calibration 
values.


I might suggest TOML, except that Python's tomllib (Python 3.11+) is 
read-only!


Personally, I'd go for lines of:

     key1: value1
     key2: value2

Simple to read, simple to write.


Just go with an .ini file. Simple, well-supported by the standard 
library. And it gives you key/value pairs.


--
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[DL Neil via Python-list]

Apologies: neglected suggested web.refs:

https://datagy.io/python-environment-variables/
https://pypi.org/project/json_environ/

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[DL Neil via Python-list]

On 12/6/23 03:37, Chris Green via Python-list wrote:

Is there a neat, pythonic way to store values which are 'sometimes'
changed?

My particular case at the moment is calibration values for ADC inputs
which are set by running a calibration program and used by lots of
programs which display the values or do calculations with them.

 From the program readability point of view it would be good to have a
Python module with the values in it but using a Python program to
write/update a Python module sounds a bit odd somehow.

I could simply write the values to a file (or a database) and I
suspect that this may be the best answer but it does make retrieving
the values different from getting all other (nearly) constant values.

Are there any Python modules aimed specifically at this sort of
requirement?


Another programming-term for these might be "environment variables". 
However, be aware that such also has a specific meaning at the Operating 
System level.


1 Sysops Environment Variables exist completely outside the code. Python 
interrogates the Sysops to fetch/set them. Can be problematic because 
only apply on single machine and are not part of change-control, VS, etc.


2 A .con file (in my tradition, likely still .uni type in MSFT) or 
similar, which contains the key:value pairs recommended elsewhere. There 
are formal .con and .uni (etc) formats. Most of the teams I've 
worked-with recently seem to settle on .JSON files which are 
very-conveniently structured as (read into/written from) a Python 
dictionary, and a single function-call interaction.


Word of warning/voice of [bitter] experience: ALWAYS *trumpet* any 
changes in these values AND output them (as you would any "assumptions" 
for a calculation) at the top of output reports. The trouble is that 
humans assume continuity but such an arrangement is NOT idempotent - 
which leads to complaints: "I ran it on Monday and got these results, 
but when I ran it again on Tuesday, the results changed"...


Yes there are Python libraries. Choose your method/format first, and 
then search - either Duckboards or straight from Pepi.


I have systems which use an DBMS for environment variables, but (a) only 
when there's a significant number, and (b) when the application is 
already connecting to the DBMS for processing [and maybe (c) because I 
know my way around such tools so they're 'easy']. Not recommended!


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[Barry Scott via Python-list]

> On 5 Dec 2023, at 14:37, Chris Green via Python-list  
> wrote:
> 
> Are there any Python modules aimed specifically at this sort of
> requirement?

I tend to use JSON for this type of thing.
Suggest that you use the options to pretty print the json that is saved so that 
a human can read it.

For example:

import json

config = {
'key2': 42,
'key1': 'value 1',
}

print(json.dumps(config, indent=4, separators=(', ', ': '), sort_keys=True))


% python $T/a.py
{
"key1": "value 1",
"key2": 42
}

Barry

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[Mats Wichmann via Python-list]

On 12/5/23 07:37, Chris Green via Python-list wrote:

Is there a neat, pythonic way to store values which are 'sometimes'
changed?

My particular case at the moment is calibration values for ADC inputs
which are set by running a calibration program and used by lots of
programs which display the values or do calculations with them.

 From the program readability point of view it would be good to have a
Python module with the values in it but using a Python program to
write/update a Python module sounds a bit odd somehow.

I could simply write the values to a file (or a database) and I
suspect that this may be the best answer but it does make retrieving
the values different from getting all other (nearly) constant values.

Are there any Python modules aimed specifically at this sort of
requirement?


A search term to look for is "data persistence"

there is lots of support at various levels - you can do simpler things 
with plain text (or binary), json data, or csv data, or configparser, or 
use pickles; if there's not a lot of values a dbapi database may, as 
already mentioned, be overkill.


--
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[MRAB via Python-list]

On 2023-12-05 14:37, Chris Green via Python-list wrote:

Is there a neat, pythonic way to store values which are 'sometimes'
changed?

My particular case at the moment is calibration values for ADC inputs
which are set by running a calibration program and used by lots of
programs which display the values or do calculations with them.

 From the program readability point of view it would be good to have a
Python module with the values in it but using a Python program to
write/update a Python module sounds a bit odd somehow.

I could simply write the values to a file (or a database) and I
suspect that this may be the best answer but it does make retrieving
the values different from getting all other (nearly) constant values.

Are there any Python modules aimed specifically at this sort of
requirement?

Some kind of key/value store sounds like the correct solution. I 
wouldn't go as far a database - that's overkill for a few calibration 
values.


I might suggest TOML, except that Python's tomllib (Python 3.11+) is 
read-only!


Personally, I'd go for lines of:

key1: value1
key2: value2

Simple to read, simple to write.

--
https://mail.python.org/mailman/listinfo/python-list

How/where to store calibration values - written by program A, read by program B

[Chris Green via Python-list]

Is there a neat, pythonic way to store values which are 'sometimes'
changed?

My particular case at the moment is calibration values for ADC inputs
which are set by running a calibration program and used by lots of
programs which display the values or do calculations with them.

From the program readability point of view it would be good to have a
Python module with the values in it but using a Python program to
write/update a Python module sounds a bit odd somehow.

I could simply write the values to a file (or a database) and I
suspect that this may be the best answer but it does make retrieving
the values different from getting all other (nearly) constant values.

Are there any Python modules aimed specifically at this sort of
requirement?

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: on writing a number as 2^s * q, where q is odd

[jak via Python-list]

Alan Bawden ha scritto:

If you like this sort of stuff, check out the book "Hacker's Delight" by
Henry Warren.  See.


Thank you for your suggestion. Really interesting. Just for fun I tried
to port the function to 64 bit:

def bit_count_64(n):
   lt = n >> 62
   n &= (~(0x3f << 62)) & ((1 << 63) - 1)
   n = (n - ((n >> 1) & 0o3)
  - ((n >> 2) & 0o1))
   n = (   (n & 0o307070707070707070707)
+ ((n & 0o070707070707070707070) >> 3))
   return (n % 63) + (0, 1, 1, 2)[lt]

n=0x
bit_count_64(n)
64

n=0x3ffe
bit_count_64(n)
61

bit_count_64(1 << 63)
1

...in C it would have been simpler :^)

--
https://mail.python.org/mailman/listinfo/python-list

Re: on writing a number as 2^s * q, where q is odd

[Alan Bawden via Python-list]

jak  writes:

   Oscar Benjamin ha scritto:
   ...
   If we now use the function being discussed:

   powers_of_2_in(n)
   (63, 1)

   we can see that the bit_count() method had to do 63 iterations to count
   the bits

I certainly hope that the bit_count method doesn't count bits by
iterating over them one-by-one.  You can count bits _much_ faster than
that.

You can count the bits in a 62-bit number as follows:

   def bit_count_62(n):
   n = (n - ((n >> 1) & 0o3)
  - ((n >> 2) & 0o1))
   n = (   (n & 0o307070707070707070707)
+ ((n & 0o070707070707070707070) >> 3))
   return n % 63

Then if you want to count the bits in arbitrarily large integers, you
iterate over them in N-bit chunks, for some N <= 62.  Your choice of N
will depend on how you represent your big integers.  Probably N is 56 or
48 or 32.

And why 62 bits?  Because the bit_count method is certainly written in
C, where every step in bit_count_62 would use 64-bit integers.

If you like this sort of stuff, check out the book "Hacker's Delight" by
Henry Warren.  See .

- Alan
-- 
https://mail.python.org/mailman/listinfo/python-list