Re: How/where to store calibration values - written by program A, read by program B

[Thomas Passin via Python-list]

On 12/6/2023 1:12 PM, MRAB via Python-list wrote:

On 2023-12-06 12:23, Thomas Passin via Python-list wrote:

On 12/6/2023 6:35 AM, Barry Scott via Python-list wrote:



On 6 Dec 2023, at 09:32, Chris Green via Python-list 
 wrote:


My requirement is *slightly* more complex than just key value pairs,
it has one level of hierarchy, e.g.:-

    KEY1:
  a: v1
  c: v3
  d: v4
    KEY2:
  a: v7
  b: v5
  d: v6

Different numbers of value pairs under each KEY.


JSON will allow you to nest dictionaries.

{
 'KEY1': {
 'a': v1
 'c': v3
 'd': v4
 }
 'KEY2': {
  'a': v7
  'b': v5
  'd': v6
 }
}

Personally I would not use .ini style these days as the format does 
not include type of the data.


Neither does JSON.  Besides, JSON is more complicated than necessary
here - in fact, your example isn't even legal JSON since lines are
missing commas.

Fun fact - for at least some, maybe most, JSON files, using eval() on
them is hugely faster than using Python's standard JSON library.  I
learned this when I wanted to ingest a large browser bookmarks JSON
file. It wouldn't matter for a much smaller file, of course.


It would be safer if you used literal_eval.


He's going to be writing his own calibration data files, though, so it 
should be safe for his purposes.


--
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[MRAB via Python-list]

On 2023-12-06 20:11, dn via Python-list wrote:

On 7/12/23 07:12, MRAB via Python-list wrote:

On 2023-12-06 12:23, Thomas Passin via Python-list wrote:

On 12/6/2023 6:35 AM, Barry Scott via Python-list wrote:



On 6 Dec 2023, at 09:32, Chris Green via Python-list 
 wrote:


My requirement is *slightly* more complex than just key value pairs,
it has one level of hierarchy, e.g.:-

    KEY1:
  a: v1
  c: v3
  d: v4
    KEY2:
  a: v7
  b: v5
  d: v6

Different numbers of value pairs under each KEY.


JSON will allow you to nest dictionaries.

{
 'KEY1': {
 'a': v1
 'c': v3
 'd': v4
 }
 'KEY2': {
  'a': v7
  'b': v5
  'd': v6
 }
}

Personally I would not use .ini style these days as the format does 
not include type of the data.


Neither does JSON.  Besides, JSON is more complicated than necessary
here - in fact, your example isn't even legal JSON since lines are
missing commas.

Fun fact - for at least some, maybe most, JSON files, using eval() on
them is hugely faster than using Python's standard JSON library.  I
learned this when I wanted to ingest a large browser bookmarks JSON
file. It wouldn't matter for a much smaller file, of course.


It would be safer if you used literal_eval.


Ah, memories of Python2...

Does this little hack still work?

What about True/False cf true/false?


Nope, nor None cf null.

If it's numbers, strings, lists and dicts, it works.

--
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[dn via Python-list]

On 7/12/23 07:12, MRAB via Python-list wrote:

On 2023-12-06 12:23, Thomas Passin via Python-list wrote:

On 12/6/2023 6:35 AM, Barry Scott via Python-list wrote:



On 6 Dec 2023, at 09:32, Chris Green via Python-list 
 wrote:


My requirement is *slightly* more complex than just key value pairs,
it has one level of hierarchy, e.g.:-

    KEY1:
  a: v1
  c: v3
  d: v4
    KEY2:
  a: v7
  b: v5
  d: v6

Different numbers of value pairs under each KEY.


JSON will allow you to nest dictionaries.

{
 'KEY1': {
 'a': v1
 'c': v3
 'd': v4
 }
 'KEY2': {
  'a': v7
  'b': v5
  'd': v6
 }
}

Personally I would not use .ini style these days as the format does 
not include type of the data.


Neither does JSON.  Besides, JSON is more complicated than necessary
here - in fact, your example isn't even legal JSON since lines are
missing commas.

Fun fact - for at least some, maybe most, JSON files, using eval() on
them is hugely faster than using Python's standard JSON library.  I
learned this when I wanted to ingest a large browser bookmarks JSON
file. It wouldn't matter for a much smaller file, of course.


It would be safer if you used literal_eval.


Ah, memories of Python2...

Does this little hack still work?

What about True/False cf true/false?

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[MRAB via Python-list]

On 2023-12-06 12:23, Thomas Passin via Python-list wrote:

On 12/6/2023 6:35 AM, Barry Scott via Python-list wrote:




On 6 Dec 2023, at 09:32, Chris Green via Python-list  
wrote:

My requirement is *slightly* more complex than just key value pairs,
it has one level of hierarchy, e.g.:-

KEY1:
  a: v1
  c: v3
  d: v4
KEY2:
  a: v7
  b: v5
  d: v6

Different numbers of value pairs under each KEY.


JSON will allow you to nest dictionaries.

{
 'KEY1': {
 'a': v1
 'c': v3
 'd': v4
 }
 'KEY2': {
  'a': v7
  'b': v5
  'd': v6
 }
}

Personally I would not use .ini style these days as the format does not include 
type of the data.


Neither does JSON.  Besides, JSON is more complicated than necessary
here - in fact, your example isn't even legal JSON since lines are
missing commas.

Fun fact - for at least some, maybe most, JSON files, using eval() on
them is hugely faster than using Python's standard JSON library.  I
learned this when I wanted to ingest a large browser bookmarks JSON
file. It wouldn't matter for a much smaller file, of course.


It would be safer if you used literal_eval.
--
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[Thomas Passin via Python-list]

On 12/6/2023 6:35 AM, Barry Scott via Python-list wrote:




On 6 Dec 2023, at 09:32, Chris Green via Python-list  
wrote:

My requirement is *slightly* more complex than just key value pairs,
it has one level of hierarchy, e.g.:-

KEY1:
  a: v1
  c: v3
  d: v4
KEY2:
  a: v7
  b: v5
  d: v6

Different numbers of value pairs under each KEY.


JSON will allow you to nest dictionaries.

{
 'KEY1': {
 'a': v1
 'c': v3
 'd': v4
 }
 'KEY2': {
  'a': v7
  'b': v5
  'd': v6
 }
}

Personally I would not use .ini style these days as the format does not include 
type of the data.


Neither does JSON.  Besides, JSON is more complicated than necessary 
here - in fact, your example isn't even legal JSON since lines are 
missing commas.


Fun fact - for at least some, maybe most, JSON files, using eval() on 
them is hugely faster than using Python's standard JSON library.  I 
learned this when I wanted to ingest a large browser bookmarks JSON 
file. It wouldn't matter for a much smaller file, of course.


--
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[Dan Purgert via Python-list]

On 2023-12-06, Stefan Ram wrote:
> Chris Green  writes:
>>KEY1:
>>  a: v1
>>  c: v3
>>  d: v4
>>KEY2:
>>  a: v7
>>  b: v5
>>  d: v6
>
>   That maps nicely to two directories with three files
>   (under an application-specific configuration directory).

Or an .ini file with two sections (although I don't think you can re-use
key-names in a single ini file)


-- 
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1  E067 6D65 70E5 4CE7 2860
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[Barry Scott via Python-list]

> On 6 Dec 2023, at 09:32, Chris Green via Python-list  
> wrote:
> 
> My requirement is *slightly* more complex than just key value pairs,
> it has one level of hierarchy, e.g.:-
> 
>KEY1:
>  a: v1
>  c: v3
>  d: v4
>KEY2:
>  a: v7
>  b: v5
>  d: v6
> 
> Different numbers of value pairs under each KEY.

JSON will allow you to nest dictionaries.

{
'KEY1': {
'a': v1
'c': v3
'd': v4
}
'KEY2': {
 'a': v7
 'b': v5
 'd': v6
}
}

Personally I would not use .ini style these days as the format does not include 
type of the data.

Also I would not use the ast.literal_eval as it makes debugging errors in the 
data harder.

Barry

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[Dan Sommers via Python-list]

On 2023-12-06 at 09:32:02 +,
Chris Green via Python-list  wrote:

> Thomas Passin  wrote:

[...]

> > Just go with an .ini file. Simple, well-supported by the standard 
> > library. And it gives you key/value pairs.
> > 
> My requirement is *slightly* more complex than just key value pairs,
> it has one level of hierarchy, e.g.:-
> 
> KEY1:
>   a: v1
>   c: v3
>   d: v4
> KEY2:
>   a: v7
>   b: v5
>   d: v6
> 
> Different numbers of value pairs under each KEY.

INI files have sections.

See .
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[Chris Green via Python-list]

Thank you everyone for all the suggestions, I now have several
possibilities to follow up. :-)

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[Chris Green via Python-list]

Thomas Passin  wrote:
> On 12/5/2023 11:50 AM, MRAB via Python-list wrote:
> > On 2023-12-05 14:37, Chris Green via Python-list wrote:
> >> Is there a neat, pythonic way to store values which are 'sometimes'
> >> changed?
> >>
> >> My particular case at the moment is calibration values for ADC inputs
> >> which are set by running a calibration program and used by lots of
> >> programs which display the values or do calculations with them.
> >>
> >>  From the program readability point of view it would be good to have a
> >> Python module with the values in it but using a Python program to
> >> write/update a Python module sounds a bit odd somehow.
> >>
> >> I could simply write the values to a file (or a database) and I
> >> suspect that this may be the best answer but it does make retrieving
> >> the values different from getting all other (nearly) constant values.
> >>
> >> Are there any Python modules aimed specifically at this sort of
> >> requirement?
> >>
> > Some kind of key/value store sounds like the correct solution. I 
> > wouldn't go as far a database - that's overkill for a few calibration 
> > values.
> > 
> > I might suggest TOML, except that Python's tomllib (Python 3.11+) is 
> > read-only!
> > 
> > Personally, I'd go for lines of:
> > 
> >      key1: value1
> >      key2: value2
> > 
> > Simple to read, simple to write.
> 
> Just go with an .ini file. Simple, well-supported by the standard 
> library. And it gives you key/value pairs.
> 
My requirement is *slightly* more complex than just key value pairs,
it has one level of hierarchy, e.g.:-

KEY1:
  a: v1
  c: v3
  d: v4
KEY2:
  a: v7
  b: v5
  d: v6

Different numbers of value pairs under each KEY.

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How/where to store calibration values - written by program A, read by program B

[Chris Green via Python-list]

Paul Rubin  wrote:
> Chris Green  writes:
> > I could simply write the values to a file (or a database) and I
> > suspect that this may be the best answer but it does make retrieving
> > the values different from getting all other (nearly) constant values.
> 
> I've used configparser for this, though its intention is files that are
> manually edited, rather than updated by a program.
> 
> You can also read and dump a json file or pickle file.  I prefer json to
> pickle for most purposes these days.
> 
> I don't like YAML and I don't like the proliferation of markup formats
> of this type.  So while I don't know exactly what TOML is, I figure it
> must be bad.
> 
> I sometimes use ast.literal_eval though it is Python specific.
> 
That's interesting, I'll add it to my armoury anyway. :-)


> Of course there is also sqlite but that is probably overkill.

It's what my current code uses but does feel a bit OTT and it isn't
particularly convenient to view when debugging. 

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list