Re: Python Dialogs
r...@zedat.fu-berlin.de (Stefan Ram) writes: > Me (indented by 2) and the chatbot (flush left). Lines lengths > 72! Is there a name for this kind of indentation, i.e. the stuff you are writing not being flush left? It is sort of contrary to what I think of as "normal" indentation. You seem to use it in all your postings, too, which hurts my brain, but I guess that's my problem :-) [snip (40 lines)] -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
"Michael F. Stemper" writes:
> On 25/03/2024 01.56, Loris Bennett wrote:
>> Grant Edwards writes:
>>
>>> On 2024-03-22, Loris Bennett via Python-list wrote:
>>>
>>>> Yes, I was mistakenly thinking that the popping the element would
>>>> leave me with the dict minus the popped key-value pair.
>>>
>>> It does.
>> Indeed, but I was thinking in the context of
>>dict_list = [d.pop('a') for d in dict_list]
>> and incorrectly expecting to get a list of 'd' without key 'a',
>> instead
>> of a list of the 'd['a]'.
> I apologize if this has already been mentioned in this thread, but are
> you aware of "d.keys()" and "d.values"?
>
> >>> d = {}
> >>> d['do'] = 'a deer, a female deer'
> >>> d['re'] = 'a drop of golden sunshine'
> >>> d['mi'] = 'a name I call myself'
> >>> d['fa'] = 'a long, long way to run'
> >>> d.keys()
> ['fa', 'mi', 'do', 're']
> >>> d.values()
> ['a long, long way to run', 'a name I call myself', 'a deer, a female deer',
> 'a drop of golden sunshine']
> >>>
Yes, I am, thank you. However, I didn't want either the keys or the
values. Instead I wanted to remove a key within a list comprehension.
Cheers,
Loris
PS: "a drop of golden *sun*" - rhymes with "a long, long way to run"
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
Grant Edwards writes:
> On 2024-03-22, Loris Bennett via Python-list wrote:
>
>> Yes, I was mistakenly thinking that the popping the element would
>> leave me with the dict minus the popped key-value pair.
>
> It does.
Indeed, but I was thinking in the context of
dict_list = [d.pop('a') for d in dict_list]
and incorrectly expecting to get a list of 'd' without key 'a', instead
of a list of the 'd['a]'.
>> Seem like there is no such function.
>
> Yes, there is. You can do that with either pop or del:
>
> >>> d = {'a':1, 'b':2, 'c':3}
> >>> d
> {'a': 1, 'b': 2, 'c': 3}
> >>> d.pop('b')
> 2
> >>> d
> {'a': 1, 'c': 3}
>
>
> >>> d = {'a':1, 'b':2, 'c':3}
> >>> del d['b']
> >>> d
> {'a': 1, 'c': 3}
>
> In both cases, you're left with the dict minus the key/value pair.
>
> In the first case, the deleted value printed by the REPL because it
> was returned by the expression "d.pop('b')" (a method call).
>
> In the second case is no value shown by the REPL because "del d['b']"
> is a statement not an expression.
Thanks for pointing out 'del'. My main problem, however, was failing to
realise that the list comprehension is populated by the return value of
the 'pop', not the popped dict.
Cheers,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
writes:
> Loris wrote:
>
> "Yes, I was mistakenly thinking that the popping the element would leave
> me with the dict minus the popped key-value pair. Seem like there is no
> such function."
>
> Others have tried to explain and pointed out you can del and then use the
> changed dict.
>
> But consider the odd concept of writing your own trivial function.
>
> def remaining(adict, anitem):
> _ = adict.pop(anitem)
> # alternatively duse del on dict and item
> return adict
>
>
remaining({"first": 1, "second": 2, "third": 3}, "second")
> {'first': 1, 'third': 3}
>
>
> Or do you want to be able to call it as in dict.remaining(key) by
> subclassing your own variant of dict and adding a similar method?
No, 'del' does indeed do what I wanted, although I have now decided I
want something else :-) Nevertheless it is good to know that 'del'
exists, so that I don't have to reinvent it.
Cheers,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
Mark Bourne writes:
> Loris Bennett wrote:
>> Hi,
>> I am using SQLAlchemy to extract some rows from a table of 'events'.
>> From the call to the DB I get a list of objects of the type
>>sqlalchemy.orm.state.InstanceState
>> I would like to print these rows to the terminal using the
>> 'tabulate'
>> package, the documentation for which says
>>The module provides just one function, tabulate, which takes a
>> list of
>>lists or another tabular data type as the first argument, and outputs
>>a nicely formatted plain-text table
>> So as I understand it, I need to convert the InstanceState-objects
>> to,
>> say, dicts, in order to print them. However I also want to remove one
>> of the keys from the output and assumed I could just pop it off each
>> event dict, thus:
>>event_dicts = [vars(e) for e in events]
>> print(type(event_dicts[0]))
>> event_dicts = [e.pop('_sa_instance_state', None) for e in event_dicts]
>> print(type(event_dicts[0]))
>
> vars() returns the __dict__ attribute of the object. It may not be a
> good idea to modify that dictionary directly (it will also affect the
> object), although it might be OK if you're not going to do anything
> else with the original objects. To be safer, you could copy the event
> objects:
> event_dicts = [dict(vars(e)) for e in events]
> or:
> event_dicts = [vars(e).copy()]
Thanks for making this clear to me. However, in the end I actually
decided to use the list comprehension without either 'dict()' or
'vars(). Instead I just select the keys I want and so don't need to pop
the unwanted key later and can simultaneously tweak the names of the
key for better printing to the terminal.
>> However, this prints
>>
>>
>> If I comment out the third line, which pops the unwanted key, I get
>>
>>
>> Why does popping one of the keys cause the elements of the list to
>> revert back to their original class?
>
> As Dieter pointed out, the main problem here is that pop() returns the
> value removed, not the dictionary with the rest of the values. You
> probably want something more like:
> for e in event_dicts:
> del e['_sa_instance_state']
> (There's not really any point popping the value if you're not going to
> do anything with it - just delete the key from the dictionary)
Yes, I was mistakenly thinking that the popping the element would leave
me with the dict minus the popped key-value pair. Seem like there is no
such function.
Cheers,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
Popping key causes dict derived from object to revert to object
Hi,
I am using SQLAlchemy to extract some rows from a table of 'events'.
>From the call to the DB I get a list of objects of the type
sqlalchemy.orm.state.InstanceState
I would like to print these rows to the terminal using the 'tabulate'
package, the documentation for which says
The module provides just one function, tabulate, which takes a list of
lists or another tabular data type as the first argument, and outputs
a nicely formatted plain-text table
So as I understand it, I need to convert the InstanceState-objects to,
say, dicts, in order to print them. However I also want to remove one
of the keys from the output and assumed I could just pop it off each
event dict, thus:
event_dicts = [vars(e) for e in events]
print(type(event_dicts[0]))
event_dicts = [e.pop('_sa_instance_state', None) for e in event_dicts]
print(type(event_dicts[0]))
However, this prints
If I comment out the third line, which pops the unwanted key, I get
Why does popping one of the keys cause the elements of the list to
revert back to their original class?
Cheers,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Configuring an object via a dictionary
Tobiah writes: > I should mention that I wanted to answer your question, > but I wouldn't actually do this. I'd rather opt for > your self.config = config solution. The config options > should have their own namespace. > > I don't mind at all referencing foo.config['option'], > or you could make foo.config an object by itself so > you can do foo.config.option. You'd fill it's attributes > in the same way I suggested for your main object. Thanks for the thoughts. I'll go for self.config = config after all, since, as you say, the clutter caused by the referencing is not that significant. Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Configuring an object via a dictionary
Hi, I am initialising an object via the following: def __init__(self, config): self.connection = None self.source_name = config['source_name'] self.server_host = config['server_host'] self.server_port = config['server_port'] self.user_base = config['user_base'] self.user_identifier = config['user_identifier'] self.group_base = config['group_base'] self.group_identifier = config['group_identifier'] self.owner_base = config['owner_base'] However, some entries in the configuration might be missing. What is the best way of dealing with this? I could of course simply test each element of the dictionary before trying to use. I could also just write self.config = config but then addressing the elements will add more clutter to the code. However, with a view to asking forgiveness rather than permission, is there some simple way just to assign the dictionary elements which do in fact exist to self-variables? Or should I be doing this completely differently? Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Accessing configuration across multiple modules
Hi, I am using Typer to create a command-line program with multiple levels of subcommands, so a typical call might look like mytool --config-file=~/test/mytool.conf serviceXYZ list people In the top-level mytool.main, I evaluate the option '--config-file' and read the config file to initialize the logging. This works fine. However, in the module which lists people, namely mytool.serviceXYZ.cli_people I need to set up a connection to an LDAP server in order to actually read the data. If the LDAP connection details are also in the config file, what is the best way of making them accessible at the point where the object wrapping the LDAP server is initialized? I found this a suggestion here which involves creating a separate module for the configuration and then importing it https://codereview.stackexchange.com/questions/269550/python-share-global-variables-across-modules-from-user-defined-config-file I think I could probably get that to work, but are there any better alternatives? Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Re: Printing dict value for possibly undefined key
DL Neil writes:
> On 11/25/2023 3:31 AM, Loris Bennett via Python-list wrote:
>> Hi,
>> I want to print some records from a database table where one of the
>> fields contains a JSON string which is read into a dict. I am doing
>> something like
>>print(f"{id} {d['foo']} {d['bar']}")
>> However, the dict does not always have the same keys, so d['foo'] or
>> d['bar'] may be undefined. I can obviously do something like
>>if not 'foo' in d:
>> d['foo']="NULL"
>>if not 'bar' in d:
>> d['bar']="NULL"
>>print(f"{id} {d['foo']} {d['bar']}")
>> Is there any more compact way of achieving the same thing?
>
>
> What does "the dict does not always have the same keys" mean?
>
> a) there are two (or...) keys, but some records don't include both;
>
> b) there may be keys other than 'foo' and 'bar' which not-known in-advance;
>
> c) something else.
Sorry for being unclear. There is either 'foo' or 'bar' or both, plus
some other keys which are always present.
> As mentioned, dict.get() solves one of these.
>
> Otherwise, there are dict methods which collect/reveal all the keys,
> all the values, or both - dict.keys(), .values(), .items(), resp.
That is a also a good point. I had forgotten about dict.keys() and
dict.values(), and hadn't been aware of dict.items().
Cheers,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Printing dict value for possibly undefined key
duncan smith writes:
> On 24/11/2023 16:35, duncan smith wrote:
>> On 24/11/2023 14:31, Loris Bennett wrote:
>>> Hi,
>>>
>>> I want to print some records from a database table where one of the
>>> fields contains a JSON string which is read into a dict. I am doing
>>> something like
>>>
>>> print(f"{id} {d['foo']} {d['bar']}")
>>>
>>> However, the dict does not always have the same keys, so d['foo'] or
>>> d['bar'] may be undefined. I can obviously do something like
>>>
>>> if not 'foo' in d:
>>> d['foo']="NULL"
>>> if not 'bar' in d:
>>> d['bar']="NULL"
>>> print(f"{id} {d['foo']} {d['bar']}")
>>>
>>> Is there any more compact way of achieving the same thing?
>>>
>>> Cheers,
>>>
>>> Loris
>>>
>> Yes. e.g.
>> d.get('foo', "NULL")
>> Duncan
>
> Or make d a defaultdict.
>
> from collections import defaultdict
>
> dic = defaultdict(lambda:'NULL')
> dic['foo'] = 'astring'
> dic['foo']
> 'astring'
> dic['bar']
> 'NULL'
>
> Duncan
>
I have gone with the 'd.get' solution, as I am just need to print the
dict to the terminal. The dict is actually from a list of dicts which
is generated by querying a database, so I don't think the defaultdict
approach would be so appropriate, but it's good to know about it.
Thanks,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
Printing dict value for possibly undefined key
Hi,
I want to print some records from a database table where one of the
fields contains a JSON string which is read into a dict. I am doing
something like
print(f"{id} {d['foo']} {d['bar']}")
However, the dict does not always have the same keys, so d['foo'] or
d['bar'] may be undefined. I can obviously do something like
if not 'foo' in d:
d['foo']="NULL"
if not 'bar' in d:
d['bar']="NULL"
print(f"{id} {d['foo']} {d['bar']}")
Is there any more compact way of achieving the same thing?
Cheers,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
Re: SQL rollback of multiple inserts involving constraints
Jacob Kruger writes: > Think performing a session/transaction flush after the first two > inserts should offer the workaround before you've committed all > transaction actions to the database finally: > > https://medium.com/@oba2311/sqlalchemy-whats-the-difference-between-a-flush-and-commit-baec6c2410a9 > > > HTH Yes, thank you, it does. I hadn't been aware of 'flush'. > Jacob Kruger > +2782 413 4791 > "Resistance is futile!...Acceptance is versatile..." > > > On 2023/11/10 11:15, Loris Bennett via Python-list wrote: >> Hi, >> >> In my MariaDB database I have a table 'people' with 'uid' as the primary >> key and a table 'groups' with 'gid' as the primary key. I have a third >> table 'memberships' with 'uid' and 'gid' being the primary key and the >> constraint that values for 'uid' and 'gid' exist in the tables 'people' >> and 'groups', respectively. I am using SQLAlchemy and writing a method >> to setup a membership for a new person in a new group. >> >> I had assumed that I should be able to perform all three inserts >> (person, group, membership) with a single transaction and then rollback >> if there is a problem. However, the problem is that if the both the >> insert into 'people' and that into 'groups' are not first committed, the >> constraint on the insertion of the membership fails. >> >> What am I doing wrong? >> >> Apologies if this is actually an SQL question rather than something >> related to SQLAlchemy. >> >> Cheers, >> >> Loris >> > -- Dr. Loris Bennett (Herr/Mr) ZEDAT, Freie Universität Berlin -- https://mail.python.org/mailman/listinfo/python-list
SQL rollback of multiple inserts involving constraints
Hi, In my MariaDB database I have a table 'people' with 'uid' as the primary key and a table 'groups' with 'gid' as the primary key. I have a third table 'memberships' with 'uid' and 'gid' being the primary key and the constraint that values for 'uid' and 'gid' exist in the tables 'people' and 'groups', respectively. I am using SQLAlchemy and writing a method to setup a membership for a new person in a new group. I had assumed that I should be able to perform all three inserts (person, group, membership) with a single transaction and then rollback if there is a problem. However, the problem is that if the both the insert into 'people' and that into 'groups' are not first committed, the constraint on the insertion of the membership fails. What am I doing wrong? Apologies if this is actually an SQL question rather than something related to SQLAlchemy. Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Re: NameError: name '__version__' is not defined
"Loris Bennett" writes:
> "Loris Bennett" writes:
>
>> Hi,
>>
>> I have two applications. One uses the system version of Python, which
>> is 3.6.8, whereas the other uses Python 3.10.8 installed in a non-system
>> path. For both applications I am using poetry with a pyproject.toml
>> file which contains the version information and __init__.py at the root
>> which contains
>>
>> try:
>> import importlib.metadata as importlib_metadata
>> except ModuleNotFoundError:
>> import importlib_metadata
>>
>> __version__ = importlib_metadata.version(__name__)
>>
>> For the application with the system Python this mechanism works, but for
>> the non-system Python I get the error:
>>
>> NameError: name '__version__' is not defined
>>
>> For the 3.6 application I have
>>
>> PYTHONPATH=/nfs/local/lib/python3.6/site-packages
>> PYTHONUSERBASE=/nfs/local
>> PYTHON_VERSION=3.6
>> PYTHON_VIRTUALENV=
>>
>> and for the 3.10 application I have
>>
>>
>> PYTHONPATH=/nfs/easybuild/software/Python/3.10.8-GCCcore-12.2.0/easybuild/python:/nfs/local/lib/python3.10/site-packages
>> PYTHONUSERBASE=/nfs/local
>> PYTHON_VERSION=3.10
>> PYTHON_VIRTUALENV=
>>
>> The applications are installed in /nfs/local/lib/python3.6/site-packages
>> and /nfs/local/lib/python3.10/site-packages, respectively.
>>
>> Can anyone see where this is going wrong? I thought it should be
>> enough that the packages with the metadata is available via PYTHONPATH,
>> but this seems not to be sufficient. So I must be overseeing something.
>
> If in the 3.10 application I add
>
> print(f"__init__ Version: {__version__}")
>
> to __init__.py the correct version is printed. So the problem is that
> the variable is not available at the point I am trying access it. The
> relevant code (a far as I can tell) in main.py looks like this:
>
> import typer
>
> app = typer.Typer()
>
>
> @app.callback()
> def version_callback(value: bool):
> if value:
> typer.echo(f"Version: {__version__}")
> raise typer.Exit()
>
>
> @app.callback()
> def common(
> ctx: typer.Context,
> version: bool = typer.Option(None, "--version",
>help="Show version",
>callback=version_callback),
> ):
> pass
>
> if __name__ == "__main__":
>
> app()
>
> This is the first time I have used typer, so it is more than likely that
> I have made some mistakes.
OK, I worked it out. Instead of
typer.echo(f"Version: {__version__}")
I need
typer.echo(f"Version: {mypackage.__version__}")
Thanks for the help :-)
Even if no-one replies, it still helps me to have to formulate the
problem for an audience of people who probably know more than I do.
Cheers,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
Re: NameError: name '__version__' is not defined
"Loris Bennett" writes:
> Hi,
>
> I have two applications. One uses the system version of Python, which
> is 3.6.8, whereas the other uses Python 3.10.8 installed in a non-system
> path. For both applications I am using poetry with a pyproject.toml
> file which contains the version information and __init__.py at the root
> which contains
>
> try:
> import importlib.metadata as importlib_metadata
> except ModuleNotFoundError:
> import importlib_metadata
>
> __version__ = importlib_metadata.version(__name__)
>
> For the application with the system Python this mechanism works, but for
> the non-system Python I get the error:
>
> NameError: name '__version__' is not defined
>
> For the 3.6 application I have
>
> PYTHONPATH=/nfs/local/lib/python3.6/site-packages
> PYTHONUSERBASE=/nfs/local
> PYTHON_VERSION=3.6
> PYTHON_VIRTUALENV=
>
> and for the 3.10 application I have
>
>
> PYTHONPATH=/nfs/easybuild/software/Python/3.10.8-GCCcore-12.2.0/easybuild/python:/nfs/local/lib/python3.10/site-packages
> PYTHONUSERBASE=/nfs/local
> PYTHON_VERSION=3.10
> PYTHON_VIRTUALENV=
>
> The applications are installed in /nfs/local/lib/python3.6/site-packages
> and /nfs/local/lib/python3.10/site-packages, respectively.
>
> Can anyone see where this is going wrong? I thought it should be
> enough that the packages with the metadata is available via PYTHONPATH,
> but this seems not to be sufficient. So I must be overseeing something.
If in the 3.10 application I add
print(f"__init__ Version: {__version__}")
to __init__.py the correct version is printed. So the problem is that
the variable is not available at the point I am trying access it. The
relevant code (a far as I can tell) in main.py looks like this:
import typer
app = typer.Typer()
@app.callback()
def version_callback(value: bool):
if value:
typer.echo(f"Version: {__version__}")
raise typer.Exit()
@app.callback()
def common(
ctx: typer.Context,
version: bool = typer.Option(None, "--version",
help="Show version",
callback=version_callback),
):
pass
if __name__ == "__main__":
app()
This is the first time I have used typer, so it is more than likely that
I have made some mistakes.
Cheers,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
NameError: name '__version__' is not defined
Hi, I have two applications. One uses the system version of Python, which is 3.6.8, whereas the other uses Python 3.10.8 installed in a non-system path. For both applications I am using poetry with a pyproject.toml file which contains the version information and __init__.py at the root which contains try: import importlib.metadata as importlib_metadata except ModuleNotFoundError: import importlib_metadata __version__ = importlib_metadata.version(__name__) For the application with the system Python this mechanism works, but for the non-system Python I get the error: NameError: name '__version__' is not defined For the 3.6 application I have PYTHONPATH=/nfs/local/lib/python3.6/site-packages PYTHONUSERBASE=/nfs/local PYTHON_VERSION=3.6 PYTHON_VIRTUALENV= and for the 3.10 application I have PYTHONPATH=/nfs/easybuild/software/Python/3.10.8-GCCcore-12.2.0/easybuild/python:/nfs/local/lib/python3.10/site-packages PYTHONUSERBASE=/nfs/local PYTHON_VERSION=3.10 PYTHON_VIRTUALENV= The applications are installed in /nfs/local/lib/python3.6/site-packages and /nfs/local/lib/python3.10/site-packages, respectively. Can anyone see where this is going wrong? I thought it should be enough that the packages with the metadata is available via PYTHONPATH, but this seems not to be sufficient. So I must be overseeing something. Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Installing package as root to a system directory
Hi, I use poetry to develop system software packages as a normal user. To install the packages I use, again as a normal user export PYTHONUSERBASE=/some/path pip3 install --user somepackage.whl and add /some/path to /usr/lib64/python3.6/site-packages/zedat.pth This works well enough, but seems to me to be a little clunky, mainly because the files don't then belong to root. The most correct way, in my case, would probably be to create an RPM out of the Python package, but that seems like it would be too much overhead. What other approaches to people use? Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Displaying CPU instruction sets used for TensorFlow build?
Hi, Does anyone know how I can display the CPU instruction sets which were used when TensorFlow was compiled? I initially compiled TF on a machine with a CPU which supports AVX512_VNNI. I subsequently recompiled on a second machine without AVX512_VNNI, but when I run a test program on the second machine, I get the error: The TensorFlow library was compiled to use AVX512_VNNI instructions, but these aren't available on your machine. I would like to check the instruction sets explicitly, so I can tell whether I am using the version of TF I think I am, or whether the test program has some sort of problem. Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
