Re: Fast lookup of bulky "table"

2023-01-16 Thread rbowman
On Mon, 16 Jan 2023 12:28:37 -0500, Thomas Passin wrote: > On 1/16/2023 11:56 AM, rbowman wrote: >> On 16 Jan 2023 15:14:06 GMT, Stefan Ram wrote: >> >> >>>When none of those reasons matter, one can use dictionaries in >>>Python as well. And then what Chandler Carruth showed us applies:

Re: Fast lookup of bulky "table"

2023-01-16 Thread Dino
On 1/16/2023 1:18 PM, Edmondo Giovannozzi wrote: As a comparison with numpy. Given the following lines: import numpy as np a = np.random.randn(400,100_000) ia = np.argsort(a[0,:]) a_elem = a[56, ia[0]] I have just taken an element randomly in a numeric table of 400x10 elements To find it w

Re: Fast lookup of bulky "table"

2023-01-16 Thread Albert-Jan Roskam
On Jan 15, 2023 05:26, Dino wrote: Hello, I have built a PoC service in Python Flask for my work, and - now that the point is made - I need to make it a little more performant (to be honest, chances are that someone else will pick up from where I left off, and implement the

Re: Fast lookup of bulky "table"

2023-01-16 Thread Thomas Passin
On 1/16/2023 11:56 AM, rbowman wrote: On 16 Jan 2023 15:14:06 GMT, Stefan Ram wrote: When none of those reasons matter, one can use dictionaries in Python as well. And then what Chandler Carruth showed us applies: I am missing something. Where is the data in your dictionary coming from

Re: Fast lookup of bulky "table"

2023-01-16 Thread Peter J. Holzer
On 2023-01-15 18:06:36 -0500, Thomas Passin wrote: > You especially want to avoid letting the database engine do full-table > scans over and over. And you never want to send a lot of rows to > Python and do post-filtering on them if you can avoid it. Another thing to avoid: Lots of small queries.

Re: Fast lookup of bulky "table"

2023-01-16 Thread Peter J. Holzer
On 2023-01-16 09:12:30 +1300, dn via Python-list wrote: > On 16/01/2023 08.36, Weatherby,Gerard wrote: > > I think any peformance improvements would have to come from a language > > change or better indexing of the data. > Expanding on @Peter's post: databases (relational or not) are best organise

Re: Fast lookup of bulky "table"

2023-01-16 Thread Edmondo Giovannozzi
Il giorno domenica 15 gennaio 2023 alle 05:26:50 UTC+1 Dino ha scritto: > Hello, I have built a PoC service in Python Flask for my work, and - now > that the point is made - I need to make it a little more performant (to > be honest, chances are that someone else will pick up from where I left >

Re: Fast lookup of bulky "table"

2023-01-16 Thread Thomas Passin
On 1/16/2023 10:14 AM, Stefan Ram wrote: However, operating systems and databases also try to cache information in main memory that is estimated to be accessed often. Yes, and you can only know by testing, when that's possible. Also, if you know that you have the same queries repeated over

Re: Fast lookup of bulky "table"

2023-01-16 Thread rbowman
On 16 Jan 2023 15:14:06 GMT, Stefan Ram wrote: > When none of those reasons matter, one can use dictionaries in Python > as well. And then what Chandler Carruth showed us applies: I am missing something. Where is the data in your dictionary coming from? -- https://mail.python.org/mailman/li

Re: Fast lookup of bulky "table"

2023-01-16 Thread Dino
On 1/16/2023 2:53 AM, David wrote: See here: https://docs.python.org/3/reference/expressions.html#assignment-expressions https://realpython.com/python-walrus-operator/ Thank you, brother. -- https://mail.python.org/mailman/listinfo/python-list

Re: Fast lookup of bulky "table"

2023-01-16 Thread Dino
Just wanted to take a moment to express my gratitude to everyone who responded here. You have all been so incredibly helpful. Thank you Dino On 1/14/2023 11:26 PM, Dino wrote: Hello, I have built a PoC service in Python Flask for my work, and - now that the point is made - I need to make