Re: Efficiency debates

2021-02-15 Thread Alan Gauld via Python-list
On 15/02/2021 02:26, Avi Gross via Python-list wrote:
> I think we have discussed this a few times.

Indeed, many times!

And there is a natural tendency for a group focused on a
programming language to fixate on language improvements. But
it's worth while to back up and look at real world scenarios
too where choice of programming language is such a minor
influence on performance that it is rarely even considered
significant.

Much more important are things like network speeds,
database speeds, disk access speeds and overall architecture
and design. These are the areas that give orders of
magnitude performance improvement.

Next comes algorithm design, caching models and the like.
These give multiple factors of improvement.

Finally, we come to language tweaks and choice of language
where a doubling of performance might be attainable. (Language
X being "10 times faster" than language Y rarely translates
to a non-trivial application being 10 times faster!)

Of course, it depends on the type of project and if its a high
computational scenario things will likely be different. But for
the big projects that run our lives - air traffic control,
taxation, payroll, billing, banking and so on, choice of
language comes way down the pecking order.

What's that quote about premature optimization?...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


-- 
https://mail.python.org/mailman/listinfo/python-list


Efficiency debates

2021-02-14 Thread Avi Gross via Python-list
I think we have discussed this a few times. There are tradeoffs in computer 
science and obviously a compiled language with some optimization using 
low-level data structures does much better at solving simple problems.

Interpreted languages often have serious overhead to start with and allow all 
kinds of flexibility  that comes with more costs such as functions that handle 
arbitrary types of arguments.

When languages keep adding not only features but accommodate many programming 
styles such as object-oriented and functional or multiple ways to do the same 
thing, there is often more bloat. In python, there are almost always many ways 
to do anything!

So, I find it easier to develop in such languages BUT if the program is to be 
run regularly and uses lots of resources and especially if it does not need 
many of the dynamic features, there are ways to try to tune it within Python or 
use a faster attached library of functions in languages like C for some parts 
or leaving Python entirely. If a program does not actually need much of what 
Python offers, fine. Use something else. I know in another language called R, 
that more and more parts are rewritten in a dialect of C/C++ so some things run 
quite fast if they call such functions. Users learn to use vectorized methods 
instead of loops to gain speed.

I have recently seen discussions that some tried to solve using nontrivial 
regular expressions when simpler and faster methods using normal string 
functions would do. Unless that application also needed much more, it could 
easily be done in C or C++ or if needed in a very localized assembler. 

Python may be closer to one-size fits all but sometimes more than is needed. 

But it is not the only example of where people choose a sledge-hammer when a 
thumbtack will do. But as humans, we may do better working in ways that work 
well with our normal thinking process or one we can learn and feels natural. 
Every language I know has problems that look easy to solve using features in it 
and others that are painful to contemplate. Lately, I have done work combining 
Python parts with R parts to use the powerful features I already know how to 
use well in each. The new R STUDIO supports Python and quite a few other 
languages now as well as notebooks and document preparation and more. But I 
make no claim it is fast. It is very convenient and sometimes that is worth 
more especially for a one-off project.

Those who want speed, go for it. If you can find libraries of functions or 
class prototypes that are just what you need, great. But if a language like 
Python has even more resources available in lots of modules or packages as well 
as more flexible data types, then maybe a hundred lines of that will be easier 
to write than thousands of lines elsewhere. How convenient is it to work with 
data.frame type objects in C/C++ as compared to using numpy and pandas?

For many of us, we work with machines with ever more power and most of the time 
it just sits there idling. Now if you have something on servers that is running 
many simultaneous copies all the time to process complex requests from users, 
then efficiency should matter. If you pay for it or worry about how much 
electricity it uses, fine.

This forum does not care if you prefer C/C++ and is meant to discuss Python for 
people interested in Python. Repeated attacks make people ignore the person and 
not take anything they say as meaningful. I tried sharing my understanding of a 
recent proposal and remain unconvinced it is trivial and certainly doubt that 
it is doable as it is too grandiose and suggests any imaginable computer 
language will trivially be handled. I am sure it would have fun dealing with a 
programming  language in Hebrew with most text going from right to left but 
numbers the other way, and with added features built-in that nobody is using 
yet. Oh, I have no intention of doing so, but I suggest a more doable goal 
would be to be able to handle five procedural languages with no objects or 
functional programming before making bold claims. If that works, expand.

A more efficient use of my time is to tell my mailer which messages to toss. 


-- 
https://mail.python.org/mailman/listinfo/python-list