Re: [Python-ideas] SI scale factors in Python
On Thu, Aug 25, 2016 at 08:46:54PM -0700, Ken Kundert wrote: > This idea is new to general purpose languages, For the record, at least some HP calculators include a "units" data type as part of the programming language "RPL", e.g. the HP-28 and HP-48 series. I've been using those for 20+ years so I'm quite familiar with how useful this feature can be. > but it has been used for over 40 > years in the circuit design community. Specifically, SPICE, an extremely > heavily > used circuit simulation package, introduced this concept in 1974. In SPICE > the > scale factor is honored but any thing after the scale factor is ignored. > Being > both a heavy user and developer of SPICE, I can tell you that in all that > time > this issue has never come up. In fact, the users never expected there to be > any > support for dimensional analysis, nor did they request it. I can't comment about the circuit design community, but you're trying to extrapolate from a single specialist application to a general purpose programming language used by people of many, many varied levels of expertise, of competence, with many different needs. It makes a lot of sense for applications to allow SI prefixes as suffixes within a restricted range. For example, the dd application allows the user to specify the amount of data to copy using either bytes or blocks, with optional suffixes: BLOCKS and BYTES may be followed by the following multiplicative suffixes: xM M, c 1, w 2, b 512, kB 1000, K 1024, MB 1000*1000, M 1024*1024, GB 1000*1000*1000, G 1024*1024*1024, and so on for T, P, E, Z, Y. (Quoting from the man page.) That makes excellent sense for a specialist application where numeric quantities always mean the same thing, or in this case, one of two things. As purely multiplicative suffixes, that even makes sense for Python: earlier I said that it was a good idea to add a simple module defining SI and IEC multiplicative constants in the std lib so that we could do x = 42*M or similar. But that's a far cry from allowing and ignoring units. > > Don't think of people writing code like this: > > > > result = 23mA + 75MHz > > > > which is obviously wrong. Think about them writing code like this: > > > > total = sum_resistors_in_parallel(input, extra) > > You say that '23mA + 75MHz' is obviously wrong, but it is only obviously > wrong > because the units are included, which is my point. If I had written '0.023 > + 76e6', it would not be obviously wrong. I understand your point and the benefit of dimensional analysis. But the problem is, as users of specialised applications we may be used to doing direct arithmetic on numeric literal values, with or without attached units: 23mA + 75MHz # error is visible 23 + 75 # error is hidden but as *programmers* we rarely do that. Generally speaking, it is rare to be doing arithmetic on literals where we might have the opportunity to attach a unit. We doing arithmetic on *variables* that have come from elsewhere. Reading the source code doesn't show us something that might be improved by adding a unit: # we hardly ever see this 23 + 75 # we almost always see something like this input + argument At best, we can choose descriptive variable names that hint what the correct dimensions should be: weight_of_contents + weight_of_container The argument that units would make it easier for the programmer to spot errors is, I think, invalid, because the programmer will hardly ever get to see the units. [...] > Indeed that is the point of dimensional analysis. However, despite the > availability of all these packages, they are rarely if ever used because you > have to invest a tremendous effort before they can be effective. For example, > consider the simple case of Ohms Law: > > V = R*I > > To perform dimensional analysis we need to know the units of V, R, and I. > These > are variables not literals, so some mechanism needs to be provided for > specifying the units of variables, even those that don't exist yet, like V. This is not difficult, and you exaggerate the amount of effort required. To my sorrow, I'm not actually familiar with any of the Python libraries for this, so I'll give an example using the HP-48GX RPL language. Suppose I have a value which I am expecting to be current in amperes. (On the HP calculator, it will be passed to me on the stack, but the equivalent in Python will be a argument passed to a function.) For simplicity, let's assume that if it is a raw number, I will trust that the user knows what they are doing and just convert it to a unit object with dimension "ampere", otherwise I expect some sort of unit object which is dimensionally compatible: 1_A CONV is the RPL program to perform this conversion on the top of the stack, and raise an error if the dimensions are incompatible. Converting to a more familiar Python-like API, I would expect so
Re: [Python-ideas] SI scale factors in Python
On 26 August 2016 at 08:34, Nick Coghlan wrote:
> On 26 August 2016 at 13:46, Ken Kundert
> wrote:
> > On Fri, Aug 26, 2016 at 10:14:53AM +1000, Steven D'Aprano wrote:
> >> On Thu, Aug 25, 2016 at 11:02:11AM -0700, Ken Kundert wrote:
> >>
> >> > Even if the language completely ignores the units, we have still
> gained by
> >> > allowing the units to be there, just like we gain when we allow user
> to add
> >> > comments to their code even though the compiler ignores them.
> >>
> >> This part of your proposal would be *worse*: you would fool the casual
> or
> >> naive user into believing that Python did dimensional analysis, while
> in fact
> >> not doing so. You would give them a false sense of security.
> >
> > This idea is new to general purpose languages, but it has been used for
> over 40
> > years in the circuit design community. Specifically, SPICE, an extremely
> heavily
> > used circuit simulation package, introduced this concept in 1974. In
> SPICE the
> > scale factor is honored but any thing after the scale factor is
> ignored. Being
> > both a heavy user and developer of SPICE, I can tell you that in all
> that time
> > this issue has never come up. In fact, the users never expected there to
> be any
> > support for dimensional analysis, nor did they request it.
>
> [snip]
>
> > And it has little to do with my proposal, which is basically this:
> >
> > Numbers with SI scale factor and units have become very popular. Using
> them is
> > a very common way of expressing either large or small numbers. And that
> is true
> > in the scientific and engineering communities, in the programming
> community
> > (even the linux sort command supports sorting on numbers with SI scale
> factors:
> > --human-numeric-sort), and even in popular culture.
> >
> > Python should support them. And I mean support with a capital S. I can
> come up
> > with many different hacks to support these ideas in Python today, and I
> have.
> > But this should not be a hack. This should be built into the language
> front and
> > center. It should be the preferred way that we specify and output real
> numbers.
>
> Thanks for the additional background Ken - that does start to build a
> much more compelling case.
>
> I now think there's another analogy you'll be able to draw on to make
> it even more compelling at a language design level: just because the
> *runtime* doesn't do dimensional analysis on static unit annotations
> doesn't mean that sufficiently clever static analysers couldn't do so
> at some point in the future. That then puts this proposal squarely in
> the same category as function annotations and gradual typing: semantic
> annotations that more clearly expressed developer intent, and aren't
> checked at runtime, but can be checked by a human during code review,
> and (optionally) by static analysers as a quality gate.
>
Unfortunately, I didn't read the whole thread, but it seems to me that
this would be just a more sophisticated version of NewType.
mypy type checker already supports NewType (not sure about pytype).
So that one can write (assuming PEP 526):
USD = NewType('USD', float)
EUR = NewType('EUR', float)
amount = EUR(100)
# later in code
new_amount: USD = amount # flagged as error by type checker
The same idea applies to physical units.
Of course type checkers do not know that e.g. 1m / 1s is 1 m/s, but
it is something they could be taught (for example by adding @overload
for division operator).
--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On 8/25/16, Ken Kundert wrote: [...] > Just allowing the units to be present, even it not > > retained, is a big advantage because it can bring a great deal of clarity to > the > meaning of the number. For example, even if the language does not flag an > error > when a user writes: > > vdiff = 1mV - 30uA It reminds me: "Metric mishap caused loss of NASA's Mars Climate orbiter. It could be nice to have language support helping to avoid something similar. [...] > 2. Or accept X in lieu of E. After all, the e is silent anyway. Thus, on > input >we accept ... > > 1Y -> 1e+24 > 1Z -> 1e+21 > -> 1X -> 1e+18 <- only difference > 1P -> 1e+15 Are SI prefixes frozen? Could not be safer to use E_ instead of X in case of possible new future prefixes? -- What you are proposing reminds me " [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484". Instead of adding constant type declaration. Sorry this is just really quick idea around thinking that it could be good to have parser possibility to check metric mishaps. distance1 = 1:km# or? -> distance1:length = 1:km distance2 = 1000:cm # or? -> distance2:length = 1000:cm length = distance1 + distance2 # our parser could yell :) (or compiler could translate it with warning) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On 26 August 2016 at 13:46, Ken Kundert wrote: > On Fri, Aug 26, 2016 at 10:14:53AM +1000, Steven D'Aprano wrote: >> On Thu, Aug 25, 2016 at 11:02:11AM -0700, Ken Kundert wrote: >> >> > Even if the language completely ignores the units, we have still gained by >> > allowing the units to be there, just like we gain when we allow user to add >> > comments to their code even though the compiler ignores them. >> >> This part of your proposal would be *worse*: you would fool the casual or >> naive user into believing that Python did dimensional analysis, while in fact >> not doing so. You would give them a false sense of security. > > This idea is new to general purpose languages, but it has been used for over > 40 > years in the circuit design community. Specifically, SPICE, an extremely > heavily > used circuit simulation package, introduced this concept in 1974. In SPICE the > scale factor is honored but any thing after the scale factor is ignored. > Being > both a heavy user and developer of SPICE, I can tell you that in all that time > this issue has never come up. In fact, the users never expected there to be > any > support for dimensional analysis, nor did they request it. [snip] > And it has little to do with my proposal, which is basically this: > > Numbers with SI scale factor and units have become very popular. Using them is > a very common way of expressing either large or small numbers. And that is > true > in the scientific and engineering communities, in the programming community > (even the linux sort command supports sorting on numbers with SI scale > factors: > --human-numeric-sort), and even in popular culture. > > Python should support them. And I mean support with a capital S. I can come up > with many different hacks to support these ideas in Python today, and I have. > But this should not be a hack. This should be built into the language front > and > center. It should be the preferred way that we specify and output real > numbers. Thanks for the additional background Ken - that does start to build a much more compelling case. I now think there's another analogy you'll be able to draw on to make it even more compelling at a language design level: just because the *runtime* doesn't do dimensional analysis on static unit annotations doesn't mean that sufficiently clever static analysers couldn't do so at some point in the future. That then puts this proposal squarely in the same category as function annotations and gradual typing: semantic annotations that more clearly expressed developer intent, and aren't checked at runtime, but can be checked by a human during code review, and (optionally) by static analysers as a quality gate. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On Fri, Aug 26, 2016 at 10:14:53AM +1000, Steven D'Aprano wrote: > On Thu, Aug 25, 2016 at 11:02:11AM -0700, Ken Kundert wrote: > > > Even if the language completely ignores the units, we have still gained by > > allowing the units to be there, just like we gain when we allow user to add > > comments to their code even though the compiler ignores them. > > This part of your proposal would be *worse*: you would fool the casual or > naive user into believing that Python did dimensional analysis, while in fact > not doing so. You would give them a false sense of security. This idea is new to general purpose languages, but it has been used for over 40 years in the circuit design community. Specifically, SPICE, an extremely heavily used circuit simulation package, introduced this concept in 1974. In SPICE the scale factor is honored but any thing after the scale factor is ignored. Being both a heavy user and developer of SPICE, I can tell you that in all that time this issue has never come up. In fact, the users never expected there to be any support for dimensional analysis, nor did they request it. > Don't think of people writing code like this: > > result = 23mA + 75MHz > > which is obviously wrong. Think about them writing code like this: > > total = sum_resistors_in_parallel(input, extra) You say that '23mA + 75MHz' is obviously wrong, but it is only obviously wrong because the units are included, which is my point. If I had written '0.023 + 76e6', it would not be obviously wrong. > > Some people have suggested that we take the next step and use the units for > > dimensional analysis, but that is highly problematic because you cannot do > > dimensional analysis unless everything is specified with the correct units, > > and that can be a huge burden for the user. > > What? > > That's the *whole point* of dimensional analysis: to ensure that the > user is not adding a length to a weight and then treating the result as > a time. To say that "it is too hard to specify the correct units, so we > should just ignore the units" boggles my mind. > > Any reasonable dimensional program should perform automatic unit > conversions: you can add inches to metres, but not inches to pounds. > There are already many of these available for Python. Indeed that is the point of dimensional analysis. However, despite the availability of all these packages, they are rarely if ever used because you have to invest a tremendous effort before they can be effective. For example, consider the simple case of Ohms Law: V = R*I To perform dimensional analysis we need to know the units of V, R, and I. These are variables not literals, so some mechanism needs to be provided for specifying the units of variables, even those that don't exist yet, like V. And what if the following is encountered: V = I Dimensional analysis says this is wrong, but the it may be that the resistance is simply being suppressed because it is unity. False positives of this sort are a tremendous problem with this form of automated dimensional analysis. Then there are things like this: V = 2*sin(f*t) In this case dimensional analysis (DA) indicates an error, but it is the wrong error. DA will complain about the fact that a dimensionless number is being assigned to a variable intended to carry a voltage. But in this case 2 has units of voltage, but they were not explicitly specified, so this is another false positive. The real error is the argument of the sin function. The sin function expects radians, which is dimensionless, and f*t is dimensionless, so there is no complaint, but it is not in radians, and so there should be an error. You could put a 'unit' of radians on pi, but that is not right either. Really it is 2*pi that gives you radians, and if you put radians on py and then used pi to compute the area of a unit circle, you would get pi*r^2 where r=1 meter, and the resulting units would be radians*m^2, which is nonsensical. Turns out there are many kinds of dimensionless numbers. For example, the following represents a voltage amplifier: Av = 4 # voltage gain (V/V) Ai = 0.03# current gain (A/A) Vout = Ai*Vin In this case Ai is expected to be unitless, which it is, so there is no error. However Ai is the ratio of two currents, not two voltages, so there actually should be an error. Now consider an expression that contains an arbitrary function call: V = f(I) How do we determine the units of the return value of f? Now, finally consider the BSIM4 MOSFET model equations. They are described in http://www-device.eecs.berkeley.edu/bsim/Files/BSIM4/BSIM460/doc/BSIM460_Manual.pdf If you look at this document you will find over 200 pages of extremely complicated and tedious model equations. The parameters of these models can have extremely complicated units. Well beyond anything I am proposing for real literals. For example, consider NOIA,
Re: [Python-ideas] SI scale factors in Python
On Thu, Aug 25, 2016, at 19:50, Steven D'Aprano wrote: > Historically, there are *three* different meanings for "MB", only one of > which is an official standard: > > http://physics.nist.gov/cuu/Units/binary.html The link doesn't work for me... is the third one the 1,024,000 bytes implicit in describing standard-formatted floppy disks as "1.44 MB" (they are actually 1440 bytes: 80 tracks, 2 sides, 18 512-byte sectors) or "1.2 MB" (15 sectors). ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On 26 August 2016 at 06:06, Ken Kundert wrote: > Here is a fairly typical example that illustrates the usefulness of supporting > SI scale factors and units in Python. Ken, To build a persuasive case, you'll find it's necessary to stop comparing Python-with-syntactic-support to Python-with-no-syntactic-support-and-no-third-party-libraries, and instead bring in the existing dimensional analysis libraries and tooling, and show how this change would improve *those*. That is, compare your proposed syntax *not* to plain Python code, but to Python code using some of the support libraries Steven D'Aprano mentioned, whether that's SymPy (as in http://docs.sympy.org/latest/modules/physics/unitsystems/examples.html ) or one of the other unit conversion libraries (as in http://stackoverflow.com/questions/2125076/unit-conversion-in-python ) It's also worth explicitly highlighting that the main intended beneficiaries would *NOT* be traditional software applications (where the data entry UI is clearly distinct from the source code) and instead engineers, scientists, and other data analysts using environments like Project Jupyter, where the code frequently *is* the data entry UI. And given that target audience, a further question that needs to be addressed is whether or not native syntactic support would be superior to what's already possible through Jupyter/IPython cell magics like https://bitbucket.org/birkenfeld/ipython-physics The matrix multiplication PEP (https://www.python.org/dev/peps/pep-0465/ ) is one of the best examples to study on how to make this kind of case well - it surveys the available options, explains how they all have a shared challenge with the status quo, and requests the simplest possible enabling change to the language definition to help them solve the problem. One potentially fruitful argument to pursue might be to make Python a better tool for teaching maths and science concepts at primary and secondary level (since Jupyter et al are frequently seen as introducing too much tooling complexity to be accessible at that level), but again, you'd need to explore whether or not anyone is currently using Python in that way, and what their feedback is in terms of the deficiencies of the status quo. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On Thu, Aug 25, 2016 at 11:02:11AM -0700, Ken Kundert wrote: > Once you > allow SI scale factors on numbers, the natural tendency is for people to want > to > add units, which is a good thing because it gives important information about > the number. We should allow it because it improves the code by making it more > self documenting. Even if the language completely ignores the units, we have > still gained by allowing the units to be there, just like we gain when we > allow > user to add comments to their code even though the compiler ignores them. This is dangerously wrong, and the analogy with comments is misleading. Everyone knows that comments are ignored by the interpreter, and even then, the ideal is to write self-documenting code, not comments: "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. " --Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22 This part of your proposal would be *worse*: you would fool the casual or naive user into believing that Python did dimensional analysis, while in fact not doing so. You would give them a false sense of security. Don't think of people writing code like this: result = 23mA + 75MHz which is obviously wrong. Think about them writing code like this: total = sum_resistors_in_parallel(input, extra) where the arguments may themselves have been passed to the current function as parameters from somewhere else. Or they may be data values read from a file. Their definitions may be buried deep in another part of the program. Their units aren't obvious to the reader without serious work. This part of your proposal makes the language *worse*: we lose the simple data validation that "23mA" is not a valid number, but without gaining the protection of dimensional analysis. To give an analogy, you are suggesting that we stick a sticker on the dashboard of our car saying "Airbag" but without actually installing an airbag. And you've removed the seat belt. The driver has to read the manual to learn that the "Airbag" is just a label, not an actual functioning airbag. > Some people have suggested that we take the next step and use the units for > dimensional analysis, but that is highly problematic because you cannot do > dimensional analysis unless everything is specified with the correct units, > and > that can be a huge burden for the user. What? That's the *whole point* of dimensional analysis: to ensure that the user is not adding a length to a weight and then treating the result as a time. To say that "it is too hard to specify the correct units, so we should just ignore the units" boggles my mind. Any reasonable dimensional program should perform automatic unit conversions: you can add inches to metres, but not inches to pounds. There are already many of these available for Python. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On Thu, Aug 25, 2016 at 09:03:32PM +0100, Paul Moore wrote: > That's not to say there's no room for debate here - the proposal is > interesting, and not without precedent (for example Windows Powershell > supports constants of the form 1MB, 1GB That's great! I know a few command line tools and scripts which do that, and it's really useful. > - which ironically are > computing-style 2*20 and 2*30 rather than SI-style 10*6 and 10*9). Do I remember correctly that Windows file Explorer displays disk sizes is decimal SI units? If so, how very Microsoft, to take a standard and confuse it rather than encourage it :-( Historically, there are *three* different meanings for "MB", only one of which is an official standard: http://physics.nist.gov/cuu/Units/binary.html -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On 25/08/2016 22:06, Ken Kundert wrote:
> Here is a fairly typical example that illustrates the usefulness of
> supporting
> SI scale factors and units in Python.
>
> This is simple simulation code that is used to test a current mirror driving
> an
> 100kOhm resistor ...
>
> Here is some simulation code that uses SI scale factors
>
> for delta in [-500nA, 0, 500nA]:
> input = 2.75uA + delta
> wait(1us)
> expected = 100kOhm*(2.75uA + delta)
> tolerance = 2.2mV
> fails = check_output(expected, tolerance)
> print('%s: I(in)=%rA, measured V(out)=%rV, expected V(out)=%rV,
> diff=%rV.' % (
> 'FAIL' if fails else 'pass',
> input, get_output(), expected, get_output() - expected
> ))
>
> with the output being:
>
> pass: I(in)=2.25uA, measured V(out)=226.7mV, expected V(out)=225mV,
> diff=1.7mV.
> pass: I(in)=2.75uA, measured V(out)=276.8mV, expected V(out)=275mV,
> diff=1.8mV.
> FAIL: I(in)=3.25uA, measured V(out)=327.4mV, expected V(out)=325mV,
> diff=2.4mV.
>
> And the same code in Python today ...
>
> for delta in [-5e-7, 0, 5e-7]:
> input = 2.75e-6 + delta
> wait(1e-6)
> expected = 1e5*(2.75e-6 + delta)
> tolerance = 2.2e-3
> fails = check_output(expected, tolerance)
> print('%s: I(in)=%eA, measured V(out)=%eV, expected V(out)=%eV,
> diff=%eV.' % (
> 'FAIL' if fails else 'pass',
> input, get_output(), expected, get_output() - expected
> ))
>
> with the output being:
>
> pass: I(in)=2.25e-6A, measured V(out)=226.7e-3V, expected V(out)=225e-3V,
> diff=1.7e-3V.
> pass: I(in)=2.75e-6A, measured V(out)=276.8e-3V, expected V(out)=275e-3V,
> diff=1.8e-3V.
> FAIL: I(in)=3.25e-6A, measured V(out)=327.4e-3V, expected V(out)=325e-3V,
> diff=2.4e-3V.
>
> There are two things to notice. In the first example the numbers are easier to
> read: for example, 500nA is easier to read the 5e-7. Second, there is
> information in the units that provides provides useful information. One can
> easily see that the input signal is a current and that the output is a
> voltage.
> Furthermore, anybody can look at this code and do a simple sanity check on
> the
> expressions even if they don't understand the system being simulated. They
> can
> check the units on the input and output expressions to assure that they are
> all
> consistent.
>
> -Ken
> ___
>
And the same code with python today
def wait(delay):
pass
def check_output(expected, tolerance):
return False
def get_output():
return 1*uA
def f():
for delta in [-500*nA, 0, 500*nA]:
input = 2.75*uA + delta
wait(1*us)
expected = 100*kOhm*(2.75*uA + delta)
tolerance = 2.2*mV
fails = check_output(expected, tolerance)
print('%s: I(in)=%rA, measured V(out)=%rV, expected V(out)=%rV,
diff=%rV.' % (
'FAIL' if fails else 'pass',
input, get_output(), expected, get_output() - expected
))
f()
pass: I(in)=2.25e-05A, measured V(out)=1e-05V, expected V(out)=22.5V,
diff=-22.4V.
pass: I(in)=2.75e-05A, measured V(out)=1e-05V, expected V(out)=27.5V,
diff=-27.4V.
pass: I(in)=3.2504e-05A, measured V(out)=1e-05V, expected
V(out)=32.51V, diff=-32.404V.
Do you really see fundamental difference with your original code?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On Fri, Aug 26, 2016 at 6:06 AM, Ken Kundert
wrote:
> Here is some simulation code that uses SI scale factors
>
> for delta in [-500nA, 0, 500nA]:
> input = 2.75uA + delta
> wait(1us)
> expected = 100kOhm*(2.75uA + delta)
> tolerance = 2.2mV
> fails = check_output(expected, tolerance)
> print('%s: I(in)=%rA, measured V(out)=%rV, expected V(out)=%rV,
> diff=%rV.' % (
> 'FAIL' if fails else 'pass',
> input, get_output(), expected, get_output() - expected
> ))
>
> And the same code in Python today ...
>
> for delta in [-5e-7, 0, 5e-7]:
> input = 2.75e-6 + delta
> wait(1e-6)
> expected = 1e5*(2.75e-6 + delta)
> tolerance = 2.2e-3
> fails = check_output(expected, tolerance)
> print('%s: I(in)=%eA, measured V(out)=%eV, expected V(out)=%eV,
> diff=%eV.' % (
> 'FAIL' if fails else 'pass',
> input, get_output(), expected, get_output() - expected
> ))
Rename a few things:
for deltaA in [-5e-7, 0, 5e-7]:
inputA = 2.75e-6 + deltaA
wait(1e-6)
expectedA = 1e5*(2.75e-6 + deltaA)
toleranceV = 2.2e-3
fails = check_output(expectedA, toleranceV)
print('%s: I(in)=%eA, measured V(out)=%eV, expected
V(out)=%eV, diff=%eV.' % (
'FAIL' if fails else 'pass',
inputA, get_output(), expectedA, get_output() - expectedA
))
I may have something wrong here (for instance, I'm not sure if
check_output should be taking an expected amperage and a tolerance in
volts), but you get the idea: it's the variables, not the literals,
that get tagged.
Another way to do this would be to use MyPy and type hinting to create
several subtypes of floating-point number:
class V(float): pass
class A(float): pass
for delta in [A(-5e-7), A(0), A(5e-7)]:
input = A(2.75e-6) + delta
# etc
or in some other way have static analysis tag the variables, based on
their origins.
> There are two things to notice. In the first example the numbers are easier to
> read: for example, 500nA is easier to read the 5e-7.
This is useful, but doesn't depend on the "A" at the end.
> Second, there is
> information in the units that provides provides useful information. One can
> easily see that the input signal is a current and that the output is a
> voltage.
> Furthermore, anybody can look at this code and do a simple sanity check on the
> expressions even if they don't understand the system being simulated. They can
> check the units on the input and output expressions to assure that they are
> all
> consistent.
These two are better served by marking the variables rather than the
literals; in fact, if properly done, the tagging can be verified by a
program, not just a human. (That's what tools like MyPy are aiming to
do.)
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
Here is a fairly typical example that illustrates the usefulness of supporting
SI scale factors and units in Python.
This is simple simulation code that is used to test a current mirror driving an
100kOhm resistor ...
Here is some simulation code that uses SI scale factors
for delta in [-500nA, 0, 500nA]:
input = 2.75uA + delta
wait(1us)
expected = 100kOhm*(2.75uA + delta)
tolerance = 2.2mV
fails = check_output(expected, tolerance)
print('%s: I(in)=%rA, measured V(out)=%rV, expected V(out)=%rV,
diff=%rV.' % (
'FAIL' if fails else 'pass',
input, get_output(), expected, get_output() - expected
))
with the output being:
pass: I(in)=2.25uA, measured V(out)=226.7mV, expected V(out)=225mV,
diff=1.7mV.
pass: I(in)=2.75uA, measured V(out)=276.8mV, expected V(out)=275mV,
diff=1.8mV.
FAIL: I(in)=3.25uA, measured V(out)=327.4mV, expected V(out)=325mV,
diff=2.4mV.
And the same code in Python today ...
for delta in [-5e-7, 0, 5e-7]:
input = 2.75e-6 + delta
wait(1e-6)
expected = 1e5*(2.75e-6 + delta)
tolerance = 2.2e-3
fails = check_output(expected, tolerance)
print('%s: I(in)=%eA, measured V(out)=%eV, expected V(out)=%eV,
diff=%eV.' % (
'FAIL' if fails else 'pass',
input, get_output(), expected, get_output() - expected
))
with the output being:
pass: I(in)=2.25e-6A, measured V(out)=226.7e-3V, expected V(out)=225e-3V,
diff=1.7e-3V.
pass: I(in)=2.75e-6A, measured V(out)=276.8e-3V, expected V(out)=275e-3V,
diff=1.8e-3V.
FAIL: I(in)=3.25e-6A, measured V(out)=327.4e-3V, expected V(out)=325e-3V,
diff=2.4e-3V.
There are two things to notice. In the first example the numbers are easier to
read: for example, 500nA is easier to read the 5e-7. Second, there is
information in the units that provides provides useful information. One can
easily see that the input signal is a current and that the output is a voltage.
Furthermore, anybody can look at this code and do a simple sanity check on the
expressions even if they don't understand the system being simulated. They can
check the units on the input and output expressions to assure that they are all
consistent.
-Ken
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On 25 August 2016 at 19:02, Ken Kundert wrote:
> This proposal basically has two parts. One part is that Python should
> naturally support printing real numbers with SI scale factors. Currently
> there
> are three formats for printing real number: %f, %d, %g. They all become
> difficult to read for even moderately large or small numbers. Exponential
> notation is hard for humans to read. That is why SI scale factors have largely
> replaced exponential notation everywhere except in programming. Adding
> another
> format for printing real numbers in human readable form seems like a modest
> extension that is long past due in all programming languages. I am asking that
> Python be the leader here. I am sure other languages will pick it up once it
> is
> implemented in Python.
This part would be easy to implement as a small PyPI module, with a
formatting function siformat(num) -> str. If that proved popular (and
it well might) then there's more of a case for adding it as a language
feature, Maybe as a custom string formatting code, or maybe just as a
standard library function. But without that sort of real-world
experience, it's likely that any proposal will get bogged down in
"what if" theoretical debate.
> The second part is the logical dual to the first: input. People should be able
> to enter numbers in Python using SI scale factors. This means as real
> literals,
> such as 2.4G, but it should also work with casting, float('2.4G'). Once you
> allow SI scale factors on numbers, the natural tendency is for people to want
> to
> add units, which is a good thing because it gives important information about
> the number. We should allow it because it improves the code by making it more
> self documenting. Even if the language completely ignores the units, we have
> still gained by allowing the units to be there, just like we gain when we
> allow
> user to add comments to their code even though the compiler ignores them.
This is much more problematic. Currently, even the long-established
decimal and rational types don't enjoy syntactic support, so I'd be
surprised if SI scale factors got syntax support first. But following
their lead, by (again) starting with a conversion function along the
lines of SI("3k") -> 3000 would be a good test of applicability. It
could easily go in the same module as you're using to trial the string
formatting function above.
I'd expect that a function to do this conversion would be a good way
of thrashing out the more controversial aspects of the proposal -
whether E means "exponent" or "exa", whether M and G get
misinterpreted as computing-style 2**20 and 2**30, etc. Having real
world experience of how to solve these questions would be invaluable
in moving forward with a proposal to add language support.
> Some people have suggested that we take the next step and use the units for
> dimensional analysis, but that is highly problematic because you cannot do
> dimensional analysis unless everything is specified with the correct units,
> and
> that can be a huge burden for the user. So instead, I am suggesting that we
> provide simple hooks that simply allow access to the units. That way people
> can
> build dimensional analysis packages using the units if they felt the need.
Dimensional analysis packages already exist (I believe) and they don't
rely on syntactic support. Any proposal to add something to the
language *really* needs to demonstrate that
1. Those packages are currently suffering from the lack of language support.
2. The proposed change would allow them to resolve existing problems
that they haven't been able to address any other way.
3. The proposed change isn't some sort of "attractive nuisance" for
naive users, leading them to think they can write dimensionally
correct programs *without* using one of the existing packages.
Python has a track record of being open to adding syntactic support if
it demonstrably helps 3rd party tools (for example, the matrix
multiplication operator was added specifically to help the numeric
Python folks address a long-standing issue they had), so this is a
genuine possibility - but such proposals need support from the groups
they are intended to help. At the moment, I'm not even aware of a
particular "dimensional analysis with Python" community, or any
particular "best of breed" package in this area that might lead such a
proposal - and a language change of this nature probably does need
that sort of backing.
That's not to say there's no room for debate here - the proposal is
interesting, and not without precedent (for example Windows Powershell
supports constants of the form 1MB, 1GB - which ironically are
computing-style 2*20 and 2*30 rather than SI-style 10*6 and 10*9). But
there's a pretty high bar for a language change like this, and it's
worth doing the groundwork to avoid wasting a lot of time on something
that's not going to be accepted in its current form.
Hope this helps,
Paul
_
Re: [Python-ideas] Adding optional parameter to shutil.rmtree to not delete root.
> On Aug 25, 2016, at 3:27 PM, Nick Jacobson via Python-ideas > wrote: > > +1 for clear_dir() > > I agree that there's no other obvious meaning that it could have. +1, but I think "cleardir" would better match naming conventions in the shutil module. (My personal rule of thumb on the use of underscores in function names is to omit them if any name component is abbreviated. So either spell it out as clear_directory or shorten as cleardir.)___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Adding optional parameter to shutil.rmtree to not delete root.
+1 for clear_dir() I agree that there's no other obvious meaning that it could have. On Thursday, August 25, 2016 9:44 AM, Nick Coghlan wrote: On 25 August 2016 at 23:14, Victor Stinner wrote: > Maybe add a different function rather add a flag? Something like > shutil.remove_dir_files(). The typical Python term for the concept would be "clear", giving shutil.clear_dir(). Like the ".clear()" method on containers, it would delete the contents, but leave the container itself alone. rmtree() could then be a thin wrapper around clear_dir() that also deletes the base directory. Alternatively, if we wanted to stick with the "rm" prefix, we could use "shutil.rmcontents()" as the name. The main downside I'd see to that is that I'd half expect it to work on files as well (truncating them to a length of zero), while clear_dir() is unambiguous. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Atomic counter / atomic increment
The only reason I can think of for wanting this simple class in the stdlib would be that on GIL-free Python implementations you could replace it with a lock-free version. But I think we'd probably want to rethink a bunch of other datastructures to be lock-free, and a 3rd party package on PyPI makes more sense to me than jumping right into the stdlib. On Thu, Aug 25, 2016 at 11:10 AM, Ben Hoyt wrote: > > As long as Python uses a GIL to protect C level function >> calls, you can use an iterator for this: >> >> import itertools >> x = itertools.count() >> ... >> mycount = next(x) >> > > Yeah, that's a neat hack -- I saw it recommended on StackOverflow, and saw > it used in the standard library somewhere. I think that's probably okay in > the *CPython* stdlib, because it's CPython so you know it has the GIL. But > this wouldn't work in other Python implementations, would it (IronPython > and Jython don't have a GIL). Or when itertools.count() is implemented in > pure Python on some system? Seems like it could blow up in someone's face > when they're least expecting it. I also think using *iter*tools is a pretty > non-obvious way to get a thread-safe counter. > > -Ben > > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Atomic counter / atomic increment
On 25.08.2016 20:10, Ben Hoyt wrote: >> As long as Python uses a GIL to protect C level function >> calls, you can use an iterator for this: >> >> import itertools >> x = itertools.count() >> ... >> mycount = next(x) >> > > Yeah, that's a neat hack -- I saw it recommended on StackOverflow, and saw > it used in the standard library somewhere. I think that's probably okay in > the *CPython* stdlib, because it's CPython so you know it has the GIL. But > this wouldn't work in other Python implementations, would it (IronPython > and Jython don't have a GIL). Or when itertools.count() is implemented in > pure Python on some system? Seems like it could blow up in someone's face > when they're least expecting it. I also think using *iter*tools is a pretty > non-obvious way to get a thread-safe counter. All true. Having an implementation in threading which hides away the details would be nice. On CPython using iterators would certainly be one of the most efficient ways of doing this. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Aug 25 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Atomic counter / atomic increment
> As long as Python uses a GIL to protect C level function > calls, you can use an iterator for this: > > import itertools > x = itertools.count() > ... > mycount = next(x) > Yeah, that's a neat hack -- I saw it recommended on StackOverflow, and saw it used in the standard library somewhere. I think that's probably okay in the *CPython* stdlib, because it's CPython so you know it has the GIL. But this wouldn't work in other Python implementations, would it (IronPython and Jython don't have a GIL). Or when itertools.count() is implemented in pure Python on some system? Seems like it could blow up in someone's face when they're least expecting it. I also think using *iter*tools is a pretty non-obvious way to get a thread-safe counter. -Ben ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
All,
This proposal basically has two parts. One part is that Python should
naturally support printing real numbers with SI scale factors. Currently there
are three formats for printing real number: %f, %d, %g. They all become
difficult to read for even moderately large or small numbers. Exponential
notation is hard for humans to read. That is why SI scale factors have largely
replaced exponential notation everywhere except in programming. Adding another
format for printing real numbers in human readable form seems like a modest
extension that is long past due in all programming languages. I am asking that
Python be the leader here. I am sure other languages will pick it up once it is
implemented in Python.
The second part is the logical dual to the first: input. People should be able
to enter numbers in Python using SI scale factors. This means as real literals,
such as 2.4G, but it should also work with casting, float('2.4G'). Once you
allow SI scale factors on numbers, the natural tendency is for people to want
to
add units, which is a good thing because it gives important information about
the number. We should allow it because it improves the code by making it more
self documenting. Even if the language completely ignores the units, we have
still gained by allowing the units to be there, just like we gain when we allow
user to add comments to their code even though the compiler ignores them.
Some people have suggested that we take the next step and use the units for
dimensional analysis, but that is highly problematic because you cannot do
dimensional analysis unless everything is specified with the correct units, and
that can be a huge burden for the user. So instead, I am suggesting that we
provide simple hooks that simply allow access to the units. That way people
can
build dimensional analysis packages using the units if they felt the need.
-Ken
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Atomic counter / atomic increment
On 25.08.2016 19:31, Ben Hoyt wrote: > I had to implement a simple atomic counter the other day to count the total > number of requests processed in a multi-threaded Python web server. > > I was doing a demo of "how cool Python is" to my colleagues, and they were > generally wowed, but one of the things that made them do a double-take > (coming mostly from Scala/Java) was that there was no atomic counter in the > standard library. > > The fact that I and many other folks have implemented such things makes me > wonder if it should be in the standard library. As long as Python uses a GIL to protect C level function calls, you can use an iterator for this: import itertools x = itertools.count() ... mycount = next(x) ... The trick here is that the CALL_FUNCTION byte code will trigger the increment of the iterator. Since this is implemented in C, the GIL will serve as lock on the iterator while it is being incremented. With Python 4.0, this will all be different, though :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Aug 25 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Atomic counter / atomic increment
I had to implement a simple atomic counter the other day to count the total number of requests processed in a multi-threaded Python web server. I was doing a demo of "how cool Python is" to my colleagues, and they were generally wowed, but one of the things that made them do a double-take (coming mostly from Scala/Java) was that there was no atomic counter in the standard library. The fact that I and many other folks have implemented such things makes me wonder if it should be in the standard library. It's pretty simple to implement, basically the handful of lines of code below (full version on GitHub Gist at https://gist.github.com/benhoyt/8c8a8d62debe8e5aa5340373f9c509c7): import threading class AtomicCounter: def __init__(self, initial=0): self.value = initial self._lock = threading.Lock() def increment(self, num=1): with self._lock: self.value += num return self.value And if you just want a one-off and don't want to write a class, it's like so: import threading counter_lock = threading.Lock() counter = 0 with counter_lock: counter += 1 value = counter print(value) But it could be this much more obvious code: import threading counter = threading.AtomicCounter() value = counter.increment() print(value) Thoughts? Would such a class make a good candidate for the standard library? (API could probably be improved.) -Ben ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Adding optional parameter to shutil.rmtree to not delete root.
On 25 August 2016 at 23:14, Victor Stinner wrote: > Maybe add a different function rather add a flag? Something like > shutil.remove_dir_files(). The typical Python term for the concept would be "clear", giving shutil.clear_dir(). Like the ".clear()" method on containers, it would delete the contents, but leave the container itself alone. rmtree() could then be a thin wrapper around clear_dir() that also deletes the base directory. Alternatively, if we wanted to stick with the "rm" prefix, we could use "shutil.rmcontents()" as the name. The main downside I'd see to that is that I'd half expect it to work on files as well (truncating them to a length of zero), while clear_dir() is unambiguous. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 525: Asynchronous Generators
Hi Yury et al,
Apologies for not participating in this discussion before -- unfortunately
I've been dealing with some boring medical issues and haven't been able to
contribute as much as early as I should have... But I do have two concerns,
one minor and one not.
Minor comment: this is purely an aesthetic issue, but the PEP should
probably have some discussion of the choice between "async yield" vs bare
"yield" as the syntax. Arguments for "async yield" would be consistency
with the other async syntax ("async for" etc.) and potentially reduced
confusion about how inside an async def, "await" yields, and "yield"
yields, but in totally different senses. Arguments for "yield" would be
terseness, I guess. Probably this is just something for Guido to pick...
Major issue: I have serious concerns about the whole async finalizer design.
1) there is a very general problem of how to handle cleanup in async code,
which isn't specific to async generators at all -- really any object that
holds resources that need async cleanup has exactly the same issue. So the
special purpose solution here makes me uncomfortable.
2) our understanding of this problem (resource cleanup in async code) is
*really* immature. You can check the async-sig archives for some discussion
-- I think it's safe to say that all the "experts" are extremely confused
about this problem's shape/scope/best practices. I guess probably there are
10ish people who kind of understand the problem and 0 people who deeply
understand it. (I was working on at least writing something up try and make
the issue more accessible, but have not managed yet, which I feel bad about
:-(.) But given this, trying to freeze one particular strategy into the
language on the 3.6 timeline raises a red flag for me.
3) specifically, I don't understand how the proposed solution in the pep
can work reliably in the presence of non-prompt garbage collection (e.g. in
the presence of cycles on cpython, or on pypy in general). What happens
when the async generator object gets collected after the event loop exits?
We could require event loops to force a full collection before exiting, but
this is a rather uncomfortable coupling! (Do all python implementations
even have a way to force a guaranteed-to-collect-ALL-garbage collection?
What are the performance implications of async code triggering such full
collections at inopportune times?) And even this didn't necessarily solve
the problem, since async generators frames could still be "live" from the
GC point of view, even if we know that their resources should be released.
(E.g. consider an exception thrown from an async gen that propagates out of
the event loop, so the event loop exits while the async gen's frame is
pinned by the traceback object.)
Python in general has been moving towards more explicit/deterministic
cleanup through "with" statements, and I think this is great. My feeling is
that we should try to continue in this vein, rather than jumping through
hoops to let people get away with sloppy coding that relies on __del__ and
usually works except when it doesn't.
For example, what if we added an optional __abreak__ coroutine to the async
iterator protocol, with the semantics that an "async for" will always
either iterate to exhaustion, or else call __abreak__, i.e.
# new style code
async for x in aiter:
...
becomes sugar for
# 3.5 style code
try:
async for x in aiter:
...
finally:
if for_loop_exited_early and hasattr(aiter, "___abreak__"):
await aiter.__abreak__(...)
Basically the idea here is that currently, what we should *really* be doing
if we care about reliable+deterministic resource cleanup is never writing
bare "for" loops but instead always writing
async with ... as aiter:
async for ... in aiter:
...
But that's really tiresome and no one does it, so let's build the "with"
into the for loop.
Of course there are cases where you want to reuse a single iterator object
in multiple loops; this would look like:
# doing something clever, so we have to explicitly take charge of cleanup
ourselves:
async with ... as aiter:
# read header (= first non-comment line)
async for header in iterlib.unclosing(aiter):
if not header.startswith("#"):
break
# continue with same iterator to read body
async for line in iterlib.unclosing(aiter):
...
where iterlib.unclosing returns a simple proxy object that passes through
__anext__ and friends but swallows __abreak__ to make it a no-op.
Or maybe not, this isn't a fully worked out proposal :-). My main concern
is that we don't rush to make a decision for 3.6, skip the hard work of
considering different designs, and end up with something we regret in
retrospect.
Some other options to consider:
- add async generators to 3.6, but defer the full cleanup discussion for
now -- so in 3.6 people will have to use "async with aclosing(aiter): ..."
or whatever, but at least we can start getting experience w
Re: [Python-ideas] Adding optional parameter to shutil.rmtree to not delete root.
Maybe add a different function rather add a flag? Something like shutil.remove_dir_files(). Victor Le 25 août 2016 4:32 AM, "Nick Jacobson via Python-ideas" < python-ideas@python.org> a écrit : > I've been finding that a common scenario is where I want to remove > everything in a directory, but leave the (empty) root directory behind, not > removing it. > > So for example, if I have a directory C:\foo and it contains subdirectory > C:\foo\bar and file C:\foo\myfile.txt, and I want to remove the > subdirectory (and everything in it) and file, leaving only C:\foo behind. > > (This is useful e.g. when the root directory has special permissions, so > it wouldn't be so simple to remove it and recreate it again.) > > A number of ways to do this have been offered here: > http://stackoverflow.com/questions/185936/delete-folder-contents-in-python > > But it'd be simpler if there were an optional parameter added to > shutil.rmtree, called removeroot. It would default to true so as to not > break any backward compatibility. If it's set to false, then it leaves the > root directory in place. > > Thanks, > > Nick > > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On Thu, Aug 25, 2016 at 9:24 PM, Steven D'Aprano wrote: > If I try to add 30 feet to 6 metres and get either 36 feet or 36 metres, > then your unit system is *worse* than useless, it is actively harmful. I > don't mind if I get 15.144 metres or 49.685039 feet or even > 5.0514947e-08 lightseconds, but I better not get 36 of anything. > To be fair, under the OP's proposal, you wouldn't get 36 *of anything* - you'd just get the bare number 36. So it's not worse than useless, just useless. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On Wed, Aug 24, 2016 at 09:28:03PM -0700, Ken Kundert wrote: > All, > I propose that support for SI scale factors be added to Python. I think there's something to be said for these ideas, but you are combining multiple ideas into one suggestion. First off, units with quantities: I think that is an excellent idea, but one best supported by a proper unit library that supports more than just SI units. There are already a few of those. See for example this Stackoverflow question: http://stackoverflow.com/questions/2125076/unit-conversion-in-python Sympy also does dimensional analysis: http://docs.sympy.org/latest/modules/physics/unitsystems/examples.html Google for more. If I try to add 30 feet to 6 metres and get either 36 feet or 36 metres, then your unit system is *worse* than useless, it is actively harmful. I don't mind if I get 15.144 metres or 49.685039 feet or even 5.0514947e-08 lightseconds, but I better not get 36 of anything. And likewise for adding 30 kilograms to 6 metres. That has to be an error, or this system will just be an attractive nuisance, luring people into a false sense of security while actually not protecting them from dimensional and unit conversion bugs at all. So I am an extremely strong -1 to the suggestion that we allow unit suffixes on numeric quantities but treat them as a no-op. Should Python support a unit conversion library in the standard library? I think perhaps not -- there's plenty of competition in the unit conversion ecosystem, both in Python and out of it, and I don't think that there's any one library that is both sufficiently "best of breed" enough and stable enough to put into the std lib. Remember that the std lib is where good libraries go to die: once they hit the std lib, stability becomes much, much more important than new features. But if you wish to argue differently, I'll be willing to hear your suggestions. Now, on to the second part of the suggestion: support for SI prefixes. I think this is simple enough, and useful enough, that we could make it part of the std lib -- and possibly even in 3.6 (possibly under a provisional basis). The library could be dead simple: # prefixes.py # Usage: # from prefixes import * # x = 123*M # like 12300 # y = 45*Ki # like 45*1024 # SI unit prefixes # http://physics.nist.gov/cuu/Units/prefixes.html Y = yotta = 10**24 Z = zetta = 10**21 [...] k = kilo = 10**3 K = k # not strictly an SI prefix, but common enough to allow it m = milli = 1e-3 µ = micro = 1e-6 # A minor PEP-8 violation, but (I hope) forgiveable. u = µ # not really an SI prefix, but very common # etc # International Electrotechnical Commission (IEC) binary prefixes # http://physics.nist.gov/cuu/Units/binary.html Ki = kibi = 1024 Mi = mibi = 1024**2 Gi = 1024**3 # etc That's practically it. Of course, this simple implementation would allow usage that was a technical violation of the SI system: x = 45*M*µ*Ki as well as usage that is semantically meaningless: x = 45 + M but those abuses are best covered by "consenting adults". (In other words, if you don't like it, don't do it.) And it wouldn't support the obsolete binary prefixes that use SI symbols with binary values (K=1024, M=1024**2, etc), but that's a good thing. They're an abomination that need to die as soon as possible. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
I have trouble imagining situations in which SI units would help readability over scientific notation. On the one hand reading it always involves mental conversion to the exponents, so this is just an extra step. E.g. this equality is evident at a glance: 1e3 * 1e6 * 1e-9 == 1 This one would take me a few seconds, and I'd need to run it twice in my head to make sure I wasn't doing it wrong: 1k * 1M * 1n == 1 The more complicated the expression, the bigger the cognitive advantage for numeric exponents. I think the OP is drawn astray by the utility of domain specific familiar dimensional units. 1MeV means something in a domain. So does 1ps. Or 1km. But really it's the units not the exponent that are helping us (plus knowing that our specific domain deals with dimensional quantities of a certain scale). But even past the ease of mental arithmetic, I don't see why one would use literals very often to start with. If they are constants, give them a better name! If they are variables, likewise. If it helps your domain, add the units to the *names*, e.g.: lightyear_m = 9.5e15 galaxy_radians = calculate_arc() dist_m = lighyear_m * galaxy_radians * arc_len That looks a lot clearer to me than sticking together raw numbers, however spelled. On Aug 25, 2016 2:56 AM, "Joao S. O. Bueno" wrote: > On 25 August 2016 at 06:06, Paul Moore wrote: > > On 25 August 2016 at 09:54, Ken Kundert > wrote: > >> 1G -> 1e+09 > >> 1M -> 1e+06 > >> 1k -> 1e+03 > > > > While these suffixes are suitable for a scientific context, in a > > computing context, 1k=1024, 1M=1024*1024 and 1G=1024*1024*1024 make > > just as much, if not more, sense (and yes, I'm aware of Gigabyte vs > > Gibibyte). > > > > If "1M" were a legal Python literal,. I would expect a lot of > > confusion over what it meant - to the extent that it would hurt > > readability badly. > > Paul > > > So, the idea of adding fixed sufixes to the core language I regard as > awful - due to these and other considerations - > But maybe, oen thign to think about is about "operatorless" > multiplication - jsut puting two tokens side by side, which currently > yieds a Syntax Error could call `__mul__` > (or a new `__direct_mul__` method on the second operator. > > That would enable a lot of interesting things in already existing > packages like SymPy - and would allow a "physics measurements" > packages that would take care of the ideas on the starting of the > thread. > > I certainly would be more confortable for mathematicians and other > people using Python in interactive environments such as iPython > notebooks. > > But other than having this as a multiplication, I am against the whole > thing. > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On Thu, Aug 25, 2016 at 6:19 PM, Ken Kundert wrote: >> So you can have 1000mm or 0.001km but not 1m? > > If the scale factor is optional, then numbers like 1m are problematic because > the m can represent either milli or meter. This is resolved by requiring the > scale factor and defining a unity scale factor. I propose '_'. So 1m > represents > milli and 1_m represents 1 meter. This could also be hashed out using a constructor-only API. You'll probably want to avoid '_', as it's just been added as a comma separator for numeric literals. >> If units are retained, what you have is no longer a simple number, but >> a value with a unit, and is a quite different beast. (For instance, >> addition would have to cope with unit mismatches (probably by throwing >> an error), and multiplication would have to combine the units (length >> * length = area).) That would be a huge new feature. > > Indeed. I am not proposing that anything be done with the units other than > possibly retain them for later output. Doing dimensional analysis on > expressions > would be a huge burden both for those implementing the language and for those > using them in a program. Just allowing the units to be present, even it not > retained, is a big advantage because it can bring a great deal of clarity to > the > meaning of the number. For example, even if the language does not flag an > error > when a user writes: > > vdiff = 1mV - 30uA > > the person that wrote the line will generally see it as a problem and fix it. How often do you do arithmetic on literals like that? More likely, what you'd do is tag your variable names, so it'll be something like: input_volts = 1m#V inefficiency = 30u#A vdiff = input_volts - inefficiency > In my experience, providing units is the most efficient form of documentation > available in numerical programming in the sense that one or two additional > characters can often clarify otherwise very confusing code. > > My feeling is that retaining the units on real literals is of little value if > you don't also extend the real variable type to hold units, or to create > another > variable type that would carry the units. Extending reals does not seem like > a good idea, but creating a new type, quantity, seems practical. In this case, > the units would be rather ephemeral in that they would not survive any > operation. Thus, the result of an operation between a quantity and either > a integer, real or quantity would always be a real, meaning that the units are > lost. In this way, units are very light-weight and only really serve as > documentation (for both programmers and end users). > > But this idea of retaining the units is the least important aspect of this > proposal. The important aspects are: > 1. It allows numbers to be entered in a clean form that is easy to type and > easy >to interpret > 2. It allows numbers to be output in a clean form that is easy to interpret. > 3. In many cases it allows units to be inserted into the code in a very > natural >and clean way to improve the clarity of the code. The decimal.Decimal and fractions.Fractions types have no syntactic support. I would suggest imitating their styles initially, sorting out all the details of which characters mean what, and appealing for syntax once it's all settled - otherwise, it's too likely that something will end up being baked into the language half-baked, if that makes any sense. >> Question, though: What happens with exa-? Currently, if the parser >> sees "1E", it'll expect to see another number, eg 1E+1 == 10.0. Will >> this double meaning cause confusion? > > Oh, I did not see this. Both SPICE and Verilog limit the scale factors to the > common ones (T, G, M, k, _, m, u, n, p, f, a). I work in electrical > engineering, > and in that domain exa never comes up. My suggestion would be to limit > ourselves > to the common scale factors as most people know them. Using P, E, Z, Y, z, and > y often actually works against us as most people are not familiar with them > and > so cannot interpret them easily. That seems pretty reasonable. At very least, it'd be something that can be extended later. Even femto and atto are rare enough that they could be dropped if necessary (pico too, perhaps). Easy scaling seems general enough to include in the language. Tagging numbers with units, though, feels like the domain of a third-party library. Maybe I'm wrong. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On 25 August 2016 at 06:06, Paul Moore wrote: > On 25 August 2016 at 09:54, Ken Kundert wrote: >> 1G -> 1e+09 >> 1M -> 1e+06 >> 1k -> 1e+03 > > While these suffixes are suitable for a scientific context, in a > computing context, 1k=1024, 1M=1024*1024 and 1G=1024*1024*1024 make > just as much, if not more, sense (and yes, I'm aware of Gigabyte vs > Gibibyte). > > If "1M" were a legal Python literal,. I would expect a lot of > confusion over what it meant - to the extent that it would hurt > readability badly. > Paul So, the idea of adding fixed sufixes to the core language I regard as awful - due to these and other considerations - But maybe, oen thign to think about is about "operatorless" multiplication - jsut puting two tokens side by side, which currently yieds a Syntax Error could call `__mul__` (or a new `__direct_mul__` method on the second operator. That would enable a lot of interesting things in already existing packages like SymPy - and would allow a "physics measurements" packages that would take care of the ideas on the starting of the thread. I certainly would be more confortable for mathematicians and other people using Python in interactive environments such as iPython notebooks. But other than having this as a multiplication, I am against the whole thing. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
On 25 August 2016 at 09:54, Ken Kundert wrote: > 1G -> 1e+09 > 1M -> 1e+06 > 1k -> 1e+03 While these suffixes are suitable for a scientific context, in a computing context, 1k=1024, 1M=1024*1024 and 1G=1024*1024*1024 make just as much, if not more, sense (and yes, I'm aware of Gigabyte vs Gibibyte). If "1M" were a legal Python literal,. I would expect a lot of confusion over what it meant - to the extent that it would hurt readability badly. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
> Question, though: What happens with exa-? Currently, if the parser > sees "1E", it'll expect to see another number, eg 1E+1 == 10.0. Will > this double meaning cause confusion? Allow me to refine my answer to this question ... Yes, that is definitely problematic. I see two possible solutions. 1. Limit ourselves to the common scale factors: T, G, M, k, _, m, u, n, p, f, a 2. Or accept X in lieu of E. After all, the e is silent anyway. Thus, on input we accept ... 1Y -> 1e+24 1Z -> 1e+21 -> 1X -> 1e+18 <- only difference 1P -> 1e+15 1T -> 1e+12 1G -> 1e+09 1M -> 1e+06 1k -> 1e+03 1_ -> 1e+00 1m -> 1e-03 1u -> 1e-06 1n -> 1e-09 1p -> 1e-12 1f -> 1e-15 1a -> 1e-18 1z -> 1e-21 1y -> 1e-24 But on output we use ... 1Y -> 1e+24 optional 1Z -> 1e+21 optional -> 1E -> 1e+18 optional 1P -> 1e+15 optional 1T -> 1e+12 1G -> 1e+09 1M -> 1e+06 1k -> 1e+03 1_ -> 1e+00 1m -> 1e-03 1u -> 1e-06 1n -> 1e-09 1p -> 1e-12 1f -> 1e-15 1a -> 1e-18 1z -> 1e-21 optional 1y -> 1e-24 optional The optional scale factors are unfamiliar to most people, and if used might result in harder to read numbers. So I propose that '%r' only outputs the common scale factors, and %R outputs all the scale factors. Or we can use '#' in the format string to indicate the 'alternate' form should be used, in this case 'alternate' means that the extended set of scale factors should be used. -Ken ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] SI scale factors in Python
> So you can have 1000mm or 0.001km but not 1m? If the scale factor is optional, then numbers like 1m are problematic because the m can represent either milli or meter. This is resolved by requiring the scale factor and defining a unity scale factor. I propose '_'. So 1m represents milli and 1_m represents 1 meter. > If units are retained, what you have is no longer a simple number, but > a value with a unit, and is a quite different beast. (For instance, > addition would have to cope with unit mismatches (probably by throwing > an error), and multiplication would have to combine the units (length > * length = area).) That would be a huge new feature. Indeed. I am not proposing that anything be done with the units other than possibly retain them for later output. Doing dimensional analysis on expressions would be a huge burden both for those implementing the language and for those using them in a program. Just allowing the units to be present, even it not retained, is a big advantage because it can bring a great deal of clarity to the meaning of the number. For example, even if the language does not flag an error when a user writes: vdiff = 1mV - 30uA the person that wrote the line will generally see it as a problem and fix it. In my experience, providing units is the most efficient form of documentation available in numerical programming in the sense that one or two additional characters can often clarify otherwise very confusing code. My feeling is that retaining the units on real literals is of little value if you don't also extend the real variable type to hold units, or to create another variable type that would carry the units. Extending reals does not seem like a good idea, but creating a new type, quantity, seems practical. In this case, the units would be rather ephemeral in that they would not survive any operation. Thus, the result of an operation between a quantity and either a integer, real or quantity would always be a real, meaning that the units are lost. In this way, units are very light-weight and only really serve as documentation (for both programmers and end users). But this idea of retaining the units is the least important aspect of this proposal. The important aspects are: 1. It allows numbers to be entered in a clean form that is easy to type and easy to interpret 2. It allows numbers to be output in a clean form that is easy to interpret. 3. In many cases it allows units to be inserted into the code in a very natural and clean way to improve the clarity of the code. > Question, though: What happens with exa-? Currently, if the parser > sees "1E", it'll expect to see another number, eg 1E+1 == 10.0. Will > this double meaning cause confusion? Oh, I did not see this. Both SPICE and Verilog limit the scale factors to the common ones (T, G, M, k, _, m, u, n, p, f, a). I work in electrical engineering, and in that domain exa never comes up. My suggestion would be to limit ourselves to the common scale factors as most people know them. Using P, E, Z, Y, z, and y often actually works against us as most people are not familiar with them and so cannot interpret them easily. -Ken ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
