Re: OT: This Swift thing
> On Jun 5, 2014, at 1:14, Alain Ketterlin wrote: > > Swift's memory management is similar to python's (ref. counting). Which > makes me think that a subset of python with the same type safety would > be an instant success. Except that while you don't need to regularly worry about cycles in python, you do in swift. Which means you get to think constantly about direct and indirect cycles, figure out where to put weak stuff, when to use a local to keep a weak property alive until it finds it's strong home, etc. -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On Fri, 06 Jun 2014 01:54:32 +0200, Sturla Molden wrote: > When is static analysis actually needed and for what purpose? The > problem seems to be that managers, team leaders, CEOs, or (insert your > favorite tite), are not qualified to answer this question. So to be on > the safe side they go for as much static analysis as possible. Replace "as much as possible" with "the bare minimum provided by the compiler", and you will be usually right :-) Managers rarely choose languages like Haskell or Ada(?) where programs can be provably shown to be correct. They choose languages with just enough compile-time type checking to get in the way of rapid development, but not enough to lead to actual correct code. Some want languages like C that offer type-checking, but not languages like Pascal and its derivatives which enforce that type-checking -- Pascal is a "bondage and discipline" language, while C lets you escape from the discipline of types with the freedom of casts and other dangerous weak-typing features. -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On 06/05/2014 04:30 PM, Marko Rauhamaa wrote: What I'm afraid of is that the Python developers are reserving the right to remove the buffer and detach attributes from the standard streams in a future version. Being afraid is silly. If you have a question, ask it. -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Friday, June 6, 2014 8:50:57 AM UTC+5:30, Chris Angelico wrote: > kernel doesn't actually do *anything* with the string, it just passes > it right along to the file system. Which is what Marko (and others like Armin) are asking of python (treated as a processing 'kernel'): "I know what I am doing with my bytes -- please channel/funnel them around as requested without being unnecessarily and unrequestedly 'intelligent'" -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On Fri, Jun 6, 2014 at 1:16 PM, Steven D'Aprano wrote: > On Fri, 06 Jun 2014 09:21:26 +1000, Chris Angelico wrote: > >> On Fri, Jun 6, 2014 at 9:02 AM, Sturla Molden >> wrote: >>> You cannot spoof the type of an object in Python. >> >> Not without fiddling around. Python 3.4 on win32: > [...] > x = Foo() > x.spam() >> <__main__.Foo object at 0x0169AB10> spams > x.__class__ = Bar > x.spam() >> <__main__.Bar object at 0x0169AB10> eats spam >> >> The thing has the same id (as shown in its repr), but has changed class. > > That's not spoofing types. That is a language feature: within limits, > instances can change their type. It has been around for a long, long > time, back to Python 1.5 or older, and it has real-world use-cases: > > http://code.activestate.com/recipes/68429-ring-buffer/ True, it's not *spoofing*, in the same way that this C code is: float x = 3.14159; int y = *(int *)&x; And I'm aware that it's a language feature (why else would it be specifically limited with a clear exception when you try to assign something you can't?). So's the C-level fiddling, albeit for different reasons and in different ways. Anyway, I was broadly agreeing - Python's type system is "objects have types", rather than "stuff is in memory and data types are just how you look at it". It just happens to be possible to fiddle if you want to :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Fri, Jun 6, 2014 at 1:11 PM, Rustom Mody wrote: > All character strings, including > | filenames, are treated by the kernel in such a way that THEY > | APPEAR TO IT ONLY AS STRINGS OF BYTES. Yep, the real issue here is file systems, not the kernel. But yes, this is one of the very few places where the kernel deals with a string - and because of the hairiness of having to handle myriad file systems in a single path (imagine multiple levels of remote mounts - I've had a case where I mount via sshfs a tree that includes a Samba mount point, and you can go a lot deeper than that), the only thing it can do is pass the bytes on unchanged. Which means, in reality, the kernel doesn't actually do *anything* with the string, it just passes it right along to the file system. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On Fri, 06 Jun 2014 09:21:26 +1000, Chris Angelico wrote: > On Fri, Jun 6, 2014 at 9:02 AM, Sturla Molden > wrote: >> You cannot spoof the type of an object in Python. > > Not without fiddling around. Python 3.4 on win32: [...] x = Foo() x.spam() > <__main__.Foo object at 0x0169AB10> spams x.__class__ = Bar x.spam() > <__main__.Bar object at 0x0169AB10> eats spam > > The thing has the same id (as shown in its repr), but has changed class. That's not spoofing types. That is a language feature: within limits, instances can change their type. It has been around for a long, long time, back to Python 1.5 or older, and it has real-world use-cases: http://code.activestate.com/recipes/68429-ring-buffer/ -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Friday, June 6, 2014 4:22:22 AM UTC+5:30, Chris Angelico wrote: > On Fri, Jun 6, 2014 at 8:35 AM, Rustom Mody wrote: > > And then ask how Linux (in your and Stallman's sense) differs from > > Windows in how the filesystem handles things like filenames? > What are you testing of the kernel? Most of the kernel doesn't > actually work with text at all - it works with integers, buffers of > memory (which could be seen as streams of bytes, but might be almost > anything), process tables, open file handles... but not usually text. > To you, "EAGAIN" might be a bit of text, but to the Linux kernel, it's > an integer (11 decimal, if I recall correctly). Is that some fancy new > form of encoding? :) | Thanks to the properties of UTF-8 encoding, the Linux kernel, the | innermost and lowest-level part of the operating system, can | handle Unicode filenames without even having the user tell it | that UTF-8 is to be used. All character strings, including | filenames, are treated by the kernel in such a way that THEY | APPEAR TO IT ONLY AS STRINGS OF BYTES. Thus, it doesn't care and | does not need to know whether a pair of consecutive bytes should | logically be treated as two characters or a single one. The only | risk of the kernel being fooled would be, for example, for a | filename to contain a multibyte Unicode character encoded in such | a way that one of the bytes used to represent it was a slash or | some other character that has a special meaning in file | names. Fortunately, as we noted, UTF-8 never uses ASCII | characters for encoding multibyte characters, so neither the | slash nor any other special character can appear as part of one | and therefore there is no risk associated with using Unicode in | filenames. | | Filesystems found on Microsoft Windows machines (NTFS and FAT) | are different in that THEY STORE FILENAMES ON DISK IN SOME | PARTICULAR ENCODING. The kernel must translate this encoding to | the system encoding, which will be UTF-8 in our case. | | If you have Windows partitions on your system, you will have to | take care that they are mounted with correct options. For FAT and | ISO9660 (used by CD-ROMs) partitions, option utf8 makes the | system translate the filesystem's character encoding to | UTF-8. For NTFS, nls=utf8 is the recommended option (utf8 should | also work). [Emphases mine] From: http://michal.kosmulski.org/computing/articles/linux-unicode.html -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On Thu, 05 Jun 2014 23:13:00 +0100, Mark Lawrence wrote: > I'll simply say that I > understand Python to be strongly, dynamically typed. Correct. Anyone who hasn't done so needs to read this: http://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ I wouldn't quite go so far as to say that weakly typed is a meaningless concept, but I would agree that there are degrees of weakness. -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Missing stack frames?
On Fri, Jun 6, 2014 at 12:16 PM, Nikolaus Rath wrote: > - Is there some way to make the call stack for destructors less confusing? First off, either don't have refloops, or explicitly break them. That'll at least make things a bit more predictable; in CPython, you'll generally see destructors called as soon as something goes out of scope. Secondly, avoid threads when you're debugging a problem! I said this right at the beginning. If you run into problems, the first thing to do is isolate the system down to a single thread. Debugging is way easier without threads getting in each other's ways. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Missing stack frames?
dieter writes:
[...]
> Someone else already mentioned that the "close" call
> can come from a destructor. Destructors can easily be called
> at not obvious places (e.g. "s = C(); ... x = [s for s in ...]";
> in this example the list comprehension calls the "C" destructor
> which is not obvious when one looks only locally).
> The destructor calls often have intervening C code (which
> one does not see). However, in your case, I do not see
> why the "cgi" module should cause a destructor call of one
> of your server components.
Paul, dieter, you are my heroes. It was indeed an issue with a
destructor. It turns out that the io.RawIOBase destructor calls
self.close(). If the instance of a derived class is part of a reference
cycle, it gets called on the next routine run of the garbage
collector -- with the stack trace originating at whatever statement was
last executed before the gc run.
The following minimal example reproduces the problem:
#!/usr/bin/env python3
import io
import traceback
import threading
class Container:
pass
class InnocentVictim(io.RawIOBase):
def close(self):
print('close called in %s by:'
% threading.current_thread().name)
traceback.print_stack()
def busywork():
numbers = []
for i in range(500):
o = Container()
o.l = numbers
numbers.append(o)
if i % 87 == 0:
numbers = []
l = [ InnocentVictim() ]
l[0].cycle = l
del l
t = threading.Thread(target=busywork)
t.start()
t.join()
If you run this, you could things like:
close called in Thread-1 by:
File "/usr/lib/python3.4/threading.py", line 888, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "./test.py", line 18, in busywork
o = Container()
File "./test.py", line 13, in close
traceback.print_stack()
Ie, a method being called by a thread that doesn't have access to the
object, and without any reference to the call in the source.
I am left wondering:
- Is there really a point in the RawIOBase destructor calling close?
- Is there some way to make the call stack for destructors less confusing?
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On 6/5/2014 7:30 PM, Marko Rauhamaa wrote:
Steven D'Aprano :
"Can be replaced" by who? By the Python developers? By me? By random
library calls?
By you. sys.stdout and friends are writable. Any code you call may
have replaced them with another file-like object, and you should
honour that.
I can of course overwrite even sys and os and open and all. That hardly
merits mentioning in the API documentation.
What I'm afraid of is that the Python developers are reserving the right
to remove the buffer and detach attributes from the standard streams in
a future version.
No, not at all.
That would be terrible.
Agreed.
If it means some other module is allowed to commandeer the standard
streams, that would be bad as well.
I think that, for the most part, library modules should either open a
file given a filename from outside or read from and write to open files
handed to them from outside, but not hard-code the std streams. The
module doc should say if the file (name or object) must be text or in
particular binary.
The warning is also a hint as to how to solve a problem, such as testing
a binary filter. Assume the module reads from and writes to .buffer and
has a main function. One approach, untested:
import sys, io, unittest
from mod import main
class Binstd:
def __init(self):
self.buffer = io.BytesIO
sys.stdin = Binstd()
sys.stdout = Binstd()
sys.stdin.buffer.write('test data')
sys.stdin.buffer.seek(0)
main()
out = sys.stdout.buffer.getvalue()
# test that out is as expected for the input
# seek to 0 and truncate for more tests
Worst of all, I don't know why the caveat had to be there.
Because the streams can be replaced for a variety of good reasons, as above.
Or is it maybe because some python command line options could cause
buffer and detach not to be there? That would explain the caveat, but
still would be kinda sucky.
The doc set documents the Python command line options, as well any that
are CPython specific. It is possible that some implementation could add
one to open stdxyz in binary mode. CPython does not really need that.
--
Terry Jan Reedy
--
https://mail.python.org/mailman/listinfo/python-list
Re: Missing stack frames?
Nikolaus Rath writes:
> Chris Angelico writes:
>> On Wed, Jun 4, 2014 at 12:30 PM, Nikolaus Rath wrote:
>>> I've instrumented one of my unit tests with a conditional
>>> 'pdb.set_trace' in some circumstances (specifically, when a function is
>>> called by a thread other than MainThread).
>>
>> I think the likelihood of this being an issue with interactive
>> debugging and threads is sufficiently high that you should avoid
>> putting the two together, at least until you can verify that the same
>> problem occurs without that combination.
>
> Here's stacktrace as obtained by traceback.print_stack():
>
> tests/t1_backends.py:563: test_extra_data[mock_s3c-zlib] PASSED
> tests/t1_backends.py:563: test_extra_data[mock_s3c-bzip2] PASSED
>
> 87 tests deselected by '-kextra'
> =
> === 5 passed, 1 skipped, 87 deselected in 0.65 seconds
>
> File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line
> 853, in close
> self.fh.close()
> File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/s3c.py", line 691,
> in close
> traceback.print_stack(file=sys.stdout)
> something is wrong
> File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line
> 853, in close
> self.fh.close()
> File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/s3c.py", line 691,
> in close
> traceback.print_stack(file=sys.stdout)
> something is wrong
> File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line
> 1050, in close
> self.fh.close()
> File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/s3c.py", line 691,
> in close
> traceback.print_stack(file=sys.stdout)
>
> Still no context before the ominous close() call. I'm very confused.
It get's even funnier if I just repeat this exercise (without changing
code). Here are some more "tracebacks" that I got:
File "/usr/bin/py.test-3", line 5, in
sys.exit(load_entry_point('pytest==2.5.2', 'console_scripts', 'py.test')())
[...]
File "/usr/lib/python3/dist-packages/py/_code/code.py", line 524, in
repr_traceback_entry
source = self._getentrysource(entry)
File "/usr/lib/python3/dist-packages/py/_code/code.py", line 450, in
_getentrysource
source = entry.getsource(self.astcache)
File "/usr/lib/python3/dist-packages/py/_code/code.py", line 199, in getsource
astnode=astnode)
File "/usr/lib/python3/dist-packages/py/_code/source.py", line 367, in
getstatementrange_ast
astnode = compile(content, "source", "exec", 1024) # 1024 for AST
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line
1050, in close
self.fh.close_real()
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/s3c.py", line 702, in
close_real
traceback.print_stack(file=sys.stdout)
or also
File "/usr/bin/py.test-3", line 5, in
sys.exit(load_entry_point('pytest==2.5.2', 'console_scripts', 'py.test')())
[...]
File "/usr/lib/python3.4/logging/__init__.py", line 1474, in callHandlers
hdlr.handle(record)
File "/usr/lib/python3.4/logging/__init__.py", line 842, in handle
rv = self.filter(record)
File "/usr/lib/python3.4/logging/__init__.py", line 699, in filter
for f in self.filters:
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line 853,
in close
self.fh.close()
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line
1050, in close
self.fh.close_real()
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/s3c.py", line 702, in
close_real
traceback.print_stack(file=sys.stdout)
and this one looks actually the way I would expect:
[...]
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line 335,
in fetch
return self.perform_read(do_read, key)
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line 58,
in wrapped
return method(*a, **kw)
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line 309,
in perform_read
return fn(fh)
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line 859,
in __exit__
self.close()
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line 853,
in close
self.fh.close()
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/common.py", line
1050, in close
self.fh.close_real()
File "/home/nikratio/in-progress/s3ql/src/s3ql/backends/s3c.py", line 702, in
close_real
traceback.print_stack(file=sys.stdout)
I am not using any C extension modules, but I guess I nevertheless have
to assume that something seriously messed up either the stack or the
traceback printing routines?
Best,
Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to use SQLite (sqlite3) more efficiently
On Fri, Jun 6, 2014 at 8:57 AM, R Johnson wrote: > Sorry about that. As you can probably tell, I'm relatively new to using > mailing lists. I'm not exactly sure why that occurred like it did. I'll try > adding "Re:" in front of the subject when I send this e-mail, and see if it > works right this time. If not, I guess you'll have to please explain to me > what I'm doing wrong. The subject line isn't as important as a header, carried invisibly through, that says that you were replying to an existing post. :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Missing stack frames?
Paul Rubin writes: > Nikolaus Rath writes: >> Still no context before the ominous close() call. I'm very confused. > > close() could be getting called from a destructor as the top level > function of a thread exits, or something like that. Shouldn't the destructor have its own stack frame then, i.e. shouldn't the first frame be in a __del__ function? Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« -- https://mail.python.org/mailman/listinfo/python-list
Re: How to use SQLite (sqlite3) more efficiently
Sorry for the attachment issue. I'm used to the wxPython-users Google
group, where posters are instructed to attach code to their post instead
of including it in the body of the message.
I placed the latest version of my sample code below, since I made a few
minor changes to it after posting it as that attachment. (I changed
REPLACE to INSERT OR REPLACE as I mentioned above, followed Chris'
suggestion to not use backslash continuation characters, and changed the
description of C++ as mentioned below.)
FYI, you can also view the version (now outdated) that I had attached on
the Google Groups website at:
https://groups.google.com/group/comp.lang.python/attach/bd64353c8dfd43ad/sqlite_test.py?part=0.1&view=1
> > OK, I'll admit that I don't know Assembly :). How about the paradox
> "Making
> > Easy Things Hard & Hard Things Easy"? Although that might make my
> > description of C++ too unfair; suggestions for improvements to my
> language
> > descriptions are welcome :).
>
> Hehe. As I'm sure you're aware, this has absolutely nothing to do with
> your SQL or Python code.
Of course :). It's just for fun. I changed the description for C++ to
"Making Easy Things Hard & Hard Things Hard" (without the -er on the end).
> (Suggestion: Always reply to an existing post if it's part of the same
> thread. Replying to your own post is fine, and it links the thread
> together nicely.)
Sorry about that. As you can probably tell, I'm relatively new to using
mailing lists. I'm not exactly sure why that occurred like it did. I'll
try adding "Re:" in front of the subject when I send this e-mail, and
see if it works right this time. If not, I guess you'll have to please
explain to me what I'm doing wrong.
Thank you again for your help.
-- Timothy
*** sqlite_test.py ***
from __future__ import print_function
import sqlite3
def get_description(conn, description):
row = conn.execute("SELECT description FROM languages WHERE name=?",
(description,)).fetchone()
if row:
return row[0]
def set_description(conn, name, description):
conn.execute("INSERT OR REPLACE INTO languages VALUES(?,?)",
(name, description))
conn.commit()
def delete_language(conn, name):
conn.execute("DELETE FROM languages WHERE name=?", (name,))
conn.commit()
conn = sqlite3.connect(":memory:")
conn.execute("CREATE TABLE IF NOT EXISTS languages(name TEXT PRIMARY KEY, "
"description TEXT NOT NULL)")
set_description(conn, "Perl", "Making Easy Things Easy & Hard Things
Possible")
set_description(conn, "Python", "Making Easy Things Easier & Hard Things
Easy")
set_description(conn, "C++", "Making Easy Things Hard & Hard Things Hard")
for language in ("Perl", "Python", "C++"):
print("%s: %s" % (language, get_description(conn, language)))
set_description(conn, "Assembly", "Making Easy Things Very Hard & "
"Hard Things Impossible (Hey, that's not fair!)")
print("Assembly: %s" % get_description(conn, "Assembly"))
set_description(conn, "Assembly",
"Making Easy Things Hard & Hard Things Easy")
print("Assembly: %s" % get_description(conn, "Assembly")) # Should be
changed
delete_language(conn, "Assembly")
print("Assembly: %s" % get_description(conn, "Assembly")) # Should be None
conn.close()
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
In article , Roy Smith wrote: > In article , > Ned Deily wrote: > > Roy is using MT-NewsWatcher as a client. > Yes. Except for the fact that it hasn't kept up with unicode, I find > the U/I pretty much perfect. I imagine at some point I'll be force to > look elsewhere, but then again, netnews is pretty much dead. I agree about the U/I, although I'm sure a lot of that has to do with familiarity. However, netnews isn't dead, it has just morphed a bit. A newsreader, like MT-NW, is great for following mailing lists like this (and most other Python-related lists) via gmane.org's bi-directional mailing list - NNTP gateways. And for this list it's usually better to read the mailing list variant via gmane.org NNTP than the Usenet group variant via a traditional USENET NNTP server because there's less spam with the former. > > BTW, don't upgrade to OS X 10.9 Mavericks if you're dependent on > > MT-NW; it finally stops working there because what was left of Open > > Transport support in OS X has finally been ripped out of 10.9. > Hmmm, good to know. I'm still on 10.7, and don't see any reason to > move. But, then again, you'd expect that from somebody who's still on > Python 2.x, wouldn't you? Heh. Well, both 10.8 and 10.9 proved various improvements, both feature and performance, over 10.7. Alas, Apple won't likely be supporting 10.7 with security updates for as long as the PSF will be supporting 2.7.x. But, by then, you'll have had a chance to re-implement MT-NW in Python. -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 06/06/14 02:13, Roy Smith wrote: > Well, you *can* play evil games with the struct module :-) But then you are asking for it, it does not happen by accident. Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: Forking PyPI package
On Fri, 6 Jun 2014 03:37:56 +1000, Chris Angelico wrote:
> On Fri, Jun 6, 2014 at 2:56 AM, Wiktor wrote:
>> I guess, I'll try to do what Chris proposed. Forget about this
>> implementation and write python script from the scratch looking only at the
>> original JavaScript version. :-/
>
> Sadly, that may be your only safe option.
>
> Let this be a lesson to all: If you want something to be free
> software, make it very clear, because "it looks like he meant that to
> be open source" just isn't enough :(
Lesson taken. ;-)
Interesting thing is, that for another 4 people, lack of license in this
script wasn't problem to publish its modified version. I've just searched
phrase "pwdhash" on GitHub, to simply check if someone else hadn't port
this script to Python3 earlier, or maybe ported it (with proper license) to
Python2 so I would have better start. And I've found practically the same
script here: https://github.com/ali01/pwdhash.py, forked then 3 times.
Of course I'm not going to act now "Oh, they could do that without
consequences, so why should I bother?" - no, I'm going to do this right (as
a good start in OS community) - but it feels awkward now. ;-)
--
Best regards, Wiktor Matuszewski
'py{}@wu{}em.pl'.format('wkm', 'ka')
--
https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
In article , Sturla Molden wrote: > On 05/06/14 22:27, Alain Ketterlin wrote: > > I have seen dozens of projects where Python was dismissed because of the > > lack of static typing, and the lack of static analysis tools. > > If you are worried your code will bring down the next Ariane launch, I > can understand this argument. If you are only worried a junior developer > will make stupid mistankes that the test suite will trap, then no... > > In Python you cannot spoof an object's type, which means the type safety > is strong. You cannot make Python silently return bytes of garbage by > using objects of incompatible types. Well, you *can* play evil games with the struct module :-) -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
In article , Ned Deily wrote: > In article <8681edf0-7a1f-4110-9f87-a8cd0988c...@googlegroups.com>, > Rustom Mody wrote: > > > On Friday, June 6, 2014 2:30:26 AM UTC+5:30, Roy Smith wrote: > > > Just for fun, I took a screen-shot of what this looks like in my > > > newsreader. URL below. Looks like something chomped on unicode pretty > > > hard :-) > > > > > > http://www.panix.com/~roy/unicode.pdf > > > > Yii > > Roy is using MT-NewsWatcher as a client. Yes. Except for the fact that it hasn't kept up with unicode, I find the U/I pretty much perfect. I imagine at some point I'll be force to look elsewhere, but then again, netnews is pretty much dead. > BTW, don't upgrade to OS X 10.9 Mavericks if you're dependent on > MT-NW; it finally stops working there because what was left of Open > Transport support in OS X has finally been ripped out of 10.9. Hmmm, good to know. I'm still on 10.7, and don't see any reason to move. But, then again, you'd expect that from somebody who's still on Python 2.x, wouldn't you? -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On Thu, Jun 5, 2014 at 2:34 PM, Albert-Jan Roskam wrote:
>> If you want to be really picky about removing exactly one line
>> terminator, then this captures all the relatively modern variations:
>> re.sub('\r?\n$|\n?\r$', line, '', count=1)
>
> or perhaps: re.sub("[^ \S]+$", "", line)
That will remove more than one terminator, plus tabs. Points for
including \f and \v though.
I suppose if we want to be absolutely correct, we should follow the
Unicode standard:
re.sub(r'\r?\n$|[\r\v\f\x85\u2028\u2029]$', line, '', count=1)
--
https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 06/06/14 01:41, Mark Lawrence wrote: > s/almost// :) Sometimes it is the right decision, like when your code is firmware for some avionics or medial life-support apparatus. Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 05/06/14 22:27, Alain Ketterlin wrote: > I have seen dozens of projects where Python was dismissed because of the > lack of static typing, and the lack of static analysis tools. If you are worried your code will bring down the next Ariane launch, I can understand this argument. If you are only worried a junior developer will make stupid mistankes that the test suite will trap, then no... In Python you cannot spoof an object's type, which means the type safety is strong. You cannot make Python silently return bytes of garbage by using objects of incompatible types. You cannot add a char to a float, and you cannot make a bit be treated bitwise as a float. You cannot make Python silently treat four bytes as an int. The worst thing that can happen is a TypeError raised at run-time. Yes, an unhandled TypeError exception could in theory bring down Ariane. But you are probably not developing for it. When is static analysis actually needed and for what purpose? The problem seems to be that managers, team leaders, CEOs, or (insert your favorite tite), are not qualified to answer this question. So to be on the safe side they go for as much static analysis as possible. But still they avoid Ada, because, you know, the journals they read are full of Java buzzwords. Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 06/06/2014 00:03, Sturla Molden wrote: Alain Ketterlin wrote: Many of these students suggest Python as the development language (they learned it and liked it), and the suggestion is (almost) always rejected, in favor of Java or C# or C/C++. And it was almost always the wrong decision... Sturla s/almost// :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Fri, Jun 6, 2014 at 9:30 AM, Marko Rauhamaa wrote: > Steven D'Aprano : > >>> "Can be replaced" by who? By the Python developers? By me? By random >>> library calls? >> >> By you. sys.stdout and friends are writable. Any code you call may >> have replaced them with another file-like object, and you should >> honour that. > > I can of course overwrite even sys and os and open and all. That hardly > merits mentioning in the API documentation. > > What I'm afraid of is that the Python developers are reserving the right > to remove the buffer and detach attributes from the standard streams in > a future version. That would be terrible. > > If it means some other module is allowed to commandeer the standard > streams, that would be bad as well. > > Worst of all, I don't know why the caveat had to be there. > > Or is it maybe because some python command line options could cause > buffer and detach not to be there? That would explain the caveat, but > still would be kinda sucky. It's more that replacng sys.std* is considered reasonably normal (unlike, say, replacing sys.float_info, which would be a weird thing to do); and you could replace them with something that doesn't have those attributes. If you're running a top-level script and you never import anything that changes the streams, you should be able to depend on those always being there. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
Steven D'Aprano : >> "Can be replaced" by who? By the Python developers? By me? By random >> library calls? > > By you. sys.stdout and friends are writable. Any code you call may > have replaced them with another file-like object, and you should > honour that. I can of course overwrite even sys and os and open and all. That hardly merits mentioning in the API documentation. What I'm afraid of is that the Python developers are reserving the right to remove the buffer and detach attributes from the standard streams in a future version. That would be terrible. If it means some other module is allowed to commandeer the standard streams, that would be bad as well. Worst of all, I don't know why the caveat had to be there. Or is it maybe because some python command line options could cause buffer and detach not to be there? That would explain the caveat, but still would be kinda sucky. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
Steven D'Aprano : > In any case, I reject your premise. ALL data types are constructed on > top of bytes, Only in a very dull sense. > and so long as you allow applications *any way* to coerce data types > to different data types, you allow them to see "inside the black box". I can't see the bytes inside Python objects, including strings, and that's how it is supposed to be. Similarly, I can't (easily) see how files are laid out on hard disks. That's a true abstraction. Nothing in linux presents data, though, except through bytes. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On Fri, Jun 6, 2014 at 9:02 AM, Sturla Molden wrote: > You cannot spoof the type of an object in Python. Not without fiddling around. Python 3.4 on win32: >>> class Foo: def spam(self): print(self,"spams") >>> class Bar: def spam(self): print(self,"eats spam") >>> x = Foo() >>> x.spam() <__main__.Foo object at 0x0169AB10> spams >>> x.__class__ = Bar >>> x.spam() <__main__.Bar object at 0x0169AB10> eats spam The thing has the same id (as shown in its repr), but has changed class. However, you can't turn it into an int: >>> x.__class__ = int Traceback (most recent call last): File "", line 1, in x.__class__ = int TypeError: __class__ assignment: only for heap types So there are limits. (Obviously with ctypes you can do anything, but at that point, you're not working with Python any more, you're fiddling with CPython's RAM. That's quite different.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Thu, 05 Jun 2014 23:21:35 +0300, Marko Rauhamaa wrote: > Terry Reedy : > >> On 6/5/2014 5:53 AM, Marko Rauhamaa wrote: >>> Chris Angelico : >>> If the standard streams are so crucial, why are their most obvious interfaces insignificant to you? >>> >>> I want the standard streams to consume and produce bytes. >> >> Easy. Read the manual entry for stdxxx. "To write or read binary data >> from/to the standard streams, use the underlying binary buffer object. >> For example, to write bytes to stdout, use >> sys.stdout.buffer.write(b'abc')" > > This note from the manual is a bit vague: > >Note that the streams can be replaced with objects (like io.StringIO) >that do not support the buffer attribute or the detach() method > > "Can be replaced" by who? By the Python developers? By me? By random > library calls? By you. sys.stdout and friends are writable. Any code you call may have replaced them with another file-like object, and you should honour that. The API could have/should have been a little more friendly, but it's conceptually simple: * Does sys.stdout have a buffer attribute? Then write raw bytes to the buffer. * If not, then write raw bytes to sys.stdout. * If either fails, then somebody has replaced stdout with something weird, and they deserve whatever horrible fate their damn fool move causes. It's not your responsibility to try to keep your application running under bizarre circumstances. -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
Alain Ketterlin wrote: > Many of these students suggest Python as the > development language (they learned it and liked it), and the suggestion > is (almost) always rejected, in favor of Java or C# or C/C++. And it was almost always the wrong decision... Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
Chris Angelico wrote: > "Type safety" means many different things to different people. What > Python has is untyped variables, and hierarchically typed objects. > It's impossible to accidentally treat an integer as a float, and have > junk data [1]. It's impossible to accidentally call a base class's > method when you ought to have called the overriding method in the > subclass, which is a risk in C++ [2]. Exactly. I have no hopes that those who claim Python lacks type safety will understand this, however. > If you mistakenly pass a list to > a function that was expecting an integer, that function will *know* > that it got a list, because objects in Python are rigidly typed. And then there is nothing that prevents the function from raising a TypeError. Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On Thu, 05 Jun 2014 22:07:18 +0200, Alain Ketterlin wrote: > Sturla Molden writes: > >> On 05/06/14 10:14, Alain Ketterlin wrote: >> >>> Type safety. >> >> Perhaps. Python has strong type safety. > > Come on. No, Sturla is correct. Python has strongly-typed values and dynamically- typed variables, which means that you get type errors at run-time, not compile-time. But you still get type errors: py> '1' + 2 Traceback (most recent call last): File "", line 1, in TypeError: Can't convert 'int' object to str implicitly [...] >> When I compile Cython modules I use LLVM on this computer. > > Cython is not Python, it is another language, with an incompatible > syntax. Cython is best considered a superset of Python, with a pure-Python compatibility mode. It can run standard Python code. -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
Alain Ketterlin wrote: >> Perhaps. Python has strong type safety. > > Come on. You cannot spoof the type of an object in Python. In C++ you can downcast any address to void* and make an object be treated as anything. You cannot make Python treat an int as a float and return garbage. Types in Python are strictly enforced. Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Thu, 05 Jun 2014 21:30:11 +0300, Marko Rauhamaa wrote: > Terry Reedy : > >> Different OSes *do* have different assumptions. Both MacOSX and current >> Windows use (UCS-2 or) UTF-16 for text. > > Linux can use anything for text; UTF-8 has become a de-facto standard. > > How text is represented is very different from whether text is a > fundamental data type. A fundamental text file is such that ordinary > operating system facilities can't see inside the black box (that is, > they are *not* encoded as far as the applications go). Wait, are they black-boxes to the *operating system* or to *applications*? They aren't the same thing. In any case, I reject your premise. ALL data types are constructed on top of bytes, and so long as you allow applications *any way* to coerce data types to different data types, you allow them to see "inside the black box". I can extract the four bytes from a C long integer, but that doesn't mean that C longs aren't fundamental data types in Unix/Linux. > I have no idea how opaque text files are in Windows or OS-X. Exactly as opaque as they are in Unix, which is to say not at all. Just open the file in binary mode, and voilà you see the underlying bytes. All you're doing is pointing out that, in modern electronic computers, the fundamental data structure which underlies all others (the indivisible protons and neutrons, so to speak, only there are 256 of them rather than 2) is the byte. We know this, and don't dispute it. (Like protons and neutrons, we can see inside bytes to the quark-like bits that make up bytes. Like quarks, bits do not exist in isolation, but only inside bytes.) >> For Windows, at least, the interface is much improved in Python 3. > > Yes, I get the feeling that Python is reaching out to Windows and OS-X > and trying to make linux look like them. Unicode support in OS-X is (I have been assured) is very good, probably better than Linux. Apple has very high standards when it comes to their apps, and provides rich Unicode-aware APIs. But Linux Unicode support is much better than Windows. Unicode support in Windows is crippled by continued reliance on legacy code pages, and by the assumption deep inside the Windows APIs that Unicode means "16 bit characters". See, for example, the amount of space spent on fixing Windows Unicode handling here: http://www.utf8everywhere.org/ -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
Alain Ketterlin wrote: >> And that's counting only CPU time. If you count wall time, your >> typical Python program spends most of its time deep inside kernel API >> calls, waiting for the user or I/O or something. > > But this is true of any IO-bound program, whatever the language. Exactly, this is true even with Swift. The Swift and the Python program would spend most of the time doing the same thing (idle processing). Thus a GUI program written in Python would not exhaust the battery faster than a program written in Swift. In both cases, the majority of the wall time is spent inside the kernel waiting for some UI event (keyboard, mouse, whatever). And in both cases the battery is spent lighting up a screen that is mostly static. And in both bases the graphics displayed on the screen is generated inside some UI framework written in Objective-C. Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
In article <8681edf0-7a1f-4110-9f87-a8cd0988c...@googlegroups.com>, Rustom Mody wrote: > On Friday, June 6, 2014 2:30:26 AM UTC+5:30, Roy Smith wrote: > > Just for fun, I took a screen-shot of what this looks like in my > > newsreader. URL below. Looks like something chomped on unicode pretty > > hard :-) > > > > http://www.panix.com/~roy/unicode.pdf > > Yii Roy is using MT-NewsWatcher as a client. Because its codebase's origins are back in classic MacOS (<= 9), it has its own *interesting* ways to deal with encodings. BTW, don't upgrade to OS X 10.9 Mavericks if you're dependent on MT-NW; it finally stops working there because what was left of Open Transport support in OS X has finally been ripped out of 10.9. -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Fri, Jun 6, 2014 at 8:35 AM, Rustom Mody wrote: > On Thursday, June 5, 2014 10:58:43 PM UTC+5:30, Chris Angelico wrote: >> On Fri, Jun 6, 2014 at 2:52 AM, Marko Rauhamaa wrote: >> > That linux text is not the same thing as Python's text. Conceptually, >> > Python text is a sequence of 32-bit integers. Linux text is a sequence >> > of 8-bit integers. > >> Point of terminology: Linux is the kernel, everything you say below >> here is talking about particular programs. > > If it helps try the following substitution: > > s/Linux/Pretty much all the distros that use Linux for their OS kernel/ You could look at the Debian Project, which is a full environment with everything you're talking about. And everything you say would be equally true of Debian Linux and Debian kfreebsd. :) > BTW the only (other) guy I know who insistently makes that distinction is > Richard Stallman. > > Are you an emacs user by any chance ? Nope! Just a terminology nerd. :) >> From what I understand, >> bash (just another Unix program) treats strings as sequences of >> codepoints, just as Python does; though its string manipulation is not >> nearly as rich as Python's, so it's harder to prove. Python is itself >> a Unix program, so you can do the exact same proofs and demonstrate >> that Linux is clearly Unicode-aware. It's not Linux you're testing. > > In these 'other programs' is it permissible to include the kernel > itself? > And then ask how Linux (in your and Stallman's sense) differs from > Windows in how the filesystem handles things like filenames? What are you testing of the kernel? Most of the kernel doesn't actually work with text at all - it works with integers, buffers of memory (which could be seen as streams of bytes, but might be almost anything), process tables, open file handles... but not usually text. To you, "EAGAIN" might be a bit of text, but to the Linux kernel, it's an integer (11 decimal, if I recall correctly). Is that some fancy new form of encoding? :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Thursday, June 5, 2014 10:58:43 PM UTC+5:30, Chris Angelico wrote: > On Fri, Jun 6, 2014 at 2:52 AM, Marko Rauhamaa wrote: > > That linux text is not the same thing as Python's text. Conceptually, > > Python text is a sequence of 32-bit integers. Linux text is a sequence > > of 8-bit integers. > Point of terminology: Linux is the kernel, everything you say below > here is talking about particular programs. If it helps try the following substitution: s/Linux/Pretty much all the distros that use Linux for their OS kernel/ BTW the only (other) guy I know who insistently makes that distinction is Richard Stallman. Are you an emacs user by any chance ? > From what I understand, > bash (just another Unix program) treats strings as sequences of > codepoints, just as Python does; though its string manipulation is not > nearly as rich as Python's, so it's harder to prove. Python is itself > a Unix program, so you can do the exact same proofs and demonstrate > that Linux is clearly Unicode-aware. It's not Linux you're testing. In these 'other programs' is it permissible to include the kernel itself? And then ask how Linux (in your and Stallman's sense) differs from Windows in how the filesystem handles things like filenames? -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On Friday, June 6, 2014 2:30:26 AM UTC+5:30, Roy Smith wrote: > Just for fun, I took a screen-shot of what this looks like in my > newsreader. URL below. Looks like something chomped on unicode pretty > hard :-) > > http://www.panix.com/~roy/unicode.pdf Yii -- https://mail.python.org/mailman/listinfo/python-list
Re: Tabs (was Re: OT: This Swift thing)
On 6/5/2014 3:05 PM, Terry Reedy wrote: On 6/5/2014 4:39 AM, Steven D'Aprano wrote: Treating \t as a single space would be pathetic but standard. Treating it as (up to) 8 spaces would be more useful, and standard. Rendering it as a picture of a banana dancing on the ceiling would be silly and non- standard. Not rendering it at all is even more stupid and less justified. <\t> would be better than nothing. Let us see what Thunderbird does *now*. Maybe complaints prompted a change. Or maybe some clients eat tabs upon sending. 4 spaces This is now 5 spaces beyond the added '> ' quote marker. 8 spaces 9 spaces beyond ... tab, which displays as 8 spaces on input Not literally 8 spaces, but one jump of 8 spaces. Anyway, it is now 5 actual spaces beyond '> '. Something changed somewhere. 2 tabs 9 actual spaces. Regardless of what shows, there is still no configuration option. And what shows is not what I expected, but better than the previous non-rendering. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 6/5/2014 4:07 PM, Alain Ketterlin wrote: When I compile Cython modules I use LLVM on this computer. Cython is not Python, it is another language, with an incompatible syntax. Cython compiles Python with optional extensions that allow additional speed ups over compiling Python as is. In other word, the Cython language is a Python superset. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re:How to use SQLite (sqlite3) more efficiently
R Johnson Wrote in message: > > I've attached some new sample code in which I've attempted to correct > various things that you mentioned. Attachments don't work well for many people using this list. I for one can't even see them. -- DaveA -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 05/06/2014 22:53, Marko Rauhamaa wrote:
Mark Lawrence :
On 05/06/2014 21:07, Alain Ketterlin wrote:
Sturla Molden writes:
On 05/06/14 10:14, Alain Ketterlin wrote:
Type safety.
Perhaps. Python has strong type safety.
Come on.
I don't understand that comment, please explain.
I guess what is referred to is static typing. It serves two purposes:
1. It makes the managers of software development teams believe the
junior developers in their teams won't be able to do too much damage
as the compiler at least enforces some rigor in the code. Hence,
"safety."
2. It makes it much easier to automatically optimize the code.
Unfortunately, it also has serious downsides:
3. The code becomes very tedious to type in. You may need hundreds of
lines of boilerplate code before it actually does anything. It also
easily makes you lose your focus.
4. The flow of the code becomes hard to understand because of the
boilerplate. Ironically, the very straitjacket that seeks to force
good quality on you prevents you from seeing the forest for the
trees.
Example:
Map makeStreetAddressMap(
List infoList) {
Map map =
new HashMap();
for (StreetInfo info : infoList)
map.put(info.getStreetAddress(), info.getZipCode());
return map;
}
vs
def make_street_address_map(info_list):
map = {}
for info in info_list:
map[info.get_street_address()] = info.get_zip_code()
return map
or:
def make_street_address_map(info_list):
return dict((info.get_street_address(), info.get_zip_code())
for info in info_list)
Marko
Interesting. We've gone from "Python has strong type safety" to "come
on" to "I guess what is referred to is static typing". I'll simply say
that I understand Python to be strongly, dynamically typed.
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
---
This email is free from viruses and malware because avast! Antivirus protection
is active.
http://www.avast.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On 6/5/2014 4:21 PM, Marko Rauhamaa wrote:
Terry Reedy :
On 6/5/2014 5:53 AM, Marko Rauhamaa wrote:
Chris Angelico :
If the standard streams are so crucial, why are their most obvious
interfaces insignificant to you?
I want the standard streams to consume and produce bytes.
Easy. Read the manual entry for stdxxx. "To write or read binary data
from/to the standard streams, use the underlying binary buffer object.
For example, to write bytes to stdout, use
sys.stdout.buffer.write(b'abc')"
This note from the manual is a bit vague:
Note that the streams can be replaced with objects (like io.StringIO)
that do not support the buffer attribute or the detach() method
"Can be replaced" by who? By the Python developers? By me? By random
library calls?
Fair question. The Python developers will not fiddle with stdxxx for 3rd
party code on 3rd party systems. We do sometimes *temporarily replace
the streams with StringIO, either directly or via test.support when
testing Python itself or stdlib modules. That is done in Lib/test, and
except for testing StringIO, it is only done as a convenience, not a
necessity.
To test a binary stream filter, you would have to do something else,
like read from and write to actual files on disk. Otherwise, you seem
unlikely to sabotage yourself, even accidentally.
Random non-stdlib library calls could sabotage you. However, in my
opinion, an imported 3rd party module should never modify std streams,
with one exception. The exception would be a module whose entire purpose
was to put the streams in a known state, as documented, and only if
intentionally asked to.
Having said that, bound methods created (first) should work regardless
of any subsequent manipulation of sys. Here is an experiment, run from
an Idle editor.
import sys
sysout = sys.stdout.write
sys.stdout = None
sysout('works anyway\n')
>>>
works anyway
(Of course, subsequent attempts to continue interactively fail. But that
is not your use case.)
--
Terry Jan Reedy
--
https://mail.python.org/mailman/listinfo/python-list
Re: any wheel experts
On 05/06/2014 22:42, Ned Deily wrote: In article , Rustom Mody wrote: On Thursday, June 5, 2014 10:21:06 PM UTC+5:30, Robin Becker wrote: I used to create exe files for windows, but the latest and greatest concept is wheels .whl files. If someone here knows (and answers!) great. Else you'll probably get more info here: https://groups.google.com/forum/?pli=1#!forum/python-virtualenv Actually, the Distutils-SIG would be a better place to ask about packaging issues, including wheels: https://mail.python.org/mailman/listinfo/distutils-sig Which I assume is gmane.comp.python.distutils.devel for people like myself who like the one stop shop :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
Mark Lawrence :
> On 05/06/2014 21:07, Alain Ketterlin wrote:
>> Sturla Molden writes:
>>> On 05/06/14 10:14, Alain Ketterlin wrote:
Type safety.
>>> Perhaps. Python has strong type safety.
>> Come on.
>
> I don't understand that comment, please explain.
I guess what is referred to is static typing. It serves two purposes:
1. It makes the managers of software development teams believe the
junior developers in their teams won't be able to do too much damage
as the compiler at least enforces some rigor in the code. Hence,
"safety."
2. It makes it much easier to automatically optimize the code.
Unfortunately, it also has serious downsides:
3. The code becomes very tedious to type in. You may need hundreds of
lines of boilerplate code before it actually does anything. It also
easily makes you lose your focus.
4. The flow of the code becomes hard to understand because of the
boilerplate. Ironically, the very straitjacket that seeks to force
good quality on you prevents you from seeing the forest for the
trees.
Example:
Map makeStreetAddressMap(
List infoList) {
Map map =
new HashMap();
for (StreetInfo info : infoList)
map.put(info.getStreetAddress(), info.getZipCode());
return map;
}
vs
def make_street_address_map(info_list):
map = {}
for info in info_list:
map[info.get_street_address()] = info.get_zip_code()
return map
or:
def make_street_address_map(info_list):
return dict((info.get_street_address(), info.get_zip_code())
for info in info_list)
Marko
--
https://mail.python.org/mailman/listinfo/python-list
Re: any wheel experts
In article , Rustom Mody wrote: > On Thursday, June 5, 2014 10:21:06 PM UTC+5:30, Robin Becker wrote: > > I used to create exe files for windows, but the latest and greatest concept > > is > > wheels .whl files. > > If someone here knows (and answers!) great. > Else you'll probably get more info here: > https://groups.google.com/forum/?pli=1#!forum/python-virtualenv Actually, the Distutils-SIG would be a better place to ask about packaging issues, including wheels: https://mail.python.org/mailman/listinfo/distutils-sig -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On Fri, Jun 6, 2014 at 7:23 AM, Mark Lawrence wrote: > On 05/06/2014 21:07, Alain Ketterlin wrote: >> >> Sturla Molden writes: >> >>> On 05/06/14 10:14, Alain Ketterlin wrote: >>> Type safety. >>> >>> Perhaps. Python has strong type safety. >> >> Come on. > > I don't understand that comment, please explain. "Type safety" means many different things to different people. What Python has is untyped variables, and hierarchically typed objects. It's impossible to accidentally treat an integer as a float, and have junk data [1]. It's impossible to accidentally call a base class's method when you ought to have called the overriding method in the subclass, which is a risk in C++ [2]. If you mistakenly pass a list to a function that was expecting an integer, that function will *know* that it got a list, because objects in Python are rigidly typed. But some languages stipulate the types that a variable can take, and that's something Python doesn't do. If you want to say that this function argument must be an integer, you have to explicitly check it inside the function. (And the Pythonic thing to do is to *not* check it, but that's a separate point.) This is something that function annotations can be used for, but I'm not seeing a huge thrust to make use of them everywhere. Why not? I suspect because the need for it just isn't as great as some people think. ChrisA [1] Note that in some circumstances, you can (deliberately) fiddle with an object's type. But you can't just reinterpret the bits in memory, the way you can in C, by casting a pointer and dereferencing it. Hence, it's impossible to *accidentally* muck this up. [2] Again, you can muck things up, by explicitly pulling up a function from the base class, rather than using method lookup on the object. But again, you can't do it accidentally. -- https://mail.python.org/mailman/listinfo/python-list
Re: Matplotlib - specifying bin widths
On 05/06/2014 16:54, Jamie Mitchell wrote: Hello all! Instead of setting the number of bins I want to set the bin width. I would like my bins to go from 1.7 to 2.4 in steps of 0.05. How do I say this in the code? Cheers, Jamie This should help http://stackoverflow.com/questions/6986986/bin-size-in-matplotlib-histogram -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 05/06/2014 21:27, Alain Ketterlin wrote: Chris Angelico writes: On Thu, Jun 5, 2014 at 7:42 PM, Alain Ketterlin wrote: Chris Angelico writes: On Thu, Jun 5, 2014 at 6:14 PM, Alain Ketterlin wrote: Swift's memory management is similar to python's (ref. counting). Which makes me think that a subset of python with the same type safety would be an instant success. In the same way that function annotations to give type information were an instant success? If they were useful, they would be used more. I have made several uses of (a variant of) http://code.activestate.com/recipes/578528-type-checking-using-python-3x-annotations/ Precisely. I don't see that there's a huge body of coders out there just itching to use "Python but with some type information", or we'd be seeing huge amounts of code, well, written in Python with type information. They've been seen as an interesting curiosity, perhaps, but not as "hey look, finally Python's massive problem is solved". So I don't think there's much call for a *new language* on the basis that it's "Python plus type information". I have seen dozens of projects where Python was dismissed because of the lack of static typing, and the lack of static analysis tools. I'm supervising our students during their internship periods in various industrial sectors. Many of these students suggest Python as the development language (they learned it and liked it), and the suggestion is (almost) always rejected, in favor of Java or C# or C/C++. -- Alain. How many tears are shed as a result of these decisions? Or do they spend all afternoon at the pub celebrating as the code has compiled, while the poor, very hard done by Python programmers have to stay behind and test their code? Let's face it, we all know that for a statically compiled language the compiler catches all errors, so there's nothing to worry about. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 05/06/2014 21:07, Alain Ketterlin wrote: Sturla Molden writes: On 05/06/14 10:14, Alain Ketterlin wrote: Type safety. Perhaps. Python has strong type safety. Come on. I don't understand that comment, please explain. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
Chris Angelico writes: > On Fri, Jun 6, 2014 at 6:07 AM, Alain Ketterlin > wrote: >>> Perhaps, perhaps not. My experience is that only a small percentage of >>> the CPU time is spent in the Python interpreter. >> >> Basically, you're saying that a major fraction of python programs is >> written in another language. An interesting argument... > > No, a major fraction of Python program execution time is deep inside > code written in another language. In extreme cases, you might have a > tiny bit of Python glue and the bulk of your code is actually, say, > FORTRAN - such as a hefty numpy number crunch - which lets you take > advantage of multiple cores, since there's no Python code running most > of the time. This is actually what I meant. I find it sad to keep Python such a glue language (the kind of language you throw away when the trend changes---like Perl for example). > And that's counting only CPU time. If you count wall time, your > typical Python program spends most of its time deep inside kernel API > calls, waiting for the user or I/O or something. But this is true of any IO-bound program, whatever the language. I see no reason why Python should be restricted to simple processing tasks. -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
In article ,
Albert-Jan Roskam wrote:
>
- Original Message -
> From: Ian Kelly
> > To: Python
> Cc:
> Sent: Thursday, June 5, 2014
> 10:18 PM
> Subject: Re: Unicode and Python - how often do you index strings?
> >
> On Thu, Jun 5, 2014 at 1:58 PM, Paul Rubin
>
> wrote:
>> Ryan Hiebert writes:
>>> How so? I was
> using line=line[:-1] for removing the trailing newline,
> and
>>> just
> replaced it with rstrip('\n'). What are you doing
> differently?
>>
>>
> rstrip removes all the newlines off the end, whether there are zero or
>>
> multiple.? In perl the difference is chomp vs chop.? line=line[:-1]
>>
> removes one character, that might or might not be a newline.
>
> Given the
> description that the input string is "a textfile line", if
> it has multiple
> newlines then it's invalid.
>
> Personally I tend toward rstrip('\r\n') so
> that I don't have
> to worry
> about files with alternative line
> terminators.
I tend to use: s.rstrip(os.linesep)
> If you want to be really
> picky about removing exactly one line
> terminator, then this captures all
> the relatively modern variations:
> re.sub('\r?\n$|\n?\r$', line, '',
> count=1)
or perhaps: re.sub("[^ \S]+$", "", line)
Just for fun, I took a screen-shot of what this looks like in my
newsreader. URL below. Looks like something chomped on unicode pretty
hard :-)
http://www.panix.com/~roy/unicode.pdf
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pandas question
On 05/06/2014 20:40, Albert-Jan Roskam wrote: Hi, [snipped as I can't answer directly] but see this http://pandas.pydata.org/community.html for support options. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
- Original Message -
> From: Ian Kelly
> To: Python
> Cc:
> Sent: Thursday, June 5, 2014 10:18 PM
> Subject: Re: Unicode and Python - how often do you index strings?
>
> On Thu, Jun 5, 2014 at 1:58 PM, Paul Rubin
> wrote:
>> Ryan Hiebert writes:
>>> How so? I was using line=line[:-1] for removing the trailing newline,
> and
>>> just replaced it with rstrip('\n'). What are you doing
> differently?
>>
>> rstrip removes all the newlines off the end, whether there are zero or
>> multiple. In perl the difference is chomp vs chop. line=line[:-1]
>> removes one character, that might or might not be a newline.
>
> Given the description that the input string is "a textfile line", if
> it has multiple newlines then it's invalid.
>
> Personally I tend toward rstrip('\r\n') so that I don't have
> to worry
> about files with alternative line terminators.
I tend to use: s.rstrip(os.linesep)
> If you want to be really picky about removing exactly one line
> terminator, then this captures all the relatively modern variations:
> re.sub('\r?\n$|\n?\r$', line, '', count=1)
or perhaps: re.sub("[^ \S]+$", "", line)
--
https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
Chris Angelico writes: > On Thu, Jun 5, 2014 at 7:42 PM, Alain Ketterlin > wrote: >> Chris Angelico writes: >> >>> On Thu, Jun 5, 2014 at 6:14 PM, Alain Ketterlin >>> wrote: Swift's memory management is similar to python's (ref. counting). Which makes me think that a subset of python with the same type safety would be an instant success. >>> >>> In the same way that function annotations to give type information >>> were an instant success? >> >> If they were useful, they would be used more. I have made several uses >> of (a variant of) >> >> http://code.activestate.com/recipes/578528-type-checking-using-python-3x-annotations/ > > Precisely. I don't see that there's a huge body of coders out there > just itching to use "Python but with some type information", or we'd > be seeing huge amounts of code, well, written in Python with type > information. They've been seen as an interesting curiosity, perhaps, > but not as "hey look, finally Python's massive problem is solved". So > I don't think there's much call for a *new language* on the basis that > it's "Python plus type information". I have seen dozens of projects where Python was dismissed because of the lack of static typing, and the lack of static analysis tools. I'm supervising our students during their internship periods in various industrial sectors. Many of these students suggest Python as the development language (they learned it and liked it), and the suggestion is (almost) always rejected, in favor of Java or C# or C/C++. -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to use SQLite (sqlite3) more efficiently
On Fri, Jun 6, 2014 at 6:12 AM, R Johnson wrote: > I forgot to mention that the scripts Peter pointed to used REPLACE instead > of INSERT OR REPLACE. The SQLite documentation says that REPLACE is an alias > for INSERT OR REPLACE provided for compatibility with other SQL database > engines. Is there a preference for one or the other? > > I had changed my code from using INSERT OR REPLACE to using REPLACE > (including my new sample), but since then changed it back. I don't care > about compatibility with other database engines, and INSERT OR REPLACE seems > more logical to me, since REPLACE sounds like a synonym for UPDATE. (Suggestion: Always reply to an existing post if it's part of the same thread. Replying to your own post is fine, and it links the thread together nicely.) You can use either, and they're equally non-standard - in my opinion, equally wrong. Whatever feels right to you. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
Terry Reedy : > On 6/5/2014 5:53 AM, Marko Rauhamaa wrote: >> Chris Angelico : >> >>> If the standard streams are so crucial, why are their most obvious >>> interfaces insignificant to you? >> >> I want the standard streams to consume and produce bytes. > > Easy. Read the manual entry for stdxxx. "To write or read binary data > from/to the standard streams, use the underlying binary buffer object. > For example, to write bytes to stdout, use > sys.stdout.buffer.write(b'abc')" This note from the manual is a bit vague: Note that the streams can be replaced with objects (like io.StringIO) that do not support the buffer attribute or the detach() method "Can be replaced" by who? By the Python developers? By me? By random library calls? Does it mean the buffer and detach are not guaranteed to stay with the API? Marko -- https://mail.python.org/mailman/listinfo/python-list
Pandas question
Hi,
I am new to Pandas. I am trying to remove the lower and upper 15 percentiles of
interest rates within a day. The index column is the date. Below is some code,
but how do I apply the trim function day-by-day? I tried using grouped() in
conjunction with apply(), but that turned out to be an efficient way to slow my
computer down and choke it. Any thoughts?
import pandas as pd
records = pd.read_table("blah.csv", sep=";", parse_dates=[2], index_col=2,
low_memory=False)
def trim(df, colname, boundaries=(0.15, 0.85)):
lo = df[colname] >= df[colname].quantile(boundaries[0])
hi = df[colname] <= df[colname].quantile(boundaries[1])
return df[lo & hi]
trimmed = trim(records, 'pct_12m') # this trims across all data, not
day-by-day, which I want.
Oh, and is something like the following possible instead of df[lo & hi]?
df[lo <= df[colname] <= hi] # looks nice, but gives ValueError
Thanks in advance!
Regards,
Albert-Jan
~~
All right, but apart from the sanitation, the medicine, education, wine, public
order, irrigation, roads, a
fresh water system, and public health, what have the Romans ever done for us?
~~
--
https://mail.python.org/mailman/listinfo/python-list
How to use SQLite (sqlite3) more efficiently
I forgot to mention that the scripts Peter pointed to used REPLACE instead of INSERT OR REPLACE. The SQLite documentation says that REPLACE is an alias for INSERT OR REPLACE provided for compatibility with other SQL database engines. Is there a preference for one or the other? I had changed my code from using INSERT OR REPLACE to using REPLACE (including my new sample), but since then changed it back. I don't care about compatibility with other database engines, and INSERT OR REPLACE seems more logical to me, since REPLACE sounds like a synonym for UPDATE. -- Timothy -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On Thu, Jun 5, 2014 at 1:58 PM, Paul Rubin wrote:
> Ryan Hiebert writes:
>> How so? I was using line=line[:-1] for removing the trailing newline, and
>> just replaced it with rstrip('\n'). What are you doing differently?
>
> rstrip removes all the newlines off the end, whether there are zero or
> multiple. In perl the difference is chomp vs chop. line=line[:-1]
> removes one character, that might or might not be a newline.
Given the description that the input string is "a textfile line", if
it has multiple newlines then it's invalid.
Personally I tend toward rstrip('\r\n') so that I don't have to worry
about files with alternative line terminators.
If you want to be really picky about removing exactly one line
terminator, then this captures all the relatively modern variations:
re.sub('\r?\n$|\n?\r$', line, '', count=1)
--
https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On Fri, Jun 6, 2014 at 6:07 AM, Alain Ketterlin wrote: >> Perhaps, perhaps not. My experience is that only a small percentage of >> the CPU time is spent in the Python interpreter. > > Basically, you're saying that a major fraction of python programs is > written in another language. An interesting argument... No, a major fraction of Python program execution time is deep inside code written in another language. In extreme cases, you might have a tiny bit of Python glue and the bulk of your code is actually, say, FORTRAN - such as a hefty numpy number crunch - which lets you take advantage of multiple cores, since there's no Python code running most of the time. And that's counting only CPU time. If you count wall time, your typical Python program spends most of its time deep inside kernel API calls, waiting for the user or I/O or something. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On Thu, Jun 5, 2014 at 2:59 PM, Chris Angelico wrote:
> On Fri, Jun 6, 2014 at 4:52 AM, Ryan Hiebert wrote:
> > 2014-06-05 13:42 GMT-05:00 Johannes Bauer :
> >
> >> On 05.06.2014 20:16, Paul Rubin wrote:
> >> > Johannes Bauer writes:
> >> >> line = line[:-1]
> >> >> Which truncates the trailing "\n" of a textfile line.
> >> >
> >> > use line.rstrip() for that.
> >>
> >> rstrip has different functionality than what I'm doing.
> >
> >
> > How so? I was using line=line[:-1] for removing the trailing newline, and
> > just replaced it with rstrip('\n'). What are you doing differently?
>
> >>> line = "Hello,\nworld!\n\n"
> >>> line[:-1]
> 'Hello,\nworld!\n'
> >>> line.rstrip('\n')
> 'Hello,\nworld!'
>
> If it's guaranteed to end with exactly one newline, then and only then
> will they be identical.
>
> OK, that's not an issue for my case, and additionally I'm using the
open(_, 'U') file iterable, so I shouldn't see multiple trailing newlines
anyway.
--
https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
Sturla Molden writes: > On 05/06/14 10:14, Alain Ketterlin wrote: > >> Type safety. > > Perhaps. Python has strong type safety. Come on. [...] >>(And with it comes better performance ---read battery >> life--- and better static analysis tools, etc.) > > Perhaps, perhaps not. My experience is that only a small percentage of > the CPU time is spent in the Python interpreter. [...] Basically, you're saying that a major fraction of python programs is written in another language. An interesting argument... >> LLVM (an Apple-managed project) for the middle- and back-end, and a >> brand new front-end incorporating a decent type system (including >> optional types for instance). > > Numba uses LLVM. As far as I know, Numba deals only with primitive types. You will gain nothing for classes. (And Numba is a JIT.) > When I compile Cython modules I use LLVM on this computer. Cython is not Python, it is another language, with an incompatible syntax. >> Swift's memory management is similar to python's (ref. counting). Which >> makes me think that a subset of python with the same type safety would >> be an instant success. > > A Python with static typing would effectively be Cython :) I don't think so. The various proposals mentioned elsewhere in this thread give concrete examples of what static typing would look like in Python. -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
Ryan Hiebert writes:
> How so? I was using line=line[:-1] for removing the trailing newline, and
> just replaced it with rstrip('\n'). What are you doing differently?
rstrip removes all the newlines off the end, whether there are zero or
multiple. In perl the difference is chomp vs chop. line=line[:-1]
removes one character, that might or might not be a newline.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On Fri, Jun 6, 2014 at 4:52 AM, Ryan Hiebert wrote:
> 2014-06-05 13:42 GMT-05:00 Johannes Bauer :
>
>> On 05.06.2014 20:16, Paul Rubin wrote:
>> > Johannes Bauer writes:
>> >> line = line[:-1]
>> >> Which truncates the trailing "\n" of a textfile line.
>> >
>> > use line.rstrip() for that.
>>
>> rstrip has different functionality than what I'm doing.
>
>
> How so? I was using line=line[:-1] for removing the trailing newline, and
> just replaced it with rstrip('\n'). What are you doing differently?
>>> line = "Hello,\nworld!\n\n"
>>> line[:-1]
'Hello,\nworld!\n'
>>> line.rstrip('\n')
'Hello,\nworld!'
If it's guaranteed to end with exactly one newline, then and only then
will they be identical.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
2014-06-05 13:42 GMT-05:00 Johannes Bauer :
> On 05.06.2014 20:16, Paul Rubin wrote:
> > Johannes Bauer writes:
> >> line = line[:-1]
> >> Which truncates the trailing "\n" of a textfile line.
> >
> > use line.rstrip() for that.
>
> rstrip has different functionality than what I'm doing.
How so? I was using line=line[:-1] for removing the trailing newline, and
just replaced it with rstrip('\n'). What are you doing differently?
--
https://mail.python.org/mailman/listinfo/python-list
Tabs (was Re: OT: This Swift thing)
On 6/5/2014 4:39 AM, Steven D'Aprano wrote: On Wed, 04 Jun 2014 22:43:05 -0400, Terry Reedy wrote: Many mail readers treat \t as a null char since it actually has no standard translation into screen space. No *single* standard. I challenge that assertion. There are two standard translations into screen space: jump to the next multiple of 8 spaces, or 1 space. You forgot the other tab standards. In the US, the default tab stops are at 1/2 inch intervals, which is 5 or 6 chars for fixed 10 or 12 pitch fonts. Most email is not code from programmers and their bizarre (to a non-programmer) and dis-functional choice (sometimes) of 8 spaces. Treating \t as a single space would be pathetic but standard. Treating it as (up to) 8 spaces would be more useful, and standard. Rendering it as a picture of a banana dancing on the ceiling would be silly and non- standard. Not rendering it at all is even more stupid and less justified. <\t> would be better than nothing. Let us see what Thunderbird does *now*. Maybe complaints prompted a change. Or maybe some clients eat tabs upon sending. 4 spaces 8 spaces tab, which displays as 8 spaces on input 2 tabs Regardless of what shows, there is still no configuration option. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On 6/5/2014 5:53 AM, Marko Rauhamaa wrote: Chris Angelico : If the standard streams are so crucial, why are their most obvious interfaces insignificant to you? I want the standard streams to consume and produce bytes. Easy. Read the manual entry for stdxxx. "To write or read binary data from/to the standard streams, use the underlying binary buffer object. For example, to write bytes to stdout, use sys.stdout.buffer.write(b'abc')" To make it easy, use bound methods. myfilter.p -- import sys sysin = sys.stdin.buffer.read sysout = sys.stdout.buffer.write syserr = sys.stderr.buffer.write --- The same trick of defining bound methods to save both writing and execution time is also useful for text filters when you use sys.stdin.read, etc, more than once in the text. When you try this, please report the result, either way. > I do a lot of system programming and connect processes to each other > with socketpairs, pipes and the like. I have dealt with plugin APIs > that communicate over stdin and stdout. Now you know how to do so on Python 3. Python is clearly on a crusade to make *text* a first class system entity. I don't believe that is possible (without casualties) in the linux world. Python text should only exist inside string objects. You are clearly on a crusade to push a falsehood. Why? On Windows and, I believe, Mac, utf-16 encoded text (C widechar type) *is* a 'first class system entity. The problem Python has with *nix is getting text bytes from the system in an unknown or worse, wrongly-claimed encoding. The Python developers do their best to cope with the differences and peculiarities of the systems it runs on. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 6/5/14 12:18 PM, Michael Torrie wrote: No they won't be used in the same niche. Objective C is certainly not used in the same niche as Python, so why would Swift? I don't expect to see any major OS X app written completely in Python, nor would I expect and of the core frameworks to be written in Python. They will be written in Swift however. OS X apps will indeed be written in Swift; esp if they will be distributed from the Apple Store--- Python apps are streng verboten in Apple land. OTOH, much of my Python work is done on the mac for the mac... just not distributed from the Apple Store. OTOH, it only makes sense to code with Apple's tools if the app is going to be Apple Store ready. OTOH, I don't view the mac as an "Apple" thing. My mac is a *nix clone (freeBSD variant) which is a stable platform for Python coding and debug|test. I won't be using Swift; however, I will be using IDLE. JFTR, Apple should have adopted Python3, IMHO. marcus -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On 05.06.2014 20:16, Paul Rubin wrote: > Johannes Bauer writes: >> line = line[:-1] >> Which truncates the trailing "\n" of a textfile line. > > use line.rstrip() for that. rstrip has different functionality than what I'm doing. Cheers, Johannes -- >> Wo hattest Du das Beben nochmal GENAU vorhergesagt? > Zumindest nicht öffentlich! Ah, der neueste und bis heute genialste Streich unsere großen Kosmologen: Die Geheim-Vorhersage. - Karl Kaos über Rüdiger Thomas in dsa -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
Terry Reedy : > Different OSes *do* have different assumptions. Both MacOSX and > current Windows use (UCS-2 or) UTF-16 for text. Linux can use anything for text; UTF-8 has become a de-facto standard. How text is represented is very different from whether text is a fundamental data type. A fundamental text file is such that ordinary operating system facilities can't see inside the black box (that is, they are *not* encoded as far as the applications go). I have no idea how opaque text files are in Windows or OS-X. > For Windows, at least, the interface is much improved in Python 3. Yes, I get the feeling that Python is reaching out to Windows and OS-X and trying to make linux look like them. > I understand that some, but not all, Latin alphabet *nix programmers > wish that Python 3 continued to be strongly in their favor. But they > are a small minority of the world's programmers, and Python 3 is aimed > at everyone on all systems. Python allows linux programmers to write native linux programs. Maybe it allows Windows programmers to write native Windows programs. I certainly hope so. I don't want to have to write Windows programs that kinda run on linux. Java suffers from that: no "import os" in Java. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: How to use SQLite (sqlite3) more efficiently
On Fri, Jun 6, 2014 at 3:42 AM, R Johnson
wrote:
> Thank you all for your replies and suggestions.
>
> To Chris's "two small points":
> I saw that using the mailing list was recommended to several other people
> who posted here using Google Groups, so I thought it might be recommended to
> me as well sometime :). I'll try to use it from now on.
> My code was tested on Python 2.7.6 on Windows 8.1 (and I just installed
> Python 2.7.7 yesterday).
Thanks! Good to know.
>> There's a general principle in Python APIs that a "mode switch"
>> parameter isn't a good thing. Even more strongly, I would be very
>> surprised if attempting to set a blank description deleted the row
>> instead of setting the description to blank. My recommendation: Split
>> this into two functions, set_description and delete_language. In
>> delete_language, just unconditionally delete, don't bother checking
>> for the row's presence first.
>
> I agree for the case of the sample code I showed here (which was really just
> a scaled-down version of some of the functions in my program). But in my
> actual program, I am using SQLite to load and save information from a
> wxPython GUI, where it's more practical to call a single save function.
> Below is the actual function (that's part of a class in my program):
>
> def save_text(self):
> if not self.editor.IsModified():
> return
> if not self.editor.IsEmpty():
> stream = cStringIO.StringIO()
> self.editor.GetBuffer().SaveStream(stream,
> richtext.RICHTEXT_TYPE_XML)
> self.conn.execute("REPLACE INTO notes VALUES(?,?)",
> (self.db_key, stream.getvalue()))
> self.editor.SetModified(False)
> else:
> self.conn.execute("DELETE FROM notes WHERE topic=?",
> (self.db_key,))
I was actually going to consider saying "don't wrap stuff up in
functions like this at all, just do your SQL queries directly in
whatever needs them", but apparently that's what you're already doing.
The mode-switch is right where it should be, at the point where the
switch happens. Your code says "if the editor is empty, do something
different", which is the easiest and clearest way to describe this.
>> > set_description(conn, "Assembly",
>> > "Making Easy Things Very Hard & Hard Things Impossible")
>>
>> Hey, that's not fair! Assembly language makes some hard things really
>> easy, like segfaulting your process. Credit where it's due! :)
>
> OK, I'll admit that I don't know Assembly :). How about the paradox "Making
> Easy Things Hard & Hard Things Easy"? Although that might make my
> description of C++ too unfair; suggestions for improvements to my language
> descriptions are welcome :).
Hehe. As I'm sure you're aware, this has absolutely nothing to do with
your SQL or Python code.
>>While /maybe/ not required for a SELECT operation, I'd put a
>>conn.commit() somewhere in there before the return(s). The standard for
>> Python DB-API interfaces is that auto-commit is turned off -- meaning the
>> SELECT has started a database transaction.
>
> I don't exactly understand why conn.commit() should be called there. I
> thought it's only necessary to call it when the database has been changed,
> which a SELECT call doesn't do. Am I misunderstanding something here?
I can't speak for SQLite3, but here's how it is with full-on databases
like DB2 and PostgreSQL. (I suspect sqlite may be a little simplified
from this, but it's likely to be the same. In any case, get into good
habits now and you won't run into problems later.) Whenever you start
working with a database, you open a unit of work, aka a transaction.
This transaction continues until it is terminated, either by a commit
or rollback, or by something breaking your connection to the db (if
your program or the database shuts down abruptly, your transaction
will be rolled back). While it's alive, it holds certain resources,
mainly locks. It's fairly obvious that updating should acquire a lock
- if some other program tries to update the same row of the same
table, it should be blocked - but even reading will grab some locks.
Depending on the database and the config, your reads might prevent
anyone else from writing, or (as with PostgreSQL) they might simply
prevent someone else from dropping that table altogether, but there'll
still be locks.
The most important thing to remember is that your commit/rollback
points should match the logical unit of work that you're doing. Figure
out what your program would do if it got shut down abruptly in the
middle of the database work - if there's any way that something could
be half-done and illogical, you should avoid committing in there,
until you've brought the database back to a stable state. Often that
means never committing until the very end of the program flow;
sometimes it means committing every little piece you do; it could be
anywhere in between.
>> >with conn:
>>
>> This isn't really doing anything useful. Yo
Re: Unicode and Python - how often do you index strings?
Johannes Bauer writes: > line = line[:-1] > Which truncates the trailing "\n" of a textfile line. use line.rstrip() for that. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On 6/5/2014 10:45 AM, Marko Rauhamaa wrote: Mostly I'm saying Python3 will not be able to hide the fact that linux data consists of bytes. It shouldn't even try. The linux OS outside the Python process talks bytes, not strings. A text file is a binary file wrapped with a codex to translate to and from a universal text format on input and output. Much of the time, the wrapping is a great user convenience. Since the wrapping is optional, nothing is forced or really hidden. A different OS might have different assumptions. Different OSes *do* have different assumptions. Both MacOSX and current Windows use (UCS-2 or) UTF-16 for text. It seems that unicode strings are better than ascii+??? strings as a universal basis for OS interfacing. For Windows, at least, the interface is much improved in Python 3. I understand that some, but not all, Latin alphabet *nix programmers wish that Python 3 continued to be strongly in their favor. But they are a small minority of the world's programmers, and Python 3 is aimed at everyone on all systems. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
How to use SQLite (sqlite3) more efficiently
Thank you all for your replies and suggestions.
To Chris's "two small points":
I saw that using the mailing list was recommended to several other
people who posted here using Google Groups, so I thought it might be
recommended to me as well sometime :). I'll try to use it from now on.
My code was tested on Python 2.7.6 on Windows 8.1 (and I just installed
Python 2.7.7 yesterday).
> There's a general principle in Python APIs that a "mode switch"
> parameter isn't a good thing. Even more strongly, I would be very
> surprised if attempting to set a blank description deleted the row
> instead of setting the description to blank. My recommendation: Split
> this into two functions, set_description and delete_language. In
> delete_language, just unconditionally delete, don't bother checking
> for the row's presence first.
I agree for the case of the sample code I showed here (which was really
just a scaled-down version of some of the functions in my program). But
in my actual program, I am using SQLite to load and save information
from a wxPython GUI, where it's more practical to call a single save
function. Below is the actual function (that's part of a class in my
program):
def save_text(self):
if not self.editor.IsModified():
return
if not self.editor.IsEmpty():
stream = cStringIO.StringIO()
self.editor.GetBuffer().SaveStream(stream,
richtext.RICHTEXT_TYPE_XML)
self.conn.execute("REPLACE INTO notes VALUES(?,?)",
(self.db_key, stream.getvalue()))
self.editor.SetModified(False)
else:
self.conn.execute("DELETE FROM notes WHERE topic=?",
(self.db_key,))
(Even if you're not familiar with wxPython, it should be fairly easy to
figure out what the code is doing. It's just saving some XML from a rich
text editor to a StringIO object, and then to an SQLite database.)
> > set_description(conn, "Assembly",
> > "Making Easy Things Very Hard & Hard Things Impossible")
>
> Hey, that's not fair! Assembly language makes some hard things really
> easy, like segfaulting your process. Credit where it's due! :)
OK, I'll admit that I don't know Assembly :). How about the paradox
"Making Easy Things Hard & Hard Things Easy"? Although that might make
my description of C++ too unfair; suggestions for improvements to my
language descriptions are welcome :).
>While /maybe/ not required for a SELECT operation, I'd put a
>conn.commit() somewhere in there before the return(s). The standard for
> Python DB-API interfaces is that auto-commit is turned off -- meaning the
> SELECT has started a database transaction.
I don't exactly understand why conn.commit() should be called there. I
thought it's only necessary to call it when the database has been
changed, which a SELECT call doesn't do. Am I misunderstanding something
here?
> >with conn:
>
> This isn't really doing anything useful. You aren't opening a new
> connection object, so there isn't really anything to close on block
exit.
See
https://docs.python.org/2/library/sqlite3.html#using-the-connection-as-a-context-manager.
I removed it from my code, though, because it doesn't really seem necessary.
I've attached some new sample code in which I've attempted to correct
various things that you mentioned. The links Peter pointed to were also
helpful to show me some improvements I could make to my code. I'd be
happy to hear any suggestions that anyone may have to improve the code
further.
-- Timothy
from __future__ import print_function
import sqlite3
def get_description(conn, description):
row = conn.execute("SELECT description FROM languages WHERE name=?",
(description,)).fetchone()
if row:
return row[0]
def set_description(conn, name, description):
conn.execute("REPLACE INTO languages VALUES(?,?)", (name, description))
conn.commit()
def delete_language(conn, name):
conn.execute("DELETE FROM languages WHERE name=?", (name,))
conn.commit()
conn = sqlite3.connect(":memory:")
conn.execute("CREATE TABLE IF NOT EXISTS languages(name TEXT PRIMARY KEY, " \
"description TEXT NOT NULL)")
set_description(conn, "Perl", "Making Easy Things Easy & Hard Things Possible")
set_description(conn, "Python", "Making Easy Things Easier & Hard Things Easy")
set_description(conn, "C++", "Making Easy Things Hard & Hard Things Harder")
for language in ("Perl", "Python", "C++"):
print("%s: %s" % (language, get_description(conn, language)))
set_description(conn, "Assembly", "Making Easy Things Very Hard & " \
"Hard Things Impossible (Hey, that's not fair!)")
print("Assembly: %s" % get_description(conn, "Assembly"))
set_description(conn, "Assembly",
"Making Easy Things Hard & Hard Things Easy")
print("Assembly: %s" % get_description(conn, "Assembly")) # Should be changed
delete_language(conn, "Assembly")
print("Assembly: %s" % get_description(conn, "Assembly")) # Should be None
conn.close
Re: Forking PyPI package
On Fri, Jun 6, 2014 at 2:56 AM, Wiktor wrote: > I guess, I'll try to do what Chris proposed. Forget about this > implementation and write python script from the scratch looking only at the > original JavaScript version. :-/ Sadly, that may be your only safe option. Let this be a lesson to all: If you want something to be free software, make it very clear, because "it looks like he meant that to be open source" just isn't enough :( ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Fri, Jun 6, 2014 at 2:54 AM, Rustom Mody wrote: > On Thursday, June 5, 2014 9:42:28 PM UTC+5:30, Chris Angelico wrote: >> On Fri, Jun 6, 2014 at 1:33 AM, Steven D'Aprano wrote: >> > In the Unix world, text formats and text >> > processing is much more common in user-space apps than binary processing. >> > Perhaps the definitive explanation and celebration of the Unix way is >> > Eric Raymond's "The Art Of Unix Programming": >> > http://www.catb.org/esr/writings/taoup/html/ch05s01.html > >> Specifically, this from the opening paragraph: >> """ >> Text streams are a valuable universal format because they're easy for >> human beings to read, write, and edit without specialized tools. These >> formats are (or can be designed to be) transparent. >> """ > > A fact that stops being true when you tie up text with encodings. > For two reasons: > > 1. The function/pair encode/decode mapping between byte-string and text >cannot be a bijection because the byte-string set is larger than the text >set. This is the error that Armin was hit by > > 2. Since there is not one but a zillion encodings possible we are not >talking of one (possibly universal) data structure but a zillion >ones: "Text streams are a universal format" - which encoding-ed >form of text?? As soon as you store or transmit ANY form of information, you need to worry about encodings. Ever heard of this thing called "network byte order"? It's part of taming the wilds of integer encodings. The theory is that the LC environment variables will carry all that crucial out-of-band information about encodings, and while the practice isn't perfect, it does still mean that there is such a thing as a text stream. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On Thu, 05 Jun 2014 18:15:31 +0100, Mark Lawrence wrote: >>> >> The problem is that thing look fine in google groups. What helps is >> getting to see what the mess looks like from Thunderbird or equivalent. >> >> > Wrong. 99.99% of people when asked politely take action so there is no > problem. The remaining 0.01% consists of one complete ignoramus. Who has actively stated he will not change. pretty much the same attitude he has constantly saying pythons unicode implementation is broken* without any valid supporting evidence. * Not just incomplete or inefficient but irrevocably broken. -- Yow! It's some people inside the wall! This is better than mopping! -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Fri, Jun 6, 2014 at 2:52 AM, Marko Rauhamaa wrote: > That linux text is not the same thing as Python's text. Conceptually, > Python text is a sequence of 32-bit integers. Linux text is a sequence > of 8-bit integers. Point of terminology: Linux is the kernel, everything you say below here is talking about particular programs. From what I understand, bash (just another Unix program) treats strings as sequences of codepoints, just as Python does; though its string manipulation is not nearly as rich as Python's, so it's harder to prove. Python is itself a Unix program, so you can do the exact same proofs and demonstrate that Linux is clearly Unicode-aware. It's not Linux you're testing. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Thu, Jun 5, 2014 at 10:17 AM, Robin Becker wrote: > in python 2 str and unicode were much more comparable. On balance I think > just reversing them ie str --> bytes and unicode --> str was probably the > right thing to do if the default conversions had been turned off. However > making bytes a crippled thing was wrong. How should e.g. bytes.upper() be implemented then? The correct behavior is entirely dependent on the encoding. Python 2 just assumes ASCII, which at best will correctly upper-case some subset of the string and leave the rest unchanged, and at worst could corrupt the string entirely. There are some things that were dropped that should not have been, but my impression is that those are being worked on, for example % formatting in PEP 461. -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On 4 June 2014 15:50, Michael Torrie wrote: > On 06/04/2014 12:50 AM, wxjmfa...@gmail.com wrote: >> [Things] > > [Reply to things] Please. Just don't. -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 06/05/2014 10:09 AM, Sturla Molden wrote: > On 05/06/14 16:33, Michael Torrie wrote: > > > In any case I'm a bit surprised by people comparing Python to Swift at > > all, implying that Python would have worked just as well and Apple > > should have chosen it to replace Objective C. > > Because if you look at the spec, Swift is essentially a statically typed > Python. I guess I'm not following your argument. Are you saying Swift should adopted Python syntax (similar to the .net language Boo) or are you saying Apple should have adopted Python instead? > Swift and Python will also be used in the same niche. C will still be > used for low-level stuff. Swift is not a replacement for C. It's a > replacement for Objective-C. No they won't be used in the same niche. Objective C is certainly not used in the same niche as Python, so why would Swift? I don't expect to see any major OS X app written completely in Python, nor would I expect and of the core frameworks to be written in Python. They will be written in Swift however. > > Swift provides a cleaner system > > for developers to work in than Obj C did (which, by the way has > > reference counting), but carries on the same object model that > > developers are used to (and existing frameworks use). > > That is what PyObjC does as well. Not quite what I mean. As you said yourself, Swift is aiming to replace ObjC. Thus core system frameworks will slowly be replaced over time with frameworks written in Swift (binary, compiled frameworks). So you'll be using PySwift in the future instead of PyObjC, which should be an easy bridge to create since the object model is not changing. -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On 05/06/2014 16:57, Mark H Harris wrote:
On 6/5/14 10:39 AM, alister wrote:
{snipped all the mess}
And you have may time been given a link explaining the problems with
posting g=from google groups but deliberately choose to not make your
replys readable.
The problem is that thing look fine in google groups. What helps is
getting to see what the mess looks like from Thunderbird or equivalent.
Wrong. 99.99% of people when asked politely take action so there is no
problem. The remaining 0.01% consists of one complete ignoramus.
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
---
This email is free from viruses and malware because avast! Antivirus protection
is active.
http://www.avast.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Fri, Jun 6, 2014 at 2:17 AM, Robin Becker wrote: > in python 2 str and unicode were much more comparable. On balance I think > just reversing them ie str --> bytes and unicode --> str was probably the > right thing to do if the default conversions had been turned off. However > making bytes a crippled thing was wrong. It's easy to build up functionality after the event. Maybe reportlab will have lots of hacks to support both 2.7 and 3.3, but in a few years you'll be able to say "supports 2.7 and 3.5" and take advantage of percent formatting and whatever else is added. But this is just the way that languages develop; you use them, you find what isn't easy, and you fix it. The nature of stability is that it takes time before you can depend on freshly-written functionality (contrast the extreme instability of running the version from source control - stuff might be fixed at any time, but you have to do all the work yourself to make sure your dependencies line up), but over time, you can depend on improvements making their way out there. Can you point to specific areas in which the bytes type is "crippled"? Comparing either to the Py2 str or the Py3 str, or to anything else? The Python core devs are listening, as evidenced by PEP 461. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: any wheel experts
On Thursday, June 5, 2014 10:21:06 PM UTC+5:30, Robin Becker wrote: > I used to create exe files for windows, but the latest and greatest concept > is > wheels .whl files. If someone here knows (and answers!) great. Else you'll probably get more info here: https://groups.google.com/forum/?pli=1#!forum/python-virtualenv -- https://mail.python.org/mailman/listinfo/python-list
Re: Forking PyPI package
On Thu, 29 May 2014 15:54:09 -0600, Ian Kelly wrote:
> On Thu, May 29, 2014 at 1:40 AM, Chris Angelico wrote:
>> If you absolutely can't get in touch with him, the only option is to
>> go back to the original protocol and manually reimplement it,
>> completely ignoring this code. It's sad but true; some code dies
>> because of a trivial thing like "Oops, I forgot to actually say that
>> this is MIT-licensed".
>
> The second part of that is that the code should actually *include* the
> license text. Just writing "BSD license" somewhere on the website or
> in package metadata is annoyingly common but somewhat questionable in
> how a judge might interpret it. For instance, there at least four
> different versions of the BSD license; which one did you mean?
OK, it's almost week now and I have no response from author of that
script.
Just like you said, there's only inscription "BSD license" on PYPI
website, and in 'PKG-INFO' and 'setup.py' files. No 'readme.txt' or
'license.txt' is included. I can see now, that in fact it means that script
isn't published under any BSD license.
I guess, I'll try to do what Chris proposed. Forget about this
implementation and write python script from the scratch looking only at the
original JavaScript version. :-/
Thank you guys.
--
Best regards, Wiktor Matuszewski
'py{}@wu{}em.pl'.format('wkm', 'ka')
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Thursday, June 5, 2014 9:42:28 PM UTC+5:30, Chris Angelico wrote: > On Fri, Jun 6, 2014 at 1:33 AM, Steven D'Aprano wrote: > > In the Unix world, text formats and text > > processing is much more common in user-space apps than binary processing. > > Perhaps the definitive explanation and celebration of the Unix way is > > Eric Raymond's "The Art Of Unix Programming": > > http://www.catb.org/esr/writings/taoup/html/ch05s01.html > Specifically, this from the opening paragraph: > """ > Text streams are a valuable universal format because they're easy for > human beings to read, write, and edit without specialized tools. These > formats are (or can be designed to be) transparent. > """ A fact that stops being true when you tie up text with encodings. For two reasons: 1. The function/pair encode/decode mapping between byte-string and text cannot be a bijection because the byte-string set is larger than the text set. This is the error that Armin was hit by 2. Since there is not one but a zillion encodings possible we are not talking of one (possibly universal) data structure but a zillion ones: "Text streams are a universal format" - which encoding-ed form of text?? -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
Steven D'Aprano : > Nevertheless, there are important abstractions that are written on top > of the bytes layer, and in the Unix and Linux world, the most > important abstraction is *text*. In the Unix world, text formats and > text processing is much more common in user-space apps than binary > processing. That linux text is not the same thing as Python's text. Conceptually, Python text is a sequence of 32-bit integers. Linux text is a sequence of 8-bit integers. It is great that lots of computer-to-computer formats are encoded in ASCII (~ UTF-8). However, nowhere in linux is there a real abstraction layer that processes Python-esque text. Case in point: $ env | grep UTF LANG=en_US.UTF-8 $ od -c <<<"Hyvää yötä" # "Good night" in Finnish 000 H y v 303 244 303 244 y 303 266 t 303 244 \n 017 The "od" utility is asked to display its input as characters. The locale info gives a hint that all text data is in UTF-8. Yet what comes out is bytes. How about: $ wc -c <<<"Hyvää yötä" 15 $ tr 'ä' 'a' <<<"Hyvää yötä" Hyv ya�taa Grep is smarter: $ grep v...y <<<"Hyvää yötä" Hyvää yötä which is why you should always prefix "grep" with LC_ALL=C in your scripts (makes it far faster, too). Marko -- https://mail.python.org/mailman/listinfo/python-list
any wheel experts
I used to create exe files for windows, but the latest and greatest concept is wheels .whl files. Does anyone know how to make the created exes that the wheel can install have an icon for windows? How is one supposed to make the wheel install a desktop short cut etc etc? The wheel is just a zip so is there perhaps a way to make the wheel installation run a post installation script? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Thu, 05 Jun 2014 17:17:05 +0100, Robin Becker wrote: > Bytes are the underlying > concept and should have remained so for simplicity's sake. Bytes are the underlying concept for classes too. Do you think that an opaque unstructured blob of bytes is "simpler" to use than a class? How would an unstructured blob of bytes be simpler to use than an array of multi-byte characters? Earlier: > I dislike the current model, but that's because I had a lot of stuff to > convert and probably made a bunch of blunders. The reportlab code is > now a mess of hacks to keep it alive for 2.7 & >=3.3; Although I've been critical of many of your statements, I am sympathetic to your pain. There's no doubt that that the transition from the old, broken system of bytes masquerading as text can be hard, especially to those who never quite get past the misleading and false paradigm that "bytes are ASCII". It may have been that there were better ways to have updated to 3.3; perhaps you were merely unfortunate to have updated too early, and had you waited to 3.4 or 3.5 things would have been better. I don't know. But whatever the situation, and despite our differences of opinion about Unicode, THANK YOU for having updated ReportLabs to 3.3. -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Thu, 05 Jun 2014 16:37:23 +0100, Robin Becker wrote: > In python 3 byte strings > are second class which I think is wrong It certainly is wrong. bytes are just as much a first-class built-in type as list, int, float, bool, set, tuple and str. There may be missing functionality (relatively easy to add new functionality), and even poor design choices (like the foolish decision to have bytes display as if they were ASCII-ish strings, a silly mistake that simply reinforces the myth that bytes and ASCII are synonymous). Python 3.4 and 3.5 are in the process of rectifying as many of these mistakes as possible, e.g. adding back % formatting. But a few mistakes in the design of bytes' API no more makes it "second-class" than the lack of dict.contains_value() method makes dict "second-class". By all means ask for better bytes functionality. But don't libel Python by pretending that bytes is anything less than one of the most important and fundamental types in the language. bytes are so important that there are TWO implementations for them, a mutable and immutable version (bytearray and bytes), while text strings only have an immutable version. -- Steven D'Aprano http://import-that.dreamwidth.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On 05/06/2014 16:50, Chris Angelico wrote: .. I wouldn't say they're second-class; it's more that the bytes type was considered to be more like a list of ints than like a Unicode string, and now that there are a few years' worth of real-world usage information to learn from, it's known that some more string-like operations will be extremely helpful. So now they're being added, which I agree is a good thing. in python 2 str and unicode were much more comparable. On balance I think just reversing them ie str --> bytes and unicode --> str was probably the right thing to do if the default conversions had been turned off. However making bytes a crippled thing was wrong. Whether b"a"[0] should be b'a' or ord(b'a') is another sticking point. The Py2 str does the first, the Py3 bytes does the second. That one's a bit hard to change, but what I'm not sure of is how significant this is to new-build Py3 code. Obviously it's a barrier to porting, but is it important on its own? However, that's still not really "byte strings are second class". .. I dislike the current model, but that's because I had a lot of stuff to convert and probably made a bunch of blunders. The reportlab code is now a mess of hacks to keep it alive for 2.7 & >=3.3; I'm probably never going to be convinced that uncode types are good. Bytes are the underlying concept and should have remained so for simplicity's sake. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 has some deadly infection
On Fri, Jun 6, 2014 at 1:33 AM, Steven D'Aprano wrote: > In the Unix world, text formats and text > processing is much more common in user-space apps than binary processing. > Perhaps the definitive explanation and celebration of the Unix way is > Eric Raymond's "The Art Of Unix Programming": > > http://www.catb.org/esr/writings/taoup/html/ch05s01.html Specifically, this from the opening paragraph: """ Text streams are a valuable universal format because they're easy for human beings to read, write, and edit without specialized tools. These formats are (or can be designed to be) transparent. """ He goes on to talk about network protocols, one of the best examples of this. I've idly speculated at times about the possibility of rewriting the Magic: The Gathering Online back-end with a view to making it easier to work with. Among other changes, I'd be wanting to make the client-server communication be plain text (an SMTP-style of protocol), with an external layer of encryption (TLS). This would mean that: 1) Internal testing can be done without TLS, making the communication absolutely transparent, easy to debug, easy to watch, everything. Adding TLS later would have zero impact on the critical code internally - it's just a layer around the outside. 2) Upgrades to crypto can simply follow industry best-practice. (Reminder, to anyone who might have been mad enough to consider this: DO NOT roll your own crypto! Ever! Even if you use a good library for the heavy lifting!) 3) A debug log of what the client has sent and received could be included, even in production, at very low cost. You don't need to decode packets and pretty-print them - you just take the lines of text, maybe adorn or color them according to which were sent/received, and dump them into a display box or log file somewhere. 4) The server is forced to acknowledge that the client might not be the one it expected. Not only do you get better security that way, but you could also call this a feature. 5) Therefore, you can debug the system with a simple TELNET or MUD client (okay, most MUD clients don't do SSL, but you can use "openssl s_client"). As someone who's debugged myriad issues using his trusty MUD client, I consider this to be a *huge* advantage. All it takes is a few simple rules, like: All communication is text, encoded down the wire as UTF-8, and consists of lines (terminated by U+000A) which consist of a word, a U+0020 space, and then parameters to the command. There, that's a rigorous definition that covers everything you'll need of it; compare with what Flash uses, by default: https://en.wikipedia.org/wiki/Action_Message_Format Sure, it might be slightly more compact going down the wire; but what do you really gain? Text wins. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: OT: This Swift thing
On 05/06/14 16:33, Michael Torrie wrote: > In any case I'm a bit surprised by people comparing Python to Swift at > all, implying that Python would have worked just as well and Apple > should have chosen it to replace Objective C. Because if you look at the spec, Swift is essentially a statically typed Python. Swift and Python will also be used in the same niche. C will still be used for low-level stuff. Swift is not a replacement for C. It's a replacement for Objective-C. > Swift provides a cleaner system > for developers to work in than Obj C did (which, by the way has > reference counting), but carries on the same object model that > developers are used to (and existing frameworks use). That is what PyObjC does as well. Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On 04.06.2014 02:39, Chris Angelico wrote: > I know the collective experience of python-list can't fail to bring up > a few solid examples here :) Just also grepped lots of code and have surprisingly few instances of index-search. Most are with constant indices. One particular example that comes up a lot is line = line[:-1] Which truncates the trailing "\n" of a textfile line. Then some indexing in the form of negative = (line[0] == "-") All in all I'm actually a bit surprised this isn't too common. Cheers, Johannes -- >> Wo hattest Du das Beben nochmal GENAU vorhergesagt? > Zumindest nicht öffentlich! Ah, der neueste und bis heute genialste Streich unsere großen Kosmologen: Die Geheim-Vorhersage. - Karl Kaos über Rüdiger Thomas in dsa -- https://mail.python.org/mailman/listinfo/python-list
Re: Adding R squared value to scatter plot
On Wednesday, May 21, 2014 1:30:16 PM UTC+1, Jason Swails wrote:
>
>
>
>
>
>
> On Wed, May 21, 2014 at 7:59 AM, Jamie Mitchell wrote:
>
> I have made a plot using the following code:
>
>
>
> python2.7
>
> import netCDF4
>
> import matplotlib.pyplot as plt
>
> import numpy as np
>
>
>
> swh_Q0_con_sw=netCDF4.Dataset('/data/cr1/jmitchel/Q0/swh/controlperiod/south_west/swhcontrol_swest_annavg1D.nc','r')
>
> hs_Q0_con_sw=swh_Q0_con_sw.variables['hs'][:]
>
> swh_Q3_con_sw=netCDF4.Dataset('/data/cr1/jmitchel/Q3/swh/controlperiod/south_west/swhcontrol_swest_annavg1D.nc','r')
>
> hs_Q3_con_sw=swh_Q3_con_sw.variables['hs'][:]
>
> swh_Q4_con_sw=netCDF4.Dataset('/data/cr1/jmitchel/Q4/swh/controlperiod/south_west/swhcontrol_swest_annavg1D.nc','r')
>
> hs_Q4_con_sw=swh_Q4_con_sw.variables['hs'][:]
>
> swh_Q14_con_sw=netCDF4.Dataset('/data/cr1/jmitchel/Q14/swh/controlperiod/south_west/swhcontrol_swest_annavg1D.nc','r')
>
> hs_Q14_con_sw=swh_Q14_con_sw.variables['hs'][:]
>
> swh_Q16_con_sw=netCDF4.Dataset('/data/cr1/jmitchel/Q16/swh/controlperiod/south_west/swhcontrol_swest_annavg1D.nc','r')
>
> hs_Q16_con_sw=swh_Q16_con_sw.variables['hs'][:]
>
> swh_Q0_fut_sw=netCDF4.Dataset('/data/cr1/jmitchel/Q0/swh/2050s/south_west/swh2050s_swest_annavg1D.nc','r')
>
> hs_Q0_fut_sw=swh_Q0_fut_sw.variables['hs'][:]
>
> swh_Q3_fut_sw=netCDF4.Dataset('/data/cr1/jmitchel/Q3/swh/2050s/south_west/swh2050s_swest_annavg1D.nc','r')
>
> hs_Q3_fut_sw=swh_Q3_fut_sw.variables['hs'][:]
>
> swh_Q4_fut_sw=netCDF4.Dataset('/data/cr1/jmitchel/Q4/swh/2050s/south_west/swh2050s_swest_annavg1D.nc','r')
>
> hs_Q4_fut_sw=swh_Q4_fut_sw.variables['hs'][:]
>
> swh_Q14_fut_sw=netCDF4.Dataset('/data/cr1/jmitchel/Q14/swh/2050s/south_west/swh2050s_swest_annavg1D.nc','r')
>
> hs_Q14_fut_sw=swh_Q14_fut_sw.variables['hs'][:]
>
> swh_Q16_fut_sw=netCDF4.Dataset('/data/cr1/jmitchel/Q16/swh/2050s/south_west/swh2050s_swest_annavg1D.nc','r')
>
> hs_Q16_fut_sw=swh_Q16_fut_sw.variables['hs'][:]
>
>
>
> fit_Q0_sw=np.polyfit(hs_Q0_con_sw,hs_Q0_fut_sw,1)
>
> fit_fn_Q0_sw=np.poly1d(fit_Q0_sw)
>
>
>
> plt.plot(hs_Q0_con_sw,hs_Q0_fut_sw,'g.')
>
> plt.plot(hs_Q0_con_sw,fit_fn_Q0_sw(hs_Q0_con_sw),'g',label='Q0 no pert')
>
>
>
> fit_Q3_sw=np.polyfit(hs_Q3_con_sw,hs_Q3_fut_sw,1)
>
> fit_fn_Q3_sw=np.poly1d(fit_Q3_sw)
>
>
>
> plt.plot(hs_Q3_con_sw,hs_Q3_fut_sw,'b.')
>
> plt.plot(hs_Q3_con_sw,fit_fn_Q3_sw(hs_Q3_con_sw),'b',label='Q3 low sens')
>
>
>
> fit_Q4_sw=np.polyfit(hs_Q4_con_sw,hs_Q4_fut_sw,1)
>
> fit_fn_Q4_sw=np.poly1d(fit_Q4_sw)
>
>
>
> plt.plot(hs_Q4_con_sw,hs_Q4_fut_sw,'y.')
>
> plt.plot(hs_Q4_con_sw,fit_fn_Q4_sw(hs_Q4_con_sw),'y',label='Q4 low sens')
>
>
>
> fit_Q14_sw=np.polyfit(hs_Q14_con_sw,hs_Q14_fut_sw,1)
>
> fit_fn_Q14_sw=np.poly1d(fit_Q14_sw)
>
>
>
> plt.plot(hs_Q14_con_sw,hs_Q14_fut_sw,'r.')
>
> plt.plot(hs_Q14_con_sw,fit_fn_Q14_sw(hs_Q14_con_sw),'r',label='Q14 high sens')
>
>
>
> fit_Q16_sw=np.polyfit(hs_Q16_con_sw,hs_Q16_fut_sw,1)
>
> fit_fn_Q16_sw=np.poly1d(fit_Q16_sw)
>
>
>
> plt.plot(hs_Q16_con_sw,hs_Q16_fut_sw,'c.')
>
> plt.plot(hs_Q16_con_sw,fit_fn_Q16_sw(hs_Q16_con_sw),'c',label='Q16 high sens')
>
>
>
> plt.legend(loc='best')
>
> plt.xlabel('Significant Wave Height annual averages NW Scotland 1981-2010')
>
> plt.ylabel('Significant Wave Height annual averages NW Scotland 2040-2069')
>
> plt.title('Scatter plot of Significant Wave Height')
>
> plt.show()
>
>
>
> --
>
>
>
> What I would like to do is display the R squared value next to the line of
> best fits that I have made.
>
>
>
> Does anyone know how to do this with matplotlib?
>
>
>
> You can add plain text or annotations with arrows using any of the API
> functions described here: http://matplotlib.org/1.3.1/users/text_intro.html
> (information specifically regarding the text call is here:
> http://matplotlib.org/1.3.1/api/pyplot_api.html#matplotlib.pyplot.text)
>
>
>
> You can also use LaTeX typesetting here, so you can make the text something
> like r'$R^2$' to display R^2 with "nice" typesetting. (I typically use raw
> strings for matplotlib text strings with LaTeX formulas in them since LaTeX
> makes extensive use of the \ character.)
>
>
>
> The onus is on you, the programmer, to determine _where_ on the plot you want
> the text to appear. Since you know what you are plotting, you can write a
> quick helper function that will compute the optimal (to you) location for the
> label to occur based on where things are drawn on the canvas. There is a
> _lot_ of flexibility here so you should be able to get your text looking
> exactly how (and where) you want it.
>
>
>
> Hope this helps,
> Jason
>
>
> --
>
> Jason M. Swails
> BioMaPS,
> Rutgers University
> Postdoctoral Researcher
Hi Jason,
Thank you for your swift response - you solved my problem!
Sorry I took a while to get back to you.
Thanks again,
Jamie
--
https://mail.python.org/mailman/listinfo/python-list
Re: Unicode and Python - how often do you index strings?
On 6/5/14 10:39 AM, alister wrote:
{snipped all the mess}
And you have may time been given a link explaining the problems with
posting g=from google groups but deliberately choose to not make your
replys readable.
The problem is that thing look fine in google groups. What helps is
getting to see what the mess looks like from Thunderbird or equivalent.
--
https://mail.python.org/mailman/listinfo/python-list
Matplotlib - specifying bin widths
Hello all! Instead of setting the number of bins I want to set the bin width. I would like my bins to go from 1.7 to 2.4 in steps of 0.05. How do I say this in the code? Cheers, Jamie -- https://mail.python.org/mailman/listinfo/python-list
