Re: django question
On Monday, January 6, 2014 8:57:22 PM UTC-5, Roy Smith wrote: > Yes, exactly. There's nothing magic about a django view. It's just a > function which is passed an instance of HttpRequest (and possibly a few > other things, depending on your url mapping), and which is expected to > return an instance of HttpResponse. Within that framework, it can call > any other functions it wants. > > For example, http://legalipsum.com/ is a silly little site I built in > django. Here's the view for the home page: Nice! > Notice how the view knows nothing about generating the actual markov > text. That's in another module, which lives somewhere on my PYTHONPATH. > ALso, the view knows nothing about how the page is laid out; only the > templates know that. If I decided to redo this in tornado or flask, > whatever, I would need to rewrite my view, but there's not much to > rewrite. Most of the logic is in the Markov chainer, and that would > cary over to the new implementation unchanged. > > BTW, my suggestion to keep business logic and presentation code distinct > isn't unique to django, it's a good idea in pretty much all systems. Thanks for these points, helpful to see in practice. I'm trying to be more mindful of good coding practices, and this will be helpful as I continue to learn Django and making web applications generally. -- https://mail.python.org/mailman/listinfo/python-list
nested dictionaries and functions in data structures.
Hello all.
I have some questions again. :-)
I wish to be able to place a function within a data structure. I would like to
use a dictionary because I could pass it a key and then the function could be
called. I couldn't find anything on the net to show me how to do this. More
then likely, not using the right search terms.
For example:
funct_call = { 'bhp' : 'bhp_counters (addict[key1][key2])', 'ospf' :
'ospf_counters (addict[key1][key2])'}
I am sure there is a way to do this.
The other issue is I cannot nest dictionaries. I have seen examples and when I
apply them to the code below. They do not provide the result I want. The
program does 3 actions.
1. Opens all the files in the directory. Each file begins with "data_". The 6
character in the file name is the occurrence of the output. Ranging from 1 to
9. The8th character plus the remaining part of the file is the output of the
command. For example:
data_1_ospf.txt
The commands stored in this file are related to OSPF. When I build the nested
dictionary I want to have "OSPF" as the primary key. Nested under "OSPF" is the
number of times the command has been captured. The file content stored as an
array and is the value of the 2nd key. data structure could look like this:
outputs = { 'ospf' : { '1' : lines_of_file[], '2' : lines of_file[]} }
Below is the code I have used to try and achieve the above. I just don't get
the nested dictionary effect I am after. Again, I am trying to use standard
core which some of the examples I have seen appeared to use. I am aware of
collection module.
#! /usr/bin/env python
# Identifying if memory leaks are occurring.
# goal is to import output to Excel.
# created on 28 Dec 2013 By Sean Murphy
import os, sys
from os.path import exists
# main code begins
if len(sys.argv) >= 2:
# storing file names into variable from command line.
filenames = sys.argv[1:]
else:
filenames = os.listdir(os.getcwd())
#print ("Error, must provide at least one file name\n")
#quit()
outputs = {} # empty dictionary (hash)
capture = "" # key used for the capture version
command = "" # key for the command output
for filename in filenames:
if exists(filename):
fp = open(filename, "r")
capture = filename[6]
command = filename[8:]
# nested dictionary. Command and then number of captures.
outputs = {command : { capture :[fp.readlines() } }
fp.close()
else:
print ("error %s doesn't exists\n" % filename)
quit()
print ("%r\n" % outputs.keys())
for key in sorted(outputs):
print (outputs[key].keys ())
Cheers
Sean
--
https://mail.python.org/mailman/listinfo/python-list
Re: nested dictionaries and functions in data structures.
- Original Message -
> Hello all.
>
> I have some questions again. :-)
>
> I wish to be able to place a function within a data structure. I
> would like to use a dictionary because I could pass it a key and
> then the function could be called. I couldn't find anything on the
> net to show me how to do this. More then likely, not using the right
> search terms.
>
> For example:
>
> funct_call = { 'bhp' : 'bhp_counters (addict[key1][key2])', 'ospf' :
> 'ospf_counters (addict[key1][key2])'}
>
> I am sure there is a way to do this.
>
> The other issue is I cannot nest dictionaries. I have seen examples
> and when I apply them to the code below. They do not provide the
> result I want. The program does 3 actions.
>
>
> 1. Opens all the files in the directory. Each file begins with
> "data_". The 6 character in the file name is the occurrence of the
> output. Ranging from 1 to 9. The8th character plus the remaining
> part of the file is the output of the command. For example:
>
> data_1_ospf.txt
>
> The commands stored in this file are related to OSPF. When I build
> the nested dictionary I want to have "OSPF" as the primary key.
> Nested under "OSPF" is the number of times the command has been
> captured. The file content stored as an array and is the value of
> the 2nd key. data structure could look like this:
>
> outputs = { 'ospf' : { '1' : lines_of_file[], '2' : lines of_file[]}
> }
>
> Below is the code I have used to try and achieve the above. I just
> don't get the nested dictionary effect I am after. Again, I am
> trying to use standard core which some of the examples I have seen
> appeared to use. I am aware of collection module.
>
> #! /usr/bin/env python
>
> # Identifying if memory leaks are occurring.
> # goal is to import output to Excel.
> # created on 28 Dec 2013 By Sean Murphy
>
> import os, sys
> from os.path import exists
>
> # main code begins
>
> if len(sys.argv) >= 2:
> # storing file names into variable from command line.
> filenames = sys.argv[1:]
> else:
> filenames = os.listdir(os.getcwd())
> #print ("Error, must provide at least one file name\n")
> #quit()
>
> outputs = {} # empty dictionary (hash)
> capture = "" # key used for the capture version
> command = "" # key for the command output
>
> for filename in filenames:
> if exists(filename):
> fp = open(filename, "r")
> capture = filename[6]
> command = filename[8:]
> # nested dictionary. Command and then number of captures.
> outputs = {command : { capture :[fp.readlines() } }
> fp.close()
> else:
> print ("error %s doesn't exists\n" % filename)
> quit()
>
> print ("%r\n" % outputs.keys())
> for key in sorted(outputs):
> print (outputs[key].keys ())
>
>
> Cheers
> Sean
outputs keeps track of the last loop only because you're assigning a new dict
on every loop. You need to update the current dict instead.
try to replace
outputs = {command : { capture :fp.readlines() } }
with (untested code)
if command not in outputs:
outputs[command] = {}
outputs[command][capture] = fp.readlines()
JM
-- IMPORTANT NOTICE:
The contents of this email and any attachments are confidential and may also be
privileged. If you are not the intended recipient, please notify the sender
immediately and do not disclose the contents to any other person, use it for
any purpose, or store or copy the information in any medium. Thank you.
--
https://mail.python.org/mailman/listinfo/python-list
Re: "More About Unicode in Python 2 and 3"
On 1/6/14 5:30 PM, Mark Lawrence wrote: On 06/01/2014 22:22, Ned Batchelder wrote: On 1/6/14 5:08 PM, Mark Lawrence wrote: On 06/01/2014 21:42, Ned Batchelder wrote: On 1/6/14 4:33 PM, Mark Lawrence wrote: That strikes me as being as useful as "The PEP 393 FSR is completely wrong but I'm not going to tell you why" approach. Please stop baiting people. What are you on about? The comment has been made that "its quite plain to me that 3.x python has a problem with everyday dealing with strings". Myself, Terry Reedy and Steven D'Aprano have all commented on this, asking for more data. We've been given nothing, which is precisely what our resident unicode expert has given us. I'm on about your comment being a gratuitous jab at someone who isn't even participating in the thread. Stop it. You arrogance really has no bounds. If you'd have done the job that you should have done in the first place and stopped that blithering idiot 16 months ago, we wouldn't still be putting up with him now. I don't understand how you can accuse me both of arrogance (presumably for speaking up about bad behavior), and also of not doing my job (presumably, speaking up about bad behavior) at the same time. 16 months ago I wasn't even a reader of this list, let alone a moderator. I don't know why you think it was my job to fix something back then. Now I have no special powers or jobs, just an interest in this list running smoothly. If you don't want to put up with someone, don't invite them into a thread by baiting them. And as I started this thread, I'll say what I please, throwing my toys out of my pram in just the same way that your pal Armin is currently doing. Starting threads doesn't give you any special rights. I don't see how your displeasure with Armin entitles you to gratuitous baiting of someone else. Declaring "I'm acting badly, and I intend to keep acting badly" is just childish. Forget the code of conduct, forget common civility, just think about your own self-interest: if you don't want to hear from someone on the list, why would you invite them in with that kind of provocation? Let sleeping dogs lie. -- Ned Batchelder, http://nedbatchelder.com -- https://mail.python.org/mailman/listinfo/python-list
Re: nested dictionaries and functions in data structures.
Thanks for that. It resolved the issue and it was so simple compared to
everything else I saw on the net.
Only outstanding thing I have to work out is how to execute functions from a
dictionary. I will continue searching on the net.
Sean
On 07/01/2014, at 9:21 PM, Jean-Michel Pichavant wrote:
>
>
> - Original Message -
>> Hello all.
>>
>> I have some questions again. :-)
>>
>> I wish to be able to place a function within a data structure. I
>> would like to use a dictionary because I could pass it a key and
>> then the function could be called. I couldn't find anything on the
>> net to show me how to do this. More then likely, not using the right
>> search terms.
>>
>> For example:
>>
>> funct_call = { 'bhp' : 'bhp_counters (addict[key1][key2])', 'ospf' :
>> 'ospf_counters (addict[key1][key2])'}
>>
>> I am sure there is a way to do this.
>>
>> The other issue is I cannot nest dictionaries. I have seen examples
>> and when I apply them to the code below. They do not provide the
>> result I want. The program does 3 actions.
>>
>>
>> 1. Opens all the files in the directory. Each file begins with
>> "data_". The 6 character in the file name is the occurrence of the
>> output. Ranging from 1 to 9. The8th character plus the remaining
>> part of the file is the output of the command. For example:
>>
>> data_1_ospf.txt
>>
>> The commands stored in this file are related to OSPF. When I build
>> the nested dictionary I want to have "OSPF" as the primary key.
>> Nested under "OSPF" is the number of times the command has been
>> captured. The file content stored as an array and is the value of
>> the 2nd key. data structure could look like this:
>>
>> outputs = { 'ospf' : { '1' : lines_of_file[], '2' : lines of_file[]}
>> }
>>
>> Below is the code I have used to try and achieve the above. I just
>> don't get the nested dictionary effect I am after. Again, I am
>> trying to use standard core which some of the examples I have seen
>> appeared to use. I am aware of collection module.
>>
>> #! /usr/bin/env python
>>
>> # Identifying if memory leaks are occurring.
>> # goal is to import output to Excel.
>> # created on 28 Dec 2013 By Sean Murphy
>>
>> import os, sys
>> from os.path import exists
>>
>> # main code begins
>>
>> if len(sys.argv) >= 2:
>># storing file names into variable from command line.
>>filenames = sys.argv[1:]
>> else:
>>filenames = os.listdir(os.getcwd())
>> #print ("Error, must provide at least one file name\n")
>> #quit()
>>
>> outputs = {} # empty dictionary (hash)
>> capture = "" # key used for the capture version
>> command = "" # key for the command output
>>
>> for filename in filenames:
>>if exists(filename):
>>fp = open(filename, "r")
>>capture = filename[6]
>>command = filename[8:]
>># nested dictionary. Command and then number of captures.
>>outputs = {command : { capture :[fp.readlines() } }
>>fp.close()
>>else:
>>print ("error %s doesn't exists\n" % filename)
>>quit()
>>
>> print ("%r\n" % outputs.keys())
>> for key in sorted(outputs):
>>print (outputs[key].keys ())
>>
>>
>> Cheers
>> Sean
>
> outputs keeps track of the last loop only because you're assigning a new dict
> on every loop. You need to update the current dict instead.
>
> try to replace
> outputs = {command : { capture :fp.readlines() } }
>
> with (untested code)
>
> if command not in outputs:
> outputs[command] = {}
> outputs[command][capture] = fp.readlines()
>
> JM
>
>
> -- IMPORTANT NOTICE:
>
> The contents of this email and any attachments are confidential and may also
> be privileged. If you are not the intended recipient, please notify the
> sender immediately and do not disclose the contents to any other person, use
> it for any purpose, or store or copy the information in any medium. Thank you.
--
https://mail.python.org/mailman/listinfo/python-list
Re: nested dictionaries and functions in data structures.
Sean Murphy wrote:
> Only outstanding thing I have to work out is how to execute functions from
> a dictionary. I will continue searching on the net.
I don't quite understand this question. Do you mean something like this?
def spam(n):
return "spam"*n
def eggs(n):
return "eggs"*n
d = {1: spam, 2: eggs}
print( d[1](3) )
print( d[2](3) )
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Bytes indexing returns an int
Does anyone know what the rationale behind making byte-string indexing return an int rather than a byte-string of length one? That is, given b = b'xyz', b[1] returns 121 rather than b'y'. This is especially surprising when one considers that it's easy to extract the ordinal value of a byte: ord(b'y') => 121 -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: "More About Unicode in Python 2 and 3"
On 1/6/14 11:01 PM, Steven D'Aprano wrote: On Mon, 06 Jan 2014 16:32:01 -0500, Ned Batchelder wrote: On 1/6/14 12:50 PM, Steven D'Aprano wrote: Ned Batchelder wrote: You are still talking about whether Armin is right, and whether he writes well, about flaws in his statistics, etc. I'm talking about the fact that an organization (Python core development) has a product (Python 3) that is getting bad press. Popular and vocal customers (Armin, Kenneth, and others) are unhappy. What is being done to make them happy? Who is working with them? They are not unique, and their viewpoints are not outliers. I'm not talking about the technical details of bytes and Unicode. I'm talking about making customers happy. Oh? How much did Armin pay for his Python support? If he didn't pay, he's not a customer. He's a user. I use the term "customer" in the larger sense of, "someone using your product that you are trying to please." I'd like to think that an open source project with only users would treat them as customers. Not in the sense of a legal obligation in exchange for money, but in the sense that the point of the work is to please them. But isn't the strength of open source that people write software that pleases *themselves*, and if others can make use of it, we all win? If GvR wrote Python to please others, it would have braces, it would be more like Perl and C, and it would probably be a mess. All else being equal, it's better for open source software if your users are happy than if they are unhappy, but at what cost? You can't make everyone happy. Of course, but there has to be a balance. One of the strengths of Python is that the core developers listen to people and will take a pragmatic approach that solves real problems. The change in 3.3 that re-added u"" literals is an interesting case: there was strong opposition to the proposal, because it weakened the purity of the new approach to Unicode. It won out because it made it easier to port code to 3.3, not because it made it easier to write a program that ran on 3.3, but because it made it easier to write a program that ran on both 2.7 and 3.3. [...] I was only avoiding talking about Unicode vs bytes because I'm not the one who needs a better way to do it, Armin and Kenneth are. You seem to be arguing from the standpoint of, "I've never had problems, so there are no problems." Certainly not. I've tried hard not to say, or imply, that Armin is wrong. I know he is an extremely competent Python developer, and I don't understand his problem domain well enough to categorically say he's wrong. I *suspect* he's doing something wrong, or at least sub-optimally, and making things more difficult for himself than they need be, but what do I know? Maybe that's just my wishful thinking. I was advocating for people learning more about his problem domain to understand where the friction was, and improve the situation. I suspect an undercurrent here is also the difference between writing Python 3 code, and writing code that can run on both Python 2 and 3. Of course. It's always harder to target multiple versions with incompatibilities than a single version. As I mentioned above, I think one of the under-exposed points of disagreement is how much Python 3 needs to make it possible to write polyglots, and how much it can focus on pure Python 3 programs. When the debate comes down to, "It's hard to write my programs!" countered with, "But Python 3 does this better than Python 2!", perhaps we need to uncover that the first person cares a lot about writing 2+3 code, and the second person does not. At least then we could understand why we seem to be talking past each other. In my original post, I provided two possible responses, one of which you've omitted: work with Armin to explain the easier way that he has missed. Isn't that just another way of saying what I said earlier? "try to educate Armin (and others) so they stop blaming Python for their own errors" Although your version is more diplomatic. It sounds like you think there isn't an easier way, and that's OK? I would love to see a Python 3 advocate work with Armin or Kenneth on the code that's caused them such pain, and find a way to make it good. Is it a good think that there's code that's hard to write in Python 3? Not in isolation. But sometimes when you design a language, you implicitly or explicitly decide that certain types of code will not be a good fit for that language: you can't write an efficient operating system kernel in Python. Maybe you can't easily do the sort of low-level network stuff that Armin is trying to do in Python 3. But I doubt it. I expect that probably half the problem is that he's missing something, or doing something wrong, and the other half is that Python 3 genuinely makes it harder than it should be. But again, what do I know? I appreciate the allowance for the possibility that Python 3 still has changes to make to impr
Re: nested dictionaries and functions in data structures.
"Sean Murphy" wrote in message
news:0cf6151e-e063-4252-9ac3-4fd4698eb...@gmail.com...
> Hello all.
>
> I have some questions again. :-)
>
> I wish to be able to place a function within a data structure. I would
> like to use a dictionary because I could pass it a key and then the
> function could be called. I couldn't find anything on the net to show me
> how to do this. More then likely, not using the right search terms.
>
> For example:
>
> funct_call = { 'bhp' : 'bhp_counters (addict[key1][key2])', 'ospf' :
> 'ospf_counters (addict[key1][key2])'}
>
> I am sure there is a way to do this.
>
[...]
There are two problems with your example.
First, it is (probably) correct to enclose the key in quotes, but definitely
incorrect to enclose the function name in quotes. Assuming you have already
declared the function, you just use the function name, without quotes, as
the value.
Secondly, you must not place brackets, with or without arguments, after the
function name. The difference is that, without brackets it *refers* to the
function, with brackets it *calls* the function. You only want to call it at
execution time, not when setting up the dictionary.
If you have a variable 'var' which contains 'bhp' or 'ospf', you would at
execution time have the following -
funct = funct_call[var] # retrieve the function from the dictionary
funct(addict[key1][key2]) # call the function with arguments
This is usually compressed into one line -
funct_call[var](addict[key1][key2])
This works if you always use the same arguments, no matter which function
you call. Things get trickier if they differ.
Frank Millman
--
https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
hi,
On Tue, Jan 07, 2014 at 10:13:29PM +1100, Steven D'Aprano wrote:
> Does anyone know what the rationale behind making byte-string indexing
> return an int rather than a byte-string of length one?
>
> That is, given b = b'xyz', b[1] returns 121 rather than b'y'.
>
> This is especially surprising when one considers that it's easy to extract
> the ordinal value of a byte:
>
> ord(b'y') => 121
Which Python version?
http://docs.python.org/2/reference/lexical_analysis.html#strings
"A prefix of 'b' or 'B' is ignored in Python 2;"
if you want to store the string literal as byte array, you have
to use "bytearray()" function:
>>> a = bytearray('xyz')
>>> a
bytearray(b'xyz')
>>> a[0]
120
>>> a[1]
121
http://docs.python.org/2/library/stdtypes.html
5.6. Sequence Types
hth,
a.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
Ervin Hegedüs wrote: > hi, > > On Tue, Jan 07, 2014 at 10:13:29PM +1100, Steven D'Aprano wrote: >> Does anyone know what the rationale behind making byte-string indexing >> return an int rather than a byte-string of length one? >> >> That is, given b = b'xyz', b[1] returns 121 rather than b'y'. >> >> This is especially surprising when one considers that it's easy to >> extract the ordinal value of a byte: >> >> ord(b'y') => 121 > > Which Python version? My apologies... I've been so taken up with various threads on this list discussing Python 3, I forgot to mention that I'm talking about Python 3. I understand the behaviour of bytes and bytearray, I'm asking *why* that specific behaviour was chosen. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: nested dictionaries and functions in data structures.
- Original Message -
> Thanks for that. It resolved the issue and it was so simple compared
> to everything else I saw on the net.
>
> Only outstanding thing I have to work out is how to execute functions
> from a dictionary. I will continue searching on the net.
>
>
> Sean
This may help you (untested code)
class Workers:
@staticmethod
def ospf(*args, **kwargs):
print args, kwargs
return 'Bar'
@staticmethod
def bhp(*args, **kwargs):
return 'Foo'
outputs = {'ospf' : {1 : {'param1' : 'foo', 'param2' : 'foo2'}}
for command, value in outputs.items():
for counter, kwargs in value:
func = getattr(Workers, command) # get the function
func(**kwargs) # the actual call
It would be possible to do it without the *args, **kwargs magic. You can write
ospf and bhp with the explicit parameters, however you must make sure you pass
the correct number of parameter in the call.
JM
-- IMPORTANT NOTICE:
The contents of this email and any attachments are confidential and may also be
privileged. If you are not the intended recipient, please notify the sender
immediately and do not disclose the contents to any other person, use it for
any purpose, or store or copy the information in any medium. Thank you.
--
https://mail.python.org/mailman/listinfo/python-list
Re: the Gravity of Python 2
On 01/07/2014 01:19 AM, Chris Angelico wrote: Can we get a run-down of everything that actually must be broken in 2.7 -> 3.3, that can't be backported via __future__, so we can start cherry-picking which bits to break in 2.8? The biggest one is going to be Unicode strings, for a large number of people (the print statement and division operators can be __future__'d, lots of other stuff got backported into 2.7). I'd be prepared to bet money that that one will NOT be broken in 2.8, meaning that it achieves nothing on that score. So how much easier will the migration actually be? That's a good question. I envision support for per-module upgrades, though I'm not 100% sure that it would work. This way a large project can incrementally start porting modules to the Python 3 unicode behavior. The 'future' module (python-future.org) effectively backports the Python 3 str and bytes into Python 2.7. The question is then what happens when they interact with Python 2 str and unicode. I'm speculating about the behavior of the 'future' module here, but here's what I could imagine. First the Python 3 behavior: py3str + py3str = py3str py3bytes + py3bytes = py3bytes py3str + py3bytes = error Then the behavior of when Python 3 str/bytes are mixed with Python 2 str and unicode: py3str + py2unicode = py2unicode py3str + py2str = py2unicode py3bytes + py2str = py2str Plus the regular old Python 2 behavior. I'm quite sure I could be getting something wrong; it's only a few days since I saw 'future' and started thinking along these lines. Regards, Martijn -- https://mail.python.org/mailman/listinfo/python-list
Re: Blog "about python 3"
Le dimanche 5 janvier 2014 23:14:07 UTC+1, Terry Reedy a écrit : > On 1/5/2014 9:23 AM, wxjmfa...@gmail.com wrote: > > > Le samedi 4 janvier 2014 23:46:49 UTC+1, Terry Reedy a écrit : > > >> On 1/4/2014 2:10 PM, wxjmfa...@gmail.com wrote: > > >>> And I could add, I *never* saw once one soul, who is > > >>> explaining what I'm doing wrong in the gazillion > > >>> of examples I gave on this list. > > > > >> If this is true, it is because you have ignored and not read my > > >> numerous, relatively polite posts. To repeat very briefly: > > >> 1. Cherry picking (presenting the most extreme case as representative). > > >> 2. Calling space saving a problem (repeatedly). > > >> 3. Ignoring bug fixes. > > ... > > > > > My examples are ONLY ILLUSTRATING, this FSR > > > is wrong by design, can be on the side of > > > memory, performance, linguistic or even > > > typography. > > > > Let me expand on 3 of my points. First, performance == time: > > > > Point 3. You correctly identified a time regression in finding a > > character in a string. I saw that the slowdown was *not* inherent in the > > FSR but had to be a glitch in the code, and reported it on pydev with > > the hope that someone would fix it even if it were not too important in > > real use cases. Someone did. > > > > Point 1. You incorrectly generalized that extreme case. I reported (a > > year ago last September) that the overall stringbench results were about > > the same. I also pointed out that there is an equally non-representative > > extreme case in the opposite direction, and that it would equally be > > wrong of me to use that to claim that FSR is faster. (It turns out that > > this FSR speed advantage *is* inherent in the design.) > > > > Memory: Point 2. A *design goal* of FSR was to save memory relative to > > UTF-32, which is what you apparently prefer. Your examples show that FSF > > successfully met its design goal. But you call that success, saving > > memory, 'wrong'. On what basis? > > > > You *claim* the FSR is 'wrong by design', but your examples only show > > that is was temporarily wrong in implementation as far as speed and > > correct by design as far as memory goes. > > Point 3: You are right. I'm very happy to agree. Point 2: This Flexible String Representation does no "effectuate" any memory optimization. It only succeeds to do the opposite of what a corrrect usage of utf* do. Ned : this has already been explained and illustrated. jmf -- https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
On 1/7/2014 6:13 AM, Steven D'Aprano wrote: Does anyone know what the rationale behind making byte-string indexing return an int rather than a byte-string of length one? That is, given b = b'xyz', b[1] returns 121 rather than b'y'. This former is the normal behavior of sequences, the latter is peculiar to strings, because there is no separate character class. A byte is a count n, 0 <= n < 256 and bytes and bytearrays are sequences of bytes. It was ultimately Guido's decision after some discussion and debate on, I believe, the py3k list. I do not remember enough to be any more specific. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Blog "about python 3"
On 1/7/2014 8:34 AM, wxjmfa...@gmail.com wrote: Le dimanche 5 janvier 2014 23:14:07 UTC+1, Terry Reedy a écrit : Memory: Point 2. A *design goal* of FSR was to save memory relative to UTF-32, which is what you apparently prefer. Your examples show that FSF successfully met its design goal. But you call that success, saving memory, 'wrong'. On what basis? Point 2: This Flexible String Representation does no "effectuate" any memory optimization. It only succeeds to do the opposite of what a corrrect usage of utf* do. Since the FSF *was* successful in saving memory, and indeed shrank the Python binary by about a megabyte, I have no idea what you mean. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
"treating bytes as chars" considered harmful? I don't know the answer to your question but the behavior seems right to me. Python 3 grudgingly allows the "abomination" of byte strings (is that what they're called? I haven't fully embraced Python3 yet). If you want a substring you use a slice. b = b'xyz' b[1:2] => b'y' also, chr(121) => 'y' which is really what the Python 3 gods prefer. On Tue, Jan 7, 2014 at 6:13 AM, Steven D'Aprano wrote: > Does anyone know what the rationale behind making byte-string indexing > return an int rather than a byte-string of length one? > > That is, given b = b'xyz', b[1] returns 121 rather than b'y'. > > This is especially surprising when one considers that it's easy to extract > the ordinal value of a byte: > > ord(b'y') => 121 > > > > -- > Steven > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
Sorry for top-posting. I thought I'd mastered gmail. -- https://mail.python.org/mailman/listinfo/python-list
Re: the Gravity of Python 2
Hi there, I just tried this out with the future module to see what it actually does, and I got this: On 01/07/2014 01:54 PM, Martijn Faassen wrote: First the Python 3 behavior: py3str + py3str = py3str Yup, of course. py3bytes + py3bytes = py3bytes Again of course. py3str + py3bytes = error Yup, that's an error. Then the behavior of when Python 3 str/bytes are mixed with Python 2 str and unicode: py3str + py2unicode = py2unicode This didn't work as I expected; I'd have expected py2unicode for compatibility reasons, as py3str cannot be combined with py2str (but in fact it *can*, odd). py3str + py2str = py2unicode This in fact returns py3str, which I didn't expect either. py3bytes + py2str = py2str And this gets me py3bytes. I'll get in touch with the author of the 'future' module to try to understand why his reasoning is different from mine, i.e. where I'm wrong. Regards, Martijn -- https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
David Robinow wrote: > "treating bytes as chars" considered harmful? Who is talking about treating bytes as chars? You're making assumptions that aren't justified by my question. > I don't know the answer to your question but the behavior seems right to > me. This issue was raised in an earlier discussion about *binary data* in Python 3. (The earlier discussion also involved some ASCII-encoded text, but that's actually irrelevant to the issue.) In Python 2.7, if you have a chunk of binary data, you can easily do this: data = b'\xE1\xE2\xE3\xE4' data[0] == b'\xE1' and it returns True just as expected. It even works if that binary data happens to look like ASCII text: data = b'\xE1a\xE2\xE3\xE4' data[1] == b'a' But in Python 3, the same code silently returns False in both cases, because indexing a bytes object gives an int. So you have to write something like these, all of which are ugly or inelegant: data = b'\xE1a\xE2\xE3\xE4' data[1] == 0x61 data[1] == ord(b'a') chr(data[1]) == 'a' data[1:2] == b'a' I believe that only the last one, the one with the slice, works in both Python 2.7 and Python 3.x. > Python 3 grudgingly allows the "abomination" of byte strings (is that > what they're called? I haven't fully embraced Python3 yet). They're not abominations. They exist for processing bytes (hence the name) and other binary data. They are necessary for low-level protocols, for dealing with email, web, files, and similar. Application code may not need to deal with bytes, but that is only because the libraries you call do the hard work for you. People trying to port these libraries from 2.7 to 3 run into this problem, and it causes them grief. This little difference between bytes in 2.7 and bytes in 3.x is a point of friction which makes porting harder, and I'm trying to understand the reason for it. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] Migrating from non-free programs to LibreOffice
On 01/05/2014 04:30 PM, Ben Finney wrote: > In short: Everything that was good about OpenOffice is now called > LibreOffice, which had to change its name only because the owners of > that name refused to let it go. Your information is a year or two out of date. OpenOffice.org is alive and well, under the auspices of the Apache foundation. And in fact development is proceeding on it quite well. I believe they released OpenOffice v4 some time ago. It is mostly feature identical to LibreOffice 4, and even has a couple of features that LibreOffice lacks. They really need to merge back into one project again, but I suspect they won't either for ideological or legal reasons. The former is very sad and not a good commentary on the viability of open source software. OpenOffice is a much better name than LibreOffice as well, especially for trying to convert users off of proprietary Office. -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] Migrating from non-free programs to LibreOffice
On 01/06/2014 08:53 AM, Grant Edwards wrote: > Yea, I think laying out a book with something like MS Word or > LibreOffice is nuts. Depending on her formatting needs, a > lighter-weight mark-up language (something like asciidoc) might suite: I've laid out a book with LibreOffice and it actually is quite capable. In fact when used correctly, LibreOffice can function much like a markup language. Instead of markup, you use styles (page, paragraph, character). The styles form the structure of the book (H1, H2, H3, etc). In fact the default styles mirror html to a degree. I tend to add my own for quotes, captions, etc. After composing the document, then you modify the styles to set the spacings, fonts, indentations, border lines, etc. The workflow is very similar to using LyX, or even a plain markup language for that matter. The weakest part of LibreOffice is embedding images. -- https://mail.python.org/mailman/listinfo/python-list
Re: the Gravity of Python 2
Hi, I've posted a documentation issue to the 'future' module which includes a further evolution of my thinking. As I expected, the author of the 'future' module has thought this through more than I had: https://github.com/PythonCharmers/python-future/issues/27 To get back to a hypothetical Python 2.8, it could implement this kind of behavior, and I think it would help support incremental upgrades. As long as you're using Py 3 bytes and str in your code, you'll be aware of errors and be forced to think about it. Other Python code in the system can remain unchanged, and to the magic of ducktyping will continue to work. You can then tackle things incrementally. Regards, Martijn -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] Migrating from non-free programs to LibreOffice
On Wed, Jan 8, 2014 at 3:45 AM, Michael Torrie wrote: > I tend to add my own [styles] > for quotes, captions, etc. After composing the document, > then you modify the styles to set the spacings, fonts, indentations, > border lines, etc. The workflow is very similar to using LyX, or even a > plain markup language for that matter. That's all very well when you put everything into a single file, but how do you manage those styles across a multi-file book? Mum's project was partially rescued by the discovery that you can import styles from another document, but that's still unworkable for repeated edits. > The weakest part of LibreOffice is embedding images. And that's why this particular book is being divided up: it's full of images. Putting the whole thing into a single file makes that file way way too big to work with (at least on the computer Mum's using - it's X times larger than her installed RAM, so Writer is constantly hammering the page file), and there's no convenient way to read in only part of the file. Hence my recommendation of a markup system like LaTeX that simply *references* images, and which deliberately isn't WYSIWYG; plus, having the concept of content, structure, and style all separate means it's not difficult to build just one file - maybe not even a whole chapter - while still being confident that all pages reference the same styles. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] Migrating from non-free programs to LibreOffice
On Wed, Jan 8, 2014 at 3:38 AM, Michael Torrie wrote:
> [OpenOffice v4] is mostly feature identical to
> LibreOffice 4, and even has a couple of features that LibreOffice lacks.
> They really need to merge back into one project again, but I suspect
> they won't either for ideological or legal reasons.
Sad. This is yet another of those politically-charged distinctions
that, quite frankly, I have no interest in. Just look at cdrtools vs
wodim, or Ruby 1.9.1 vs Ruby whatever-the-other-is, or for that matter
(switching genres a bit) Caucasian vs {Aboriginal, Native American,
dark skinned, etc, etc, etc}. I can't be bothered figuring out the
differences between them all, and it's disappointing that the
difference has to matter to someone.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: the Gravity of Python 2
On Wed, Jan 8, 2014 at 3:42 AM, Martijn Faassen wrote: > To get back to a hypothetical Python 2.8, it could implement this kind of > behavior, and I think it would help support incremental upgrades. As long as > you're using Py 3 bytes and str in your code, you'll be aware of errors and > be forced to think about it. Other Python code in the system can remain > unchanged, and to the magic of ducktyping will continue to work. You can > then tackle things incrementally. I'm still not sure how Python 2.8 needs to differ from 2.7. Maybe the touted upgrade path is simply a Python 2.7 installer plus a few handy libraries/modules that will now be preinstalled? These modules look great (I can't say, as I don't have a huge Py2 codebase to judge based on), and they presumably work on the existing Pythons. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: [way OT] Migrating from non-free programs to LibreOffice
On 01/07/2014 09:58 AM, Chris Angelico wrote: > On Wed, Jan 8, 2014 at 3:45 AM, Michael Torrie wrote: >> I tend to add my own [styles] >> for quotes, captions, etc. After composing the document, >> then you modify the styles to set the spacings, fonts, indentations, >> border lines, etc. The workflow is very similar to using LyX, or even a >> plain markup language for that matter. > > That's all very well when you put everything into a single file, but > how do you manage those styles across a multi-file book? Mum's project > was partially rescued by the discovery that you can import styles from > another document, but that's still unworkable for repeated edits. Sorry should have been clearer on this. I do use multiple documents with LO with a master document. The table of contents can even be generated across documents. I believe my TOC is in my master document, along with the front matter. As for styles, basically you create a master template style that you use as a basis for each of your files (master document as well as the subdocuments). I make all my changes to the master template style and then when I open the various documents LO will update the templates. They aren't linked templates per se; they are copied. But the mechanism works okay, if a bit clunky. > >> The weakest part of LibreOffice is embedding images. > > And that's why this particular book is being divided up: it's full of > images. Putting the whole thing into a single file makes that file way > way too big to work with (at least on the computer Mum's using - it's > X times larger than her installed RAM, so Writer is constantly > hammering the page file), and there's no convenient way to read in > only part of the file. Hence my recommendation of a markup system like > LaTeX that simply *references* images, and which deliberately isn't > WYSIWYG; plus, having the concept of content, structure, and style all > separate means it's not difficult to build just one file - maybe not > even a whole chapter - while still being confident that all pages > reference the same styles. LO does reference images if you would like. But I find embedding the whole works is just more self-contained. And with multiple file documents the chances of losing data or messing with pagination are contained to individual sections. -- https://mail.python.org/mailman/listinfo/python-list
Re: [way OT] Migrating from non-free programs to LibreOffice
On Wed, Jan 8, 2014 at 4:10 AM, Michael Torrie wrote: > LO does reference images if you would like. But I find embedding the > whole works is just more self-contained. And with multiple file > documents the chances of losing data or messing with pagination are > contained to individual sections. Referencing isn't sufficient to prevent the massive memory use, though. Whatever's in the document will need to be loaded into RAM in order to edit the file. It's a fundamental limitation of big files and WYSIWYG editors. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: [way OT] Migrating from non-free programs to LibreOffice
On 01/07/2014 10:14 AM, Chris Angelico wrote: > On Wed, Jan 8, 2014 at 4:10 AM, Michael Torrie wrote: >> LO does reference images if you would like. But I find embedding the >> whole works is just more self-contained. And with multiple file >> documents the chances of losing data or messing with pagination are >> contained to individual sections. > > Referencing isn't sufficient to prevent the massive memory use, > though. Whatever's in the document will need to be loaded into RAM in > order to edit the file. It's a fundamental limitation of big files and > WYSIWYG editors. I suppose. Working with multiple-part documents mitigates this issue to a large extent. Granted my sections are only about 30 pages or so, and only about 20 mb of imagery per section. -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] Migrating from non-free programs to LibreOffice
Apologies to the list for the noise! Should have replied off-list. -- https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
On 01/07/2014 07:19 AM, David Robinow wrote: Python 3 grudgingly allows the "abomination" of byte strings (is that what they're called?) No, that is *not* what they're called. If you find any place in the Python3 docs that does call them bytestrings please submit a bug report. On 01/07/2014 08:12 AM, Steven D'Aprano wrote: People trying to port these libraries from 2.7 to 3 run into this problem, and it causes them grief. This little difference between bytes in 2.7 and bytes in 3.x is a point of friction which makes porting harder, and I'm trying to understand the reason for it. If I recall correctly the way it was explained to me: bytes (lists, arrays, etc.) is a container, and when a container is indexed you get whatever the container held. If you slice the container you get a smaller container with the appropriate items. bytes (and bytearrays) are containers of ints, so indexing returns an int. One big problem with this whole scenario is that bytes then lies about what it contains. (And I hate lies! [1]) Anyway, I believe that's the rationale behind the change. -- ~Ethan~ [1] http://www.quickmeme.com/meme/3ts325 -- https://mail.python.org/mailman/listinfo/python-list
Re: [Python-ideas] RFC: bytestring as a str representation [was: a new bytestring type?]
On 2014-01-07 17:46, Andrew Barnert wrote:
> I think Stephen's name "7-bit" is confusing people. If you try to
> interpret the name sensibly, you get Steven's broken interpretation.
> But if you read it as a nonsense word and work through the logic, it
> all makes sense.
>
> On Jan 7, 2014, at 7:44, Steven D'Aprano wrote:
>
I was thinking about Ethan's suggestion of introducing a new bytestring
class and a lot of these suggestions are what I thought the bytestring
class could do.
[snip]
>>
>> Suppose we take a pure-ASCII byte-string and decode it:
>>
>>b'abcd'.decode('ascii-compatible')
>>
That would be:
bytestring(b'abcd')
or even:
bytestring('abcd')
[snip]
>
>> Suppose we take a byte-string with a non-ASCII byte:
>>
>>b'abc\xFF'.decode('ascii-compatible')
>>
That would be:
bytestring(b'abc\xFF')
Bytes outside the ASCII range would be mapped to Unicode low
surrogates:
bytestring(b'abc\xFF') == bytestring('abc\uDCFF')
[snip]
>> Presumably they will compare equal, yes?
>
> I would hope not. One of them has the Unicode character U+FF, the
> other has smuggled byte 0xFF, so they'd better not compare equal.
>
> However, the latter should compare equal to 'abc\uDCFF'. That's the
> entire key here: the new representation is nothing but a more compact
> way to represent strings that contain nothing but ASCII and surrogate
> escapes.
>
[snip]
>>
>> A concrete example:
>>
>>s = b'abcd'.decode('ascii-compatible')
>>t = 'x' # ASCII-compatible
>>s + t
>>=> returns 'abcdx', with the "7-bit repr" flag cleared.
s = bytestring(b'abcd')
t = 'x' # ASCII-compatible
s + t
=> returns 'abcdx'
>
> Right. Here both s and t are normal 8-bit strings reprs in the first
> place, so the new logic doesn't even get invoked. So yes, that's what
> it returns.
>
>>s = b'abcd'.decode('ascii-compatible')
>>t = 'ÿ' # U+00FF, non-ASCII.
>>
>>s + t
>>=> returns 'abcd\uDCFF', with the "7-bit repr" flag set
s = bytestring(b'abcd')
t = 'ÿ' # U+00FF, non-ASCII.
s + t
=> returns 'abcd\xFF'
[snip]
There were also some other equivalences I was considering:
bytestring(b'abc') == b'abc'
bytestring(b'abc') == 'abc'
The problem there is that it wouldn't be transitive because:
b'abc' != 'abc'
--
https://mail.python.org/mailman/listinfo/python-list
Re: [Python-ideas] RFC: bytestring as a str representation [was: a new bytestring type?]
On 01/07/2014 10:22 AM, MRAB wrote:
On 2014-01-07 17:46, Andrew Barnert wrote:
On Jan 7, 2014, at 7:44, Steven D'Aprano wrote:
I was thinking about Ethan's suggestion of introducing a new bytestring
class and a lot of these suggestions are what I thought the bytestring
class could do.
Suppose we take a pure-ASCII byte-string and decode it:
b'abcd'.decode('ascii-compatible')
That would be:
bytestring(b'abcd')
or even:
bytestring('abcd')
[snip]
Suppose we take a byte-string with a non-ASCII byte:
b'abc\xFF'.decode('ascii-compatible')
That would be:
bytestring(b'abc\xFF')
Bytes outside the ASCII range would be mapped to Unicode low
surrogates:
bytestring(b'abc\xFF') == bytestring('abc\uDCFF')
Not sure what you mean here. The resulting bytes should be 'abc\xFF' and of
length 4.
--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
07.01.14 18:12, Steven D'Aprano написав(ла): In Python 2.7, if you have a chunk of binary data, you can easily do this: data = b'\xE1\xE2\xE3\xE4' data[0] == b'\xE1' and it returns True just as expected. data[0] == b'\xE1'[0] works as expected in both Python 2.7 and 3.x. -- https://mail.python.org/mailman/listinfo/python-list
Difficulty with ez_setup.py
I am trying to run ez_setup.py on a fresh installation of Python 2.7.6 in a Win XP environment, but I keep getting an error. Here's the traceback: C:\Python27\Lib>python ez_setup.py Extracting in c:\docume~1\dick\locals~1\temp\tmpkjidy0 Now working in c:\docume~1\dick\locals~1\temp\tmpkjidy0\setuptools-2.0.2 Installing Setuptools Traceback (most recent call last): File "setup.py", line 17, in exec(init_file.read(), command_ns) File "", line 8, in File "c:\docume~1\dick\locals~1\temp\tmpkjidy0\setuptools-2.0.2\setuptools\__init__.py", line 11, in from setuptools.extension import Extension File "c:\docume~1\dick\locals~1\temp\tmpkjidy0\setuptools-2.0.2\setuptools\extension.py", line 5, in from setuptools.dist import _get_unpatched File "c:\docume~1\dick\locals~1\temp\tmpkjidy0\setuptools-2.0.2\setuptools\dist.py", line 16, in import pkg_resources File "c:\docume~1\dick\locals~1\temp\tmpkjidy0\setuptools-2.0.2\pkg_resources.py", line 31, in from pkgutil import get_importer ImportError: cannot import name get_importer Something went wrong during the installation. See the error message above. It's been awhile since I last installed ez_install, and I remember it being a pain, but I don't remember this issue. What am I doing wrong? On a possibly related note, is there a specific place that ez_setup.py is expected to be in when this is run? -- rzed -- https://mail.python.org/mailman/listinfo/python-list
Re: django question
Django is great On Tuesday, January 7, 2014 12:55:07 AM UTC-7, CM wrote: > On Monday, January 6, 2014 8:57:22 PM UTC-5, Roy Smith wrote: > > > > > Yes, exactly. There's nothing magic about a django view. It's just a > > > function which is passed an instance of HttpRequest (and possibly a few > > > other things, depending on your url mapping), and which is expected to > > > return an instance of HttpResponse. Within that framework, it can call > > > any other functions it wants. > > > > > > For example, http://legalipsum.com/ is a silly little site I built in > > > django. Here's the view for the home page: > > > > Nice! > > > > > Notice how the view knows nothing about generating the actual markov > > > text. That's in another module, which lives somewhere on my PYTHONPATH. > > > ALso, the view knows nothing about how the page is laid out; only the > > > templates know that. If I decided to redo this in tornado or flask, > > > whatever, I would need to rewrite my view, but there's not much to > > > rewrite. Most of the logic is in the Markov chainer, and that would > > > cary over to the new implementation unchanged. > > > > > > BTW, my suggestion to keep business logic and presentation code distinct > > > isn't unique to django, it's a good idea in pretty much all systems. > > > > Thanks for these points, helpful to see in practice. I'm trying to be > > more mindful of good coding practices, and this will be helpful as I continue > > to learn Django and making web applications generally. -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] Migrating from non-free programs to LibreOffice
Michael Torrie writes: > On 01/05/2014 04:30 PM, Ben Finney wrote: > > In short: Everything that was good about OpenOffice is now called > > LibreOffice, which had to change its name only because the owners of > > that name refused to let it go. > > Your information is a year or two out of date. […] I'm aware of the facts you listed, and they don't change what I stated above. > They really need to merge back into one project again, but I suspect > they won't either for ideological or legal reasons. The former is very > sad and not a good commentary on the viability of open source > software. On the contrary: It's a sad commentary on what happens to any software project under a centralised party that loses interest in it and just sits on it for years. The fact that we have the vibrant free-software office suites we now have is a testament to the strength of free software — the developers were able to rescue the project from the grip of a corporation that would never have freed it otherwise. Chris Angelico writes: > Sad. This is yet another of those politically-charged distinctions > that, quite frankly, I have no interest in. I raised the point because you're giving advice to others on which software to use. If you have no interest in the distinctions, perhaps you should cultivate such an interest to know what software you're referring to when advising others. -- \“With Lisp or Forth, a master programmer has unlimited power | `\ and expressiveness. With Python, even a regular guy can reach | _o__) for the stars.” —Raymond Hettinger | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Blog "about python 3"
On 8 January 2014 00:34, wrote:
>
> Point 2: This Flexible String Representation does no
> "effectuate" any memory optimization. It only succeeds
> to do the opposite of what a corrrect usage of utf*
> do.
>
UTF-8 is a variable-width encoding that uses less memory to encode code
points with lower numerical values, on a per-character basis e.g. if a code
point <= U+007F it will use a single byte to encode; if <= U+07FF two bytes
will be used; ... up to a maximum of 6 bytes for code points >= U+400.
FSR is a variable-width memory structure that uses the width of the code
point with the highest numerical value in the string e.g. if all code
points in the string are <= U+00FF a single byte will be used per
character; if all code points are <= U+ two bytes will be used per
character; and in all other cases 4 bytes will be used per character.
In terms of memory usage the difference is that UTF-8 varies its width
per-character, whereas the FSR varies its width per-string. For any
particular string, UTF-8 may well result in using less memory than the FSR,
but in other (quite common) cases the FSR will use less memory than UTF-8
e.g. if the string contains only contains code points <= U+00FF, but some
are between U+0080 and U+00FF (inclusive).
In most cases the FSR uses the same or less memory than earlier versions of
Python 3 and correctly handles all code points (just like UTF-8). In the
cases where the FSR uses more memory than previously, the previous
behaviour was incorrect.
No matter which representation is used, there will be a certain amount of
overhead (which is the majority of what most of your examples have shown).
Here are examples which demonstrate cases where UTF-8 uses less memory,
cases where the FSR uses less memory, and cases where they use the same
amount of memory (accounting for the minimum amount of overhead required
for each).
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64
bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>>
>>> fsr = u""
>>> utf8 = fsr.encode("utf-8")
>>> min_fsr_overhead = sys.getsizeof(fsr)
>>> min_utf8_overhead = sys.getsizeof(utf8)
>>> min_fsr_overhead
49
>>> min_utf8_overhead
33
>>>
>>> fsr = u"\u0001" * 1000
>>> utf8 = fsr.encode("utf-8")
>>> sys.getsizeof(fsr) - min_fsr_overhead
1000
>>> sys.getsizeof(utf8) - min_utf8_overhead
1000
>>>
>>> fsr = u"\u0081" * 1000
>>> utf8 = fsr.encode("utf-8")
>>> sys.getsizeof(fsr) - min_fsr_overhead
1024
>>> sys.getsizeof(utf8) - min_utf8_overhead
2000
>>>
>>> fsr = u"\u0001\u0081" * 1000
>>> utf8 = fsr.encode("utf-8")
>>> sys.getsizeof(fsr) - min_fsr_overhead
2024
>>> sys.getsizeof(utf8) - min_utf8_overhead
3000
>>>
>>> fsr = u"\u0101" * 1000
>>> utf8 = fsr.encode("utf-8")
>>> sys.getsizeof(fsr) - min_fsr_overhead
2025
>>> sys.getsizeof(utf8) - min_utf8_overhead
2000
>>>
>>> fsr = u"\u0101\u0081" * 1000
>>> utf8 = fsr.encode("utf-8")
>>> sys.getsizeof(fsr) - min_fsr_overhead
4025
>>> sys.getsizeof(utf8) - min_utf8_overhead
4000
Indexing a character in UTF-8 is O(N) - you have to traverse the the string
up to the character being indexed. Indexing a character in the FSR is O(1).
In all cases the FSR has better performance characteristics for indexing
and slicing than UTF-8.
There are tradeoffs with both UTF-8 and the FSR. The Python developers
decided the priorities for Unicode handling in Python were:
1. Correctness
a. all code points must be handled correctly;
b. it must not be possible to obtain part of a code point (e.g. the
first byte only of a multi-byte code point);
2. No change in the Big O characteristics of string operations e.g.
indexing must remain O(1);
3. Reduced memory use in most cases.
It is impossible for UTF-8 to meet both criteria 1b and 2 without
additional auxiliary data (which uses more memory and increases complexity
of the implementation). The FSR meets all 3 criteria.
Tim Delaney
--
https://mail.python.org/mailman/listinfo/python-list
RFD: pt.comp.lang.python
REQUEST FOR DISCUSSION (RFD) unmoderated group pt.comp.lang.python This is a formal Request for Discussion (RFD) for the creation of the unmoderated newsgroup pt.comp.lang.python. NEWSGROUPS LINE: pt.comp.lang.python Portuguese version of comp.lang.python newsgroup RATIONALE: Portuguese is the seventh most spoken language in the world with around 215 million native speakers in all continents, and the third most spoken in Europe. Python is taught in most universities from Angola to Brazil, not only in Computer Science (CS) courses but mostly in biological sciences (e.g., biopython is used to teach genome and proteome sequence analysis), earth sciences (python is widely used as a scripting language in open source GIS software), statistics (numpy, pandas), etc. The focus of most courses outside CS is neither on developing programming skills nor on python as a language, resulting in huge deficiencies in python programming, and many questions remain unanswered. This is more so in developing countries such as Mozambique, Guinea-Bissau, Cape verde, etc. where the english language is neither well spoken nor read. For those reasons, I propose the creation of the pt.comp.lang.python newsgroup, similar to it.comp.lang.python, es.comp.lang.python or de.comp.lang.python. CHARTER: The newsgroup pt.comp.lang.python is intended for general discussions and questions about Python with some flexibility for more general programming topics (e.g. software design) all using the Portuguese language. Posters to this newsgroup are asked to refrain from harrassment, personal attacks, and disruptive communication. Articles containing binary computer files including, but not limited to, programs or images, are forbidden. HTML posts are discouraged. Users should exercise their best judgement and use consideration for the copyright laws of their country. If in doubt, ask. PROCEDURE: For more information on the newsgroup creation process, please see: http://www.big-8.org/wiki/How_to_Create_a_New_Big-8_Newsgroup Those who wish to influence the development of this RFD and its final resolution should subscribe to news.groups.proposals and participate in the relevant threads in that newsgroup. This is both a courtesy to groups in which discussion of creating a new group is off-topic as well as the best method of making sure that one's comments or criticisms are heard. All discussion of active proposals should be posted to news.groups.proposals To this end, the followup header of this RFD has been set to news.groups.proposals. If desired by the readership of closely affected groups, the discussion may be crossposted to those groups, but care must be taken to ensure that all discussion appears in news.groups.proposals as well. We urge those who would like to read or post in the proposed newsgroup to make a comment to that effect in this thread; we ask proponents to keep a list of such positive posts with the relevant message ID. Such lists of positive feedback for the proposal may constitute good evidence that the group will be well-used if it is created. DISTRIBUTION: This document has been posted to the following newsgroups: news.announce.newgroups news.groups.proposals comp.lang.python PROPONENT: César Ribeiro -- https://mail.python.org/mailman/listinfo/python-list
Re: Blog "about python 3"
On 1/7/2014 9:54 AM, Terry Reedy wrote: On 1/7/2014 8:34 AM, wxjmfa...@gmail.com wrote: Le dimanche 5 janvier 2014 23:14:07 UTC+1, Terry Reedy a écrit : Memory: Point 2. A *design goal* of FSR was to save memory relative to UTF-32, which is what you apparently prefer. Your examples show that FSF successfully met its design goal. But you call that success, saving memory, 'wrong'. On what basis? Point 2: This Flexible String Representation does no "effectuate" any memory optimization. It only succeeds to do the opposite of what a corrrect usage of utf* do. Since the FSF *was* successful in saving memory, and indeed shrank the Python binary by about a megabyte, I have no idea what you mean. Tim Delaney apparently did, and answered on the basis of his understanding. Note that I said that the design goal was 'save memory RELATIVE TO UTF-32', not 'optimize memory'. UTF-8 was not considered an option. Nor was any form of arithmetic coding https://en.wikipedia.org/wiki/Arithmetic_coding to truly 'optimize memory'. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] Migrating from non-free programs to LibreOffice
On Wed, Jan 8, 2014 at 8:54 AM, Ben Finney wrote: > Chris Angelico writes: > >> Sad. This is yet another of those politically-charged distinctions >> that, quite frankly, I have no interest in. > > I raised the point because you're giving advice to others on which > software to use. If you have no interest in the distinctions, perhaps > you should cultivate such an interest to know what software you're > referring to when advising others. The mention of OpenOffice was extremely tangential. I was simply making a point about there being a "best option" that's hard to attain (but probably worthwhile long-term) and a "next-best option" that's less of a jump for the end user. If you take that original post and switch the name OpenOffice to Libre Office, it will not make a penn'orth of difference to the argument. If I were giving advice in that post, the advice being given was surely "use LaTeX". ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
Ethan Furman wrote: > On 01/07/2014 07:19 AM, David Robinow wrote: >> >> Python 3 grudgingly allows the "abomination" of byte strings (is that >> what they're called?) > > No, that is *not* what they're called. If you find any place in the > Python3 docs that does call them bytestrings please submit a bug report. The name of the class is "bytes", but what they represent *is* a string of bytes, hence "byte-string". It's a standard computer science term for distinguishing strings of text from strings of bytes. > On 01/07/2014 08:12 AM, Steven D'Aprano wrote: >> People trying to port these libraries from 2.7 to 3 run into this >> problem, and it causes them grief. This little difference between bytes >> in 2.7 and bytes in 3.x is a point of friction which makes porting >> harder, and I'm trying to understand the reason for it. > > If I recall correctly the way it was explained to me: > > bytes (lists, arrays, etc.) is a container, and when a container is > indexed you get whatever the container held. If you slice the container > you get a smaller container with the appropriate items. (There's also a bytearray type, which is best considered as an array. Hence the name.) Why decide that the bytes type is best considered as a list of bytes rather than a string of bytes? It doesn't have any list methods, it looks like a string and people use it as a string. As you have discovered, it is an inconvenient annoyance that indexing returns an int instead of a one-byte byte-string. I think that, in hindsight, this was a major screw-up in Python 3. > bytes (and bytearrays) are containers of ints, so indexing returns an int. > One big problem with this whole scenario is > that bytes then lies about what it contains. (And I hate lies! [1]) > > Anyway, I believe that's the rationale behind the change. > > -- > ~Ethan~ > > [1] http://www.quickmeme.com/meme/3ts325 -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
On Wed, Jan 8, 2014 at 11:15 AM, Steven D'Aprano wrote: > Why decide that the bytes type is best considered as a list of > bytes rather than a string of bytes? It doesn't have any list methods, it > looks like a string and people use it as a string. As you have discovered, > it is an inconvenient annoyance that indexing returns an int instead of a > one-byte byte-string. > > I think that, in hindsight, this was a major screw-up in Python 3. Which part was? The fact that it can be represented with a (prefixed) quoted string? bytes_value = (41, 42, 43, 44) string = bytes_value.decode() # "ABCD" I think it's more convenient to let people use a notation similar to what was used in Py2, but perhaps this is an attractive nuisance, if it gives rise to issues like this. If a bytes were more like a tuple of ints (not a list - immutability is closer) than it is like a string, would that be clearer? Perhaps the solution isn't even a code one, but a mental one. "A bytes is like a tuple of ints" might be a useful mantra. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
On 01/07/2014 04:15 PM, Steven D'Aprano wrote: Ethan Furman wrote: On 01/07/2014 07:19 AM, David Robinow wrote: Python 3 grudgingly allows the "abomination" of byte strings (is that what they're called?) No, that is *not* what they're called. If you find any place in the Python3 docs that does call them bytestrings please submit a bug report. The name of the class is "bytes", but what they represent *is* a string of bytes, hence "byte-string". It's a standard computer science term for distinguishing strings of text from strings of bytes. I do not disagree with your statements, yet calling the bytes type a bytestring suggests things which are not true, such as indexing returning another bytestring. The core-dev have thus agreed to not call it that in the documentation in hopes of lessening any confusion. On 01/07/2014 08:12 AM, Steven D'Aprano wrote: People trying to port these libraries from 2.7 to 3 run into this problem, and it causes them grief. This little difference between bytes in 2.7 and bytes in 3.x is a point of friction which makes porting harder, and I'm trying to understand the reason for it. If I recall correctly the way it was explained to me: bytes (lists, arrays, etc.) is a container, and when a container is indexed you get whatever the container held. If you slice the container you get a smaller container with the appropriate items. (There's also a bytearray type, which is best considered as an array. Hence the name.) Why decide that the bytes type is best considered as a list of bytes rather than a string of bytes? It doesn't have any list methods, it looks like a string and people use it as a string. As you have discovered, it is an inconvenient annoyance that indexing returns an int instead of a one-byte byte-string. I think that, in hindsight, this was a major screw-up in Python 3. The general consensus seems to be agreement (more or less) with that feeling, but alas it is too late to change it now. -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
On 2014-01-08, Chris Angelico wrote: > On Wed, Jan 8, 2014 at 11:15 AM, Steven D'Aprano > wrote: >> Why decide that the bytes type is best considered as a list of >> bytes rather than a string of bytes? It doesn't have any list methods, it >> looks like a string and people use it as a string. As you have discovered, >> it is an inconvenient annoyance that indexing returns an int instead of a >> one-byte byte-string. >> >> I think that, in hindsight, this was a major screw-up in Python 3. > > Which part was? The fact that b'ASDF'[0] in Python2 yeilds something different than it does in Python3 -- one yields b'A' and the other yields 0x41. It makes portable code a lot harder to write. I don't really have any preference for one over the other, but changing it for no apparent reason was a horrible idea. -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Programing Challenge: Constructing a Tree Given Its Edges.
Programing Challenge: Constructing a Tree Given Its Edges.
Show you are the boss.
http://xahlee.info/perl-python/python_construct_tree_from_edge.html
here's plain text.
── ── ── ── ──
Problem: given a list of edges of a tree: [child, parent], construct the tree.
Here's a sample input: [[0, 2], [3, 0], [1, 4], [2, 4]].
Here's a sample print of a tree data structure:
4
1
2
0
3
this means, the root node's name is 4. It has 2 branches, named 1 and 2. The
branche named 2 has 1 children, named 0, and it has one children named 3. The
node's names are given as integers, but you can assume them to be strings.
You can choose your own data structure for the output. For example, nested list
with 1st element being the node name, or use nested hash of hash, where key is
the node name, and value is its children.
Here's sample data structure in python, using hash of hash.
{
"4": {
"1": {},
"2": {
"0": {
"3": {}
}
}
}
}
Other data structure are accepted.
Code it using your favorite language. I'll be able to look at and verify in
Mathematica, Python, Ruby, Perl, PHP, JavaScript, Emacs Lisp, Java. But other
langs, such as Clojure and other lisps, OCaml, Haskell, erlang, Fortress, APL,
HOL, Coq, Lua, Factor, Rust, Julia … are very welcome. 〔☛ Proliferation of
Computing Languages〕
You should be able to do this within 8 hours from scratch. Took me 5 hours.
Python solution will be posted in a week, on 2014-01-14 (or sooner if many
showed solutions). Post your solution in comment (or link to github or
pastebin).
--
https://mail.python.org/mailman/listinfo/python-list
Re: Bytes indexing returns an int
On Wed, Jan 8, 2014 at 1:34 PM, Grant Edwards wrote: > On 2014-01-08, Chris Angelico wrote: >>> I think that, in hindsight, this was a major screw-up in Python 3. >> >> Which part was? > > The fact that b'ASDF'[0] in Python2 yeilds something different than it > does in Python3 -- one yields b'A' and the other yields 0x41. Fair enough. Either can be justified, changing is awkward. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
ANN: PyQt v5.2 Released
PyQt5 v5.2 has been released and is available from http://www.riverbankcomputing.com/software/pyqt/download5. PyQt5 is a comprehensive set of bindings for v5 of Digia's Qt cross-platform application framework. It supports Python v3, v2.7 and v2.6. The highlights of this release include full support for Qt v5.2, and the QtBluetooth, QtPositioning, QtMacExtras, QtWinExtras and QtX11Extras modules. Windows installers are provided which contain everything needed for PyQt5 development (including Qt, Qt Designer, QScintilla, and MySQL, PostgreSQL, SQLite and ODBC drivers) except Python itself. Installers are provided for the 32 and 64 bit versions of Python v3.3. PyQt5 is implemented as a set of 27 extension modules including support for: - non-GUI infrastructure including event loops, threads, i18n, user and application settings, mapped files and shared memory - GUI infrastructure including window system integration, event handling, 2D graphics, basic imaging, fonts, OpenGL - a comprehensive set of desktop widgets - WebKit - full integration with Quick2 and QML allowing new Quick items to be implemented in Python and created in QML - event driven network programming - multimedia including cameras, audio and radios - Bluetooth - global positioning using satellite, Wi-Fi or text file sources - sensors including accelerometers, altimeters, compasses, gyroscopes, magnetometers, and light, pressure, proximity, rotation and temperature sensors - serial ports - SQL - printing - DBus - XPath, XQuery, XSLT and XML Schema validation - a help system for creating and viewing searchable documentation - unit testing of GUI applications. -- https://mail.python.org/mailman/listinfo/python-list
Re: python finance
On 08/01/2014 00:32, Dennis Lee Bieber wrote: On Tue, 7 Jan 2014 11:04:11 +1100, Chris Angelico declaimed the following: "Python finance" could also be interpreted in many other ways, including "I want to write a finance application in Python", and "How does the PSF get its money?". Or a misplaced need for funding to support a pet Burmese Python Or European? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
