Re: HTML Code - Line Number
On Friday, 27 April 2012 18:09:57 UTC+1, smac...@comcast.net wrote:
> Hello,
>
> For scrapping purposes, I am having a bit of trouble writing a block
> of code to define, and find, the relative position (line number) of a
> string of HTML code. I can pull out one string that I want, and then
> there is always a line of code, directly beneath the one I can pull
> out, that begins with the following:
>
>
> However, because this string of HTML code above is not unique to just
> the information I need (which I cannot currently pull out), I was
> hoping there is a way to effectively say "if you find the html string
> _ in the line of HTML code above, and the string valign="top" class="body_co comcast.net> writes:
>
> Hello,
>
> I am having some difficulty generating the output I want from web
> scraping. Specifically, the script I wrote, while it runs without any
> errors, is not writing to the output file correctly. It runs, and
> creates the output .txt file; however, the file is blank (ideally it
> should be populated with a list of names).
>
> I took the base of a program that I had before for a different data
> gathering task, which worked beautifully, and edited it for my
> purposes here. Any insight as to what I might be doing wrote would be
> highly appreciated. Code is included below. Thanks!
[quoting reply to first thread]
I would approach it like this...
import lxml.html
QUERY = '//tr[@bgcolor="#F1F3F4"][td[starts-with(@class, "body_cols")]]'
url = 'http://www.skadden.com/Index.cfm?contentID=44&alphaSearch=A'
tree = lxml.html.parse(url).getroot()
trs = tree.xpath(QUERY)
for tr in trs:
tds = [el.text_content() for el in tr.iterfind('td')]
print tds
hth
Jon.
[/quote]
> following, then pull everything that follows this second string.
>
> Any thoughts as to how to define a function to do this, or do this
> some other way? All insight is much appreciated! Thanks.
comcast.net> writes:
>
> Hello,
>
[snip]
> Any thoughts as to how to define a function to do this, or do this
> some other way? All insight is much appreciated! Thanks.
>
[quote in reply to second thread]
Did you not see my reply to your previous thread?
And why do you want the line number?
[/quote]
I'm trying this on GG, as the mailing list gateway one or t'other does nee seem
to work (mea culpa no doubt).
So may have obscured the issue more with my quoting and snipping, or what not.
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
int object
Notebook Actions Cell Actions Format Output Insert Move Run Autoindent: Kernel Actions Kill kernel upon exit: Help Python IPython Links NumPy SciPy MPL SymPy run selected cell Shift-Enter : run selected cell in-place Ctrl-Enter : show keyboard shortcuts Ctrl-m h : Configuration Tooltip on tab: Smart completer: milliseconds Time before tooltip : In [4]: impor from __future__ import division from numpy import* import numpy as np import matplotlib.pyplot as plt from scipy.special import jv from scipy.special import yn h_cross=1 m=1 E=np.linspace(0.1,10,100) V0=-100 R=2 K=(2*E)**0.5 K_P=(2*(E-V0))**0.5 '''r=np.linspace(-10,10,1000) def V(r): if r>0 and r in () 49 delta_l=np.vectorize(delta_l) 50 ---> 51 print delta_l(K,0) 52 /home/deba/epd-7.2-1-rh5-x86_64/lib/python2.7/site-packages/numpy/lib/function_base.pyc in __call__(self, *args) 1860 for arg in args: 1861 newargs.append(asarray(arg).flat[0]) -> 1862 theout = self.thefunc(*newargs) 1863 if isinstance(theout, tuple): 1864 self.nout = len(theout) /home/deba/ in delta_l(K, l) 43 sph_jv_P=(l*sph_jv(l,K*R)/(K*R))-sph_jv(l,K*R) 44 sph_yn_P=(l*sph_yn(l,K*R)/(K*R))-sph_yn(l,K*R) ---> 45 Beta_l=l-(K_P*R(sph_jv(l+1,K_P*R))/(sph_jv(l,K_P*R))) 46 return arctan((K*R*sph_jv_P-Beta_l*sph_jv(l,K*R))/(K*R*sph_yn_P-Beta_l*sph_yn(l,K*R))) 47 TypeError: 'int' object is not callable -- http://mail.python.org/mailman/listinfo/python-list
Re: HTML Code - Line Number
smac2...@comcast.net wrote: > >For scrapping purposes, I am having a bit of trouble writing a block >of code to define, and find, the relative position (line number) of a >string of HTML code. I can pull out one string that I want, and then >there is always a line of code, directly beneath the one I can pull >out, that begins with the following: > > >However, because this string of HTML code above is not unique to just >the information I need (which I cannot currently pull out), I was >hoping there is a way to effectively say "if you find the html string >_ in the line of HTML code above, and the string valign="top" class="body_cols_middle"> in the line immediately >following, then pull everything that follows this second string. Regular expression-based screen scraping is extremely delicate. All it takes is one tweak to the HTML, and your scraping fails although the page continues to look the same. A much better plan is to use sgmllib to write yourself a mini HTML parser. You can handle "td" tags with the attributes you want, and count down until you get to the "td" tag you want. -- Tim Roberts, t...@probo.com Providenza & Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: CPython thread starvation
On 4/27/2012 9:55 PM, Paul Rubin wrote: John Nagle writes: I may do that to prevent the stall. But the real problem was all those DNS requests. Parallizing them wouldn't help much when it took hours to grind through them all. True dat. But building a DNS cache into the application seems like a kludge. Unless the number of requests is insane, running a caching nameserver on the local box seems cleaner. I know. When I have a bit more time, I'll figure out why CentOS 5 and Webmin didn't set up a caching DNS resolver by default. Sometimes the number of requests IS insane. When the system hits a page with a thousand links, it has to resolve all of them. (Beyond a thousand links, we classify it as link spam and stop. The record so far is a page with over 10,000 links.) John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: CPython thread starvation
John Nagle writes: >I may do that to prevent the stall. But the real problem was all > those DNS requests. Parallizing them wouldn't help much when it took > hours to grind through them all. True dat. But building a DNS cache into the application seems like a kludge. Unless the number of requests is insane, running a caching nameserver on the local box seems cleaner. -- http://mail.python.org/mailman/listinfo/python-list
Re: CPython thread starvation
On 4/27/2012 9:20 PM, Paul Rubin wrote: John Nagle writes: The code that stored them looked them up with "getaddrinfo()", and did this while a lock was set. Don't do that!! Added a local cache in the program to prevent this. Performance much improved. Better to release the lock while the getaddrinfo is running, if you can. I may do that to prevent the stall. But the real problem was all those DNS requests. Parallizing them wouldn't help much when it took hours to grind through them all. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: CPython thread starvation
John Nagle writes: > The code that stored them looked them up with "getaddrinfo()", and > did this while a lock was set. Don't do that!! >Added a local cache in the program to prevent this. > Performance much improved. Better to release the lock while the getaddrinfo is running, if you can. -- http://mail.python.org/mailman/listinfo/python-list
Re: CPython thread starvation
On Sat, Apr 28, 2012 at 1:35 PM, John Nagle wrote: > On CentOS, "getaddrinfo()" at the > glibc level doesn't always cache locally (ref > https://bugzilla.redhat.com/show_bug.cgi?id=576801). Python > doesn't cache either. How do you manage your local cache? The Python getaddrinfo function doesn't return a positive TTL (much less a negative one). Do you pick an arbitrary TTL, or cache indefinitely? I had the same issue in a PHP server (yeah I know, but I was maintaining a project that someone else started) - fortunately there is a PHP function that gives a TTL on all successful lookups, though it still won't for failures. I couldn't find anything on cache control anywhere in the Python socket module docs. Perhaps the simplest option is to throw down a local BIND to manage the caching for you, but that does seem a little like overkill. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: CPython thread starvation
On 4/27/2012 6:25 PM, Adam Skutt wrote: On Apr 27, 2:54 pm, John Nagle wrote: I have a multi-threaded CPython program, which has up to four threads. One thread is simply a wait loop monitoring the other three and waiting for them to finish, so it can give them more work to do. When the work threads, which read web pages and then parse them, are compute-bound, I've had the monitoring thread starved of CPU time for as long as 120 seconds. How exactly are you determining that this is the case? Found the problem. The threads, after doing their compute intensive work of examining pages, stored some URLs they'd found. The code that stored them looked them up with "getaddrinfo()", and did this while a lock was set. On CentOS, "getaddrinfo()" at the glibc level doesn't always cache locally (ref https://bugzilla.redhat.com/show_bug.cgi?id=576801). Python doesn't cache either. So huge numbers of DNS requests were being made. For some pages being scanned, many of the domains required accessing a rather slow DNS server. The combination of thousands of instances of the same domain, a slow DNS server, and no caching slowed the crawler down severely. Added a local cache in the program to prevent this. Performance much improved. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: Borg identity
Adam Skutt writes: > On Apr 27, 12:56 pm, Steven D'Aprano +comp.lang.pyt...@pearwood.info> wrote: > > On Thu, 26 Apr 2012 04:42:36 -0700, Adam Skutt wrote: > > > Steven D'Aprano wrote: > > >> The Borg design pattern, for example, would be an excellent > > >> candidate for ID:identity being treated as many-to-one. > > > > > How would inheritance work if I did that? > > > > You don't inherit from Borg instances, and instances inherit from > > their class the same as any other instance. > > I think you misunderstood me. Define a Borg class where somehow > identity is the same for all instances. The resulting class would not follow the Borg pattern, so it's not a Borg class. Remember that Borg names a design pattern, so if you've got a class that doesn't follow that pattern then a complaint that it doesn't keep the promises of the Borg pattern is merely tautological. -- \“The deepest sin against the human mind is to believe things | `\ without evidence.” —Thomas Henry Huxley, _Evolution and | _o__)Ethics_, 1893 | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: CPython thread starvation
On Apr 27, 2:54 pm, John Nagle wrote: > I have a multi-threaded CPython program, which has up to four > threads. One thread is simply a wait loop monitoring the other > three and waiting for them to finish, so it can give them more > work to do. When the work threads, which read web pages and > then parse them, are compute-bound, I've had the monitoring thread > starved of CPU time for as long as 120 seconds. How exactly are you determining that this is the case? > I know that the CPython thread dispatcher sucks, but I didn't > realize it sucked that bad. Is there a preference for running > threads at the head of the list (like UNIX, circa 1979) or > something like that? Not in CPython, which is at the mercy of what the operating system does. Under the covers, CPython uses a semaphore on Windows, which do not have FIFO ordering as per http://msdn.microsoft.com/en-us/library/windows/desktop/ms685129(v=vs.85).aspx. As a result, I think your thread is succumbing to the same issues that impact signal delivery as described on 22-24 and 35-41 of http://www.dabeaz.com/python/GIL.pdf. I'm not sure there's any easy or reliable way to "fix" that from your code. I am not a WinAPI programmer though, and I'd suggest finding one to help you out. It doesn't appear possible to change the scheduling policy for semaphore programatically, and I don't know closely they pay any attention to thread priority. That's just a guess though, and finding out for sure would take some low-level debugging. However, it seems to be the most probable situation assuming your code is correct. > > (And yes, I know about "multiprocessing". These threads are already > in one of several service processes. I don't want to launch even more > copies of the Python interpreter. Why? There's little harm in launching more instances. Processes have some additional startup and memory overhead compared to threads, but I can't imagine it woudl be an issue. Given what you're trying to do, I'd expect to run out of other resources long before I ran out of memory because I created too many processes or threads. > The threads are usually I/O bound, > but when they hit unusually long web pages, they go compute-bound > during parsing.) If your concern is being CPU oversubscribed by using lots of processes, I suspect it's probably misplaced. A whole mess of CPU- bound tasks is pretty much the easiest case for a scheduler to handle. Adam -- http://mail.python.org/mailman/listinfo/python-list
Re: CPython thread starvation
On 27/04/2012 23:30, Dennis Lee Bieber wrote: Oh, continuation thought... If the workers are calling into C-language operations, unless those operations release the GIL, it doesn't matter what the OS or Python thread switch timings are. The OS may interrupt the thread (running C-language code), pass control to the Python interpreter which finds the GIL is locked, and just blocks -- control passes back to the interrupted thread. Any long-running C-language function should release the GIL while doing things with local data -- and reacquire the GIL when it needs to manipulate Python data structures or returning... The OP mentioned parsing webpages. If that involves the re module at some point, it doesn't release the GIL while it's looking for matches. -- http://mail.python.org/mailman/listinfo/python-list
ControlDesk, Python, ActiveX, TableEditor
I am trying to hook into the TableEditor on Controldesk using Python. Unfortunately there are no good examples of how to write the code anywhere I have looked. Does anyone know where to look? OR does anyone have code? Thanks Chuck -- http://mail.python.org/mailman/listinfo/python-list
Re: Borg identity [was Re: why () is () and [] is [] work in other way?]
On Apr 27, 2:40 pm, Steven D'Aprano wrote:
> On Fri, 27 Apr 2012 10:33:34 -0700, Adam Skutt wrote:
> >> Why should the caller care whether they are dealing with a singleton
> >> object or an unspecified number of Borg objects all sharing state? A
> >> clever interpreter could make many Borg instances appear to be a
> >> singleton. A really clever one could also make a singleton appear to be
> >> many Borg instances.
>
> > Trivial: to break cyclical references in a deep copy operation.
>
> I asked why the *caller* should care. If the caller has to break cyclical
> references manually, the garbage collector is not doing its job.
It's a necessary requirement to serialize any cyclical structure.
Garbage collection has nothing to do with it. If I have some
structure such that A --> B --> A, I need to be able to determine that
I've seen 'A' before in order to serialize the structure to disk, or I
will never write it out successfully.
There are plenty of situations where we legitimately care whether two
pointers are the same and don't give one whit about the state of
objects they point to. You cannot conflate the two tests, and that's
precisely what your 'give all borg instances the same identity' idea
does.
> I think that if you're talking about per-instance members of a Borg
> class, you're confused as to what Borg means.
I'm not. I'm talking about per-instance members of a subclass of a
Borg class. There's nothing about the Borg pattern that forbids such
behavior, which is one of the reasons it's such a terrible idea in
general. Borg implies promises that it cannot readily keep.
> Since all instances share state, you can't have *per-instance* data.
I most certainly can do so in a subclass. Shared state in a parent
doesn't mandate shared state in a child.
> Not at all. Obviously each Borg subclass will have it's own fake
> identity.
> When I said that Borg instances are indistinguishable except for
> identity, I thought that was obvious that I was talking about instances
> of a single type. Mea culpa.
>
> Clearly if x is an instance of Borg, and y is an instance of
> BorgSubclass, you can distinguish them by looking at the type. The point
> is that you shouldn't be able to distinguish instances of a single type.
No, that's not the least bit obvious nor apparent, and it still
violates LSP. It means every function that takes a Borg as an
argument must know about every subclass in order to distinguish
between them.
The serialization function above would need to do so. Imagine an
object x that holds a Borg object and a BorgSubclass object. If the
serialization function keeps a list of objects it has seen before and
uses that to determine whether to write the object out, it will fail
to write out one or the other if we implemented your harebrained 'All
Borg objects have the same identity' idea.
Your idea means that 'x.borg is x.subborg' must return True. It also
means either x.borg isn't going to be written out, or x.subborg isn't
going to be written out. The program is broken.
If you modify your idea to ignore subtypes, than this function breaks:
def write_many(value, channel1, channel2):
channel1.write(value)
if channel2 is not channel1:
channel2.write(value)
Calling write_many("foo", x.borg, x.subborg) now gives different
behavior than write_many("foo", x.borg, x.borg). That's probably not
what the programmer intended!
Like it or not, whether you have only one object with shared state or
infinite objects with the same shared state is not an implementation
detail. Just because you write code that doesn't care about that fact
does not make it an implementation detail. I can write code that
depends on that fact, and there's not a single thing you can do to
stop me.
This is why the Borg pattern is a bad idea in general, because it
encourages programmers to write code that is subtly wrong. If you
have a Borg class, you can't ignore the fact that you have multiple
objects even if you want to do so. You will eventually end up writing
incorrect code as a result. Yet, many people try to do precisely
that, your idea is attempting to do precisely that!
> Oh please, enough of the religion of LSP.
>
> Barbara Liskov first introduced this idea in 1987, twenty years after
> Simula 67 first appeared and thirty years after MIT researchers came up
> with the concept of object oriented programming. That's hardly
> fundamental to the concept of OOP.
> People have, and still do, violate LSP all the time.
People write code with security flaws all of the time too. This
doesn't even being to approach being an reasonable argument. It's
completely disingenuous. People come up with ideas and fail to
properly formalize them all of the time. People come up with useful,
revolutionary ideas and get parts of them wrong all of the time.
If you violate LSP, then you enable interface users to write buggy
code. Correct class hierarchies must follow it; correct interface
implementations must f
Re: CPython thread starvation
John Nagle writes: >I know that the CPython thread dispatcher sucks, but I didn't > realize it sucked that bad. Is there a preference for running > threads at the head of the list (like UNIX, circa 1979) or > something like that? I think it's left up to the OS thread scheduler, Windows in your case. See http://www.dabeaz.com/python/NewGIL.pdf starting around slide 18. One idea that comes to mind is putting a periodic interrupt and signal handler into your main thread, to make sure the GIL gets released every so often. -- http://mail.python.org/mailman/listinfo/python-list
Re: CPython thread starvation
On 4/27/2012 20:54, John Nagle wrote: I have a multi-threaded CPython program, which has up to four threads. One thread is simply a wait loop monitoring the other three and waiting for them to finish, so it can give them more work to do. When the work threads, which read web pages and then parse them, are compute-bound, I've had the monitoring thread starved of CPU time for as long as 120 seconds. It's sleeping for 0.5 seconds, then checking on the other threads and for new work do to, so the work thread isn't using much compute time. How exactly are these waiting and checking performed? Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: (3.2) Overload print() using the C API?
On 27/04/2012 6:55 PM, Stefan Behnel wrote:
Peter Faulks, 27.04.2012 10:36:
On 27/04/2012 5:15 PM, Stefan Behnel wrote:
Peter Faulks, 26.04.2012 19:57:
I want to extend an embedded interpreter so that calls to print() are
automagically sent to a C++ gui (windows exe) via a callback function in
the DLL.
Then I'll be able to do this:
test.py
import printoverload
printoverload.set_stdout()
printoverload.set_stderr()
print("this will be sent to a C function in printoverload.pyd")
---
Why would you want to divert only "print" instead of changing sys.stdout in
general? Not all output comes from print calls.
Because I don't want to have to poll the stdout buffer.
You don't have to. It's delivered right at your door and even rings the
bell when it arrives to hand over the parcel in person.
I want the script
itself to update a window in the host application (via the extension) every
time the script calls print().
Then replace sys.stdout (and maybe also sys.stderr) by another object that
does what you want whenever its write() method is called.
But I guess that won't work if the script raises an exception...
Yep, you better catch those yourself. The C-API function you use for
executing the Python code in the first place will tell you when there was
an error.
BTW, my mention of Cython wasn't just a joke. You might want to look at it
because it makes these things essentially trivial compared to plain C-API
code in C.
Stefan
OK, I _think_ I'm getting warmer... But I wonder, do I need to create
new sys.stdout / sys.stderr objects? Can I not simply assign a custom C
function to the write() methods of the existing sys.stdout / sys.stderr
objects?
--
http://mail.python.org/mailman/listinfo/python-list
Re: HTML Code - Line Number
comcast.net> writes: > > Hello, > [snip] > Any thoughts as to how to define a function to do this, or do this > some other way? All insight is much appreciated! Thanks. > Did you not see my reply to your previous thread? And why do you want the line number? Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: hex to bin 16 bit word
On 2012-04-27, Grant Edwards wrote:
> On 2012-04-27, Grant Edwards wrote:
>> On 2012-04-27, Paul Rubin wrote:
>>> python writes:
What to decode hex '0xC0A8' and return signed short int.
>>>
>>> Is this right?
>>>
>>> n = int('0xC0A8', 16)
>>> if n >= 0x:
>>>n -= 0x1
>>
>> Yes, as long as the input value doesn't exceed 0x1. This is
>> probably better:
>>
>> n = int('0xc0a8'16) & 0x
>
> Oops, missed the "signed" part of the requirement.
>
>n = int('0xc0a8',16) & 0x
>
>if (n & 0x8000): n |= -1 & ~0x
or this:
if (n & 0x8000): n = -((~n & 0x7fff) + 1)
--
Grant Edwards grant.b.edwardsYow! I'm definitely not
at in Omaha!
gmail.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: hex to bin 16 bit word
On 2012-04-27, Grant Edwards wrote:
> On 2012-04-27, Paul Rubin wrote:
>> python writes:
>>> What to decode hex '0xC0A8' and return signed short int.
>>
>> Is this right?
>>
>> n = int('0xC0A8', 16)
>> if n >= 0x:
>>n -= 0x1
>
> Yes, as long as the input value doesn't exceed 0x1. This is
> probably better:
>
> n = int('0xc0a8'16) & 0x
Oops, missed the "signed" part of the requirement.
n = int('0xc0a8',16) & 0x
if (n & 0x8000): n |= -1 & ~0x
--
Grant Edwards grant.b.edwardsYow! It's a lot of fun
at being alive ... I wonder if
gmail.commy bed is made?!?
--
http://mail.python.org/mailman/listinfo/python-list
Re: hex to bin 16 bit word
On 27/04/2012 20:32, Ian Kelly wrote:
On Fri, Apr 27, 2012 at 12:56 PM, Paul Rubin wrote:
python writes:
What to decode hex '0xC0A8' and return signed short int.
Is this right?
n = int('0xC0A8', 16)
if n>= 0x:
n -= 0x1
No.
n = int('0xC0A8', 16)
if n>= 0x8000:
n -= 0x1
Or:
n = int('0xC0A8', 16)
n -= (n & 0x8000) * 2
:-)
--
http://mail.python.org/mailman/listinfo/python-list
Re: hex to bin 16 bit word
On Fri, Apr 27, 2012 at 12:56 PM, Paul Rubin wrote:
> python writes:
>> What to decode hex '0xC0A8' and return signed short int.
>
> Is this right?
>
> n = int('0xC0A8', 16)
> if n >= 0x:
> n -= 0x1
No.
n = int('0xC0A8', 16)
if n >= 0x8000:
n -= 0x1
--
http://mail.python.org/mailman/listinfo/python-list
Re: hex to bin 16 bit word
Steven D'Aprano writes:
>> Is this right?
>>
>> n = int('0xC0A8', 16)
>> if n >= 0x:
>>n -= 0x1
> No.
Oops, I meant n >= 0x7fff. Checking the sign bit.
Grant Edwards writes:
> Yes, as long as the input value doesn't exceed 0x1. This is
> probably better:
>
> n = int('0xc0a8'16) & 0x
Yeah, I was relying on the specification that the input was 16 bits.
--
http://mail.python.org/mailman/listinfo/python-list
Re: hex to bin 16 bit word
On 2012-04-27, Paul Rubin wrote:
> python writes:
>> What to decode hex '0xC0A8' and return signed short int.
>
> Is this right?
>
> n = int('0xC0A8', 16)
> if n >= 0x:
>n -= 0x1
Yes, as long as the input value doesn't exceed 0x1. This is
probably better:
n = int('0xc0a8'16) & 0x
--
Grant Edwards grant.b.edwardsYow! What PROGRAM are they
at watching?
gmail.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: setting an array element with sequence problem problem
On 4/27/12 7:18 PM, Debashish Saha wrote: You will want to ask numpy questions on the numpy mailing list. http://www.scipy.org/Mailing_Lists When you do, I recommend stating a clear question in addition to the code and the traceback (although those are very much appreciated). But just to get you on your way, remove the np.vectorize() line (you don't need it) and double-check the order of arguments that you are passing to delta_l(K,l). from __future__ import division from numpy import* import numpy as np import matplotlib.pyplot as plt from scipy.special import jv from scipy.special import yn h_cross=1 m=1 E=np.linspace(0.1,10,100) V0=-100 R=2 K=(2*E)**0.5 K_P=(2*(E-V0))**0.5 '''r=np.linspace(-10,10,1000) def V(r): if r>0 and r 168 exec compile(scripttext, filename, 'exec') in glob, loc 169 else: 170 def execfile(fname, *where): C:\Users\Neeraj\Desktop\14.py in() 45 l=6 46 delta_l=np.vectorize(delta_l) ---> 47 plt.plot(K,delta_l(l,K)) 48 49 display(gcf()) C:\Python27\lib\site-packages\numpy\lib\function_base.pyc in __call__(self, *args) 1880 if self.nout == 1: 1881 _res = array(self.ufunc(*newargs),copy=False, -> 1882 subok=True,dtype=self.otypes[0]) 1883 else: 1884 _res = tuple([array(x,copy=False,subok=True,dtype=c) \ ValueError: setting an array element with a sequence. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: hex to bin 16 bit word
On Fri, 27 Apr 2012 11:56:45 -0700, Paul Rubin wrote:
> python writes:
>> What to decode hex '0xC0A8' and return signed short int.
>
> Is this right?
>
> n = int('0xC0A8', 16)
> if n >= 0x:
>n -= 0x1
No.
>>> struct.unpack('>h',b'\xC0\xA8')
(-16216,)
>>> n = int('0xC0A8', 16)
>>> if n >= 0x:
... n -= 0x1
...
>>> n
49320
Should be -16216.
Personally, I don't see why the OP doesn't just use struct.unpack to
unpack a struct.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: hex to bin 16 bit word
python writes:
> What to decode hex '0xC0A8' and return signed short int.
Is this right?
n = int('0xC0A8', 16)
if n >= 0x:
n -= 0x1
--
http://mail.python.org/mailman/listinfo/python-list
CPython thread starvation
I have a multi-threaded CPython program, which has up to four threads. One thread is simply a wait loop monitoring the other three and waiting for them to finish, so it can give them more work to do. When the work threads, which read web pages and then parse them, are compute-bound, I've had the monitoring thread starved of CPU time for as long as 120 seconds. It's sleeping for 0.5 seconds, then checking on the other threads and for new work do to, so the work thread isn't using much compute time. I know that the CPython thread dispatcher sucks, but I didn't realize it sucked that bad. Is there a preference for running threads at the head of the list (like UNIX, circa 1979) or something like that? (And yes, I know about "multiprocessing". These threads are already in one of several service processes. I don't want to launch even more copies of the Python interpreter. The threads are usually I/O bound, but when they hit unusually long web pages, they go compute-bound during parsing.) Setting "sys.setcheckinterval" from the default to 1 seems to have little effect. This is on Windows 7. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
RE: HTML Code - Line Number
> Hello, > > For scrapping purposes, I am having a bit of trouble writing a block > of code to define, and find, the relative position (line number) of a > string of HTML code. I can pull out one string that I want, and then > there is always a line of code, directly beneath the one I can pull > out, that begins with the following: > > > However, because this string of HTML code above is not unique to just > the information I need (which I cannot currently pull out), I was > hoping there is a way to effectively say "if you find the html string > _ in the line of HTML code above, and the string valign="top" class="body_cols_middle"> in the line immediately > following, then pull everything that follows this second string. > > Any thoughts as to how to define a function to do this, or do this > some other way? All insight is much appreciated! Thanks. You may have more long-term success in scraping by using an HTML parser like Beautiful Soup. Alternately, store the line and the previous line while looping and do something like the following. if found: results.append( line ) continue criteria1 = ' valign="top" class="body_cols_middle">' in line criteria2 = '' in previous_line if criteria1 and criteria2 : found = True < maybe add rest of line to results > Ramit Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology 712 Main Street | Houston, TX 77002 work phone: 713 - 216 - 5423 -- This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. -- http://mail.python.org/mailman/listinfo/python-list
Re: Borg identity [was Re: why () is () and [] is [] work in other way?]
On Fri, 27 Apr 2012 10:33:34 -0700, Adam Skutt wrote: >> Why should the caller care whether they are dealing with a singleton >> object or an unspecified number of Borg objects all sharing state? A >> clever interpreter could make many Borg instances appear to be a >> singleton. A really clever one could also make a singleton appear to be >> many Borg instances. > > Trivial: to break cyclical references in a deep copy operation. I asked why the *caller* should care. If the caller has to break cyclical references manually, the garbage collector is not doing its job. If you're going to propose underpowered or buggy environments as an objection, then I'll simply respond that I'm not talking about any specific (underpowered or buggy) implementation, I'm talking about what is logically possible. [...] >> > How would inheritance work if I did that? >> >> You don't inherit from Borg instances, and instances inherit from their >> class the same as any other instance. > > I think you misunderstood me. Define a Borg class where somehow > identity is the same for all instances. Inherit from that class and add > per-instance members. I think that if you're talking about per-instance members of a Borg class, you're confused as to what Borg means. Since all instances share state, you can't have *per-instance* data. > Now, identity can't be the same for all > instances. As a result, you've just violated the Liskov Substituion > Principal: code that relies on all Borg class instances having the same > identity will fail when passed an instance of the subclass. Not at all. Obviously each Borg subclass will have it's own fake identity. Code that requires instances of different types to be identical is fundamentally broken, since the mere fact that they are different types means they can't be identical. I'll accept the blame for your confusion as I glossed over something which I thought was obvious, but clearly wasn't. When I said that Borg instances are indistinguishable except for identity, I thought that was obvious that I was talking about instances of a single type. Mea culpa. Clearly if x is an instance of Borg, and y is an instance of BorgSubclass, you can distinguish them by looking at the type. The point is that you shouldn't be able to distinguish instances of a single type. > It's impossible to combine identities and not violate LSP, unless you > forbid subclasses. Your idea violates one of the most fundamental > tenants of object-oriented programming. This is because object identity > is one of the fundamental consequences of object-oriented programming. > You can't do away with it, and any attempt to do so really just suggests > that you don't understand OOP at all. Oh please, enough of the religion of LSP. Barbara Liskov first introduced this idea in 1987, twenty years after Simula 67 first appeared and thirty years after MIT researchers came up with the concept of object oriented programming. That's hardly fundamental to the concept of OOP. People have, and still do, violate LSP all the time. LSP may be best practice but it's hardly essential. OOP was useful before LSP and it will remain useful in the face of violations. Besides: - In real life, subtypes often violate LSP. An electric car is a type of car, but it has no petrol tank. Wolf spiders have eyes, except for the Kauaʻi cave wolf spider, which is is a subtype of wolf spider but is completely eyeless. - Subclasses in Eiffel are not necessarily subtypes and may not be substitutable for superclasses. If it's good enough for Eiffel, it's good enough for my hypothetical Borg subclasses. You can always declare that Bertrand Meyer doesn't "understand OOP at all" too. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
RE: why () is () and [] is [] work in other way?
> > I have spent some time searching for a bug in my code, it was due to > > different work of "is" with () and []: > () is () > > True > [] is [] > > False > > > > (Python 2.7.2+ (default, Oct 4 2011, 20:03:08) > > [GCC 4.6.1] ) > > > > Is this what it should be or maybe yielding unified result is better? > > D. > > Congratulations, you've managed to introduce the most useless thread on > this ng/ml that I've come across in the ten years that I've been using > (c)Python. > > I've never ever thought about using this weird thing called "id" in my > code, (which comes later in all the responses) so why do some people > think it's so important? > Somewhere out there (possibly in an alternate universe) Bill Clinton is reading this thread and nodding vigorously about an argument around the [key]word "is". Ramit Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology 712 Main Street | Houston, TX 77002 work phone: 713 - 216 - 5423 This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python id() does not return an address [was Re: why () is () and [] is [] work in other way?]
On Sat, Apr 28, 2012 at 3:51 AM, Adam Skutt wrote: > Yes, there is a way. You add a function deref() to the language. In > CPython, that simply treats the passed value as a memory address and > treats it as an object, perhaps with an optional check. In Jython, > it'd access a global table of numbers as keys with the corresponding > objects as values, and return them. The value of id() is absolutely > an address, even in Jython. The fact the values can move about is > irrelevant. Python already as dereferenceable addresses. Look. def address(obj,table=[]): for i,o in enumerate(table): if obj is o: return i table.append(obj) return len(table)-1 def deref(addr): return address.__defaults__[0][addr] You can take the address of an object (interning it, effectively), and later dereference it. Proves nothing. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Borg identity [was Re: why () is () and [] is [] work in other way?]
On Fri, Apr 27, 2012 at 11:38 AM, Chris Angelico wrote: > On Sat, Apr 28, 2012 at 3:33 AM, Adam Skutt wrote: >> I think you misunderstood me. Define a Borg class where somehow >> identity is the same for all instances. Inherit from that class and >> add per-instance members. Now, identity can't be the same for all >> instances. As a result, you've just violated the Liskov Substituion >> Principal: code that relies on all Borg class instances having the >> same identity will fail when passed an instance of the subclass. > > Why would you subclass a Borg? For the same reasons one might want to subclass a Singleton. One of the big advantages of Borg is that it makes this easier, and it also allows the option of either having instances of subclasses share the same state as instances of base classes (usually preferred, because it preserves LSP) or of allowing each individual subclass to have its own unique shared state. -- http://mail.python.org/mailman/listinfo/python-list
Re: Direct vs indirect [was Re: why () is () and [] is [] work in other way?]
On Apr 27, 1:06 pm, Steven D'Aprano wrote: > On Thu, 26 Apr 2012 04:42:36 -0700, Adam Skutt wrote: > > On Apr 26, 5:10 am, Steven D'Aprano > +comp.lang.pyt...@pearwood.info> wrote: > >> But I was actually referring to something more fundamental than that. > >> The statement "a is b" is a *direct* statement of identity. "John is my > >> father." "id(a) == id(b)" is *indirect*: "The only child of John's > >> grandfather is the parent of the mother-in-law of my sister-in-law" > >> sort of thing. (Excuse me if I got the relationships mixed up.) > > > Again, the fact that you somehow think this absurd family tree is > > relevant only shows you're fundamentally confused about what object > > oriented identity means. That's rather depressing, seeing as I've given > > you a link to the definition. > > Perhaps you failed to notice that this "absurd" family tree, as you put > it, consists of grandparent+parent+sibling+in-law. What sort of families > are you familiar with that this seems absurd to you? No, I noticed, but who talks like that? It's not remotely comparable to the sort of difference we're talking about. > > I think you have inadvertently demonstrated the point I am clumsily > trying to make. Even when two expressions are logically equivalent, the > form of the expressions make a big difference to the comprehensibility of > the text. And if we were talking about 30, 20, 5, maybe even 2 line function versus it's name, you might have a point. We're not talking about such things though, and it's pretty disingenuous to pretend otherwise. Yet, that's precisely what you did with your absurd family relationship. > Which would you rather read? > > for item in sequence[1:]: ... > > for item in sequence[sum(ord(c) for c in 'avocado') % 183:]: ... > > The two are logically equivalent, so logically you should have no > preference between the two, yes? No, they're not logically equivalent. The first won't even execute, as sequence is undefined. You need two lines in the first case. > A statement is "direct" in the sense I mean if it explicitly states the > thing you intend it to state. And in the case of the two ways to compare identity, both statements state exactly what I intend to state. They're synonyms. > > "a is b" is a direct test of whether a is b. (Duh.) > > "id(a) == id(b)" is an indirect test of whether a is b, since it requires > at least three indirect steps: the knowledge of what the id() function > does, the knowledge of what the == operator does, and the knowledge that > equal IDs imply identity. The problem is that using 'is' correctly requires understanding all of those three things. Adam -- http://mail.python.org/mailman/listinfo/python-list
setting an array element with sequence problem problem
from __future__ import division from numpy import* import numpy as np import matplotlib.pyplot as plt from scipy.special import jv from scipy.special import yn h_cross=1 m=1 E=np.linspace(0.1,10,100) V0=-100 R=2 K=(2*E)**0.5 K_P=(2*(E-V0))**0.5 '''r=np.linspace(-10,10,1000) def V(r): if r>0 and r 168 exec compile(scripttext, filename, 'exec') in glob, loc 169 else: 170 def execfile(fname, *where): C:\Users\Neeraj\Desktop\14.py in () 45 l=6 46 delta_l=np.vectorize(delta_l) ---> 47 plt.plot(K,delta_l(l,K)) 48 49 display(gcf()) C:\Python27\lib\site-packages\numpy\lib\function_base.pyc in __call__(self, *args) 1880 if self.nout == 1: 1881 _res = array(self.ufunc(*newargs),copy=False, -> 1882 subok=True,dtype=self.otypes[0]) 1883 else: 1884 _res = tuple([array(x,copy=False,subok=True,dtype=c) \ ValueError: setting an array element with a sequence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Borg identity [was Re: why () is () and [] is [] work in other way?]
On Fri, Apr 27, 2012 at 11:33 AM, Adam Skutt wrote: > On Apr 27, 12:56 pm, Steven D'Aprano +comp.lang.pyt...@pearwood.info> wrote: >> On Thu, 26 Apr 2012 04:42:36 -0700, Adam Skutt wrote: >> > You're going to have to explain the value of an "ID" that's not 1:1 with >> > an object's identity, for at least the object's lifecycle, for a >> > programmer. If you can't come up with a useful case, then you haven't >> > said anything of merit. >> >> I gave an example earlier, but you seem to have misunderstood it, so I'll >> give more detail. >> >> In the Borg design pattern, every Borg instance shares state and are >> indistinguishable, with only one exception: object identity. We can >> distinguish two Borg instances by using "is". >> >> Since the whole point of the pattern is for Borg instances to be >> indistinguishable, the existence of a way to distinguish Borg instances >> is a flaw and may be undesirable. At least, it's exposing an >> implementation detail which some people argue should not be exposed. >> > > Then people should stop with such idiocy like the Borg pattern. It's a > bad example from an even worse idea. I would argue exactly the opposite. One issue I have with the Singleton pattern is that the very fact that a class is a singleton (or a borg) is an implementation detail. Shared identity exposes that implementation detail, which I consider a flaw. Borg, OTOH, actually gets this right. >> Why should the caller care whether they are dealing with a singleton >> object or an unspecified number of Borg objects all sharing state? A >> clever interpreter could make many Borg instances appear to be a >> singleton. A really clever one could also make a singleton appear to be >> many Borg instances. > > Trivial: to break cyclical references in a deep copy operation. The cycle will break anyway, since there are only a finite number of instances. You just might end up with as many copy instances as there were original instances in the structure. But going back to Steven's point here, that fact should not be important to the caller. > I think you misunderstood me. Define a Borg class where somehow > identity is the same for all instances. Why? The whole point of Borg is to share state, not identity. If you really want shared identity, use a Singleton. > Inherit from that class and > add per-instance members. Now, identity can't be the same for all > instances. As a result, you've just violated the Liskov Substituion > Principal: code that relies on all Borg class instances having the > same identity will fail when passed an instance of the subclass. That has nothing to do with Borg. That has to do with having misguidedly defined identity to be the same for all instances. You wouldn't do that for a regular class. Why force it on a Borg class? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python id() does not return an address [was Re: why () is () and [] is [] work in other way?]
On Apr 27, 1:12 pm, Steven D'Aprano wrote: > On Thu, 26 Apr 2012 04:42:36 -0700, Adam Skutt wrote: > > On Apr 26, 5:10 am, Steven D'Aprano > +comp.lang.pyt...@pearwood.info> wrote: > >> Solution to *what problem*? > > > This confusion that many people have over what 'is' does, including > > yourself. > > I have no confusion over what "is" does. False. If you did, then you would not have suggested the difference in True/False result between "id([1,2]) == id([1, 2])" and "[1, 2] is [1, 2]" matters. You would understand that the result of an identity test with temporary objects is meaningless, since identity is only meaningful while the objects are alive. That's a fundamental mistake. > >> > An address is an identifier: a number that I can use to access a > >> > value[1]. > > >> Then by your own definition, Python's id() does not return an address, > >> since you cannot use it to access a value. > > > The fact Python lacks explicit dereferencing doesn't change the fact > > that id() returns an address. Replace 'can' with 'could' or 'could > > potentially' or the whole phrase with 'represents' if you wish. It's a > > rather pointless thing to quibble over. > > You can't treat id() as an address. Did you miss my post when I > demonstrated that Jython returns IDs generated on demand, starting from > 1? In general, there is *no way even in principle* to go from a Python ID > to the memory location (address) of the object with that ID, because in > general objects *may not even have a fixed address*. Objects in Jython > don't, because the Java virtual machine can move them in memory. Yes, there is a way. You add a function deref() to the language. In CPython, that simply treats the passed value as a memory address and treats it as an object, perhaps with an optional check. In Jython, it'd access a global table of numbers as keys with the corresponding objects as values, and return them. The value of id() is absolutely an address, even in Jython. The fact the values can move about is irrelevant. Again, if this wasn't possible, then you couldn't implement 'is'. Implementing 'is' requires a mechanism for comparing objects that doesn't involve ensuring the contents of the two operands in memory is the same. > > Would you call the result of casting a C pointer to an int an address? > > If so, you must call the result of id() an address as well-- you can't > > dereference either of them. If not, then you need to provide an > > alternate name for the result of casting a C pointer to an int. > > I don't need to do anything of the sort. Yes, you do, because you called such a thing an address when talking about CPython. Even if my definition is wrong (it's not), your definition is wrong too. > (And for the record, in C you can cast an integer into a pointer, > although the results are implementation-specific. There's no equivalent > in Python.) Yes, but the lack of that operation doesn't mean that id() doesn't return an address. Adam -- http://mail.python.org/mailman/listinfo/python-list
Variables and values [was Re: why () is () and [] is [] work in other way?]
On Fri, 27 Apr 2012 10:15:32 -0700, Adam Skutt wrote: > If I write a program in Python that treats variables as if they were > values, it will be incorrect. I'm sorry, it is unclear to me what distinction you are making between variables and values. Can you give simple examples of both incorrect and correct code demonstrating what you mean? (I know what distinction *I* would make, but I'm not sure if it is the same one you are making.) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Borg identity [was Re: why () is () and [] is [] work in other way?]
On Sat, Apr 28, 2012 at 3:33 AM, Adam Skutt wrote: > I think you misunderstood me. Define a Borg class where somehow > identity is the same for all instances. Inherit from that class and > add per-instance members. Now, identity can't be the same for all > instances. As a result, you've just violated the Liskov Substituion > Principal: code that relies on all Borg class instances having the > same identity will fail when passed an instance of the subclass. Why would you subclass a Borg? ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Borg identity [was Re: why () is () and [] is [] work in other way?]
On Apr 27, 12:56 pm, Steven D'Aprano wrote: > On Thu, 26 Apr 2012 04:42:36 -0700, Adam Skutt wrote: > > You're going to have to explain the value of an "ID" that's not 1:1 with > > an object's identity, for at least the object's lifecycle, for a > > programmer. If you can't come up with a useful case, then you haven't > > said anything of merit. > > I gave an example earlier, but you seem to have misunderstood it, so I'll > give more detail. > > In the Borg design pattern, every Borg instance shares state and are > indistinguishable, with only one exception: object identity. We can > distinguish two Borg instances by using "is". > > Since the whole point of the pattern is for Borg instances to be > indistinguishable, the existence of a way to distinguish Borg instances > is a flaw and may be undesirable. At least, it's exposing an > implementation detail which some people argue should not be exposed. > Then people should stop with such idiocy like the Borg pattern. It's a bad example from an even worse idea. > Why should the caller care whether they are dealing with a singleton > object or an unspecified number of Borg objects all sharing state? A > clever interpreter could make many Borg instances appear to be a > singleton. A really clever one could also make a singleton appear to be > many Borg instances. Trivial: to break cyclical references in a deep copy operation. > John's argument is that Python should raise an exception if you compare > "2 is 2", or for that matter "3579 is 3579", which is foolish. > >> identities. The Borg design pattern, for example, would be an excellent > >> candidate for ID:identity being treated as many-to-one. > > > How would inheritance work if I did that? > > You don't inherit from Borg instances, and instances inherit from their > class the same as any other instance. I think you misunderstood me. Define a Borg class where somehow identity is the same for all instances. Inherit from that class and add per-instance members. Now, identity can't be the same for all instances. As a result, you've just violated the Liskov Substituion Principal: code that relies on all Borg class instances having the same identity will fail when passed an instance of the subclass. It's impossible to combine identities and not violate LSP, unless you forbid subclasses. Your idea violates one of the most fundamental tenants of object-oriented programming. This is because object identity is one of the fundamental consequences of object-oriented programming. You can't do away with it, and any attempt to do so really just suggests that you don't understand OOP at all. Adam -- http://mail.python.org/mailman/listinfo/python-list
Re: (3.2) Overload print() using the C API?
On 4/27/2012 4:55 AM, Stefan Behnel wrote: I want the script itself to update a window in the host application (via the extension) every time the script calls print(). Then replace sys.stdout (and maybe also sys.stderr) by another object that does what you want whenever its write() method is called. This is essentially what Idle does to display python output to its window. >>> import sys; sys.stdout -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Python id() does not return an address [was Re: why () is () and [] is [] work in other way?]
Steven, your posts are leaking out of their respective thread(s). Is this intentional? ~Temia -- When on earth, do as the earthlings do. -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On Apr 27, 11:01 am, Kiuhnm wrote: > On 4/27/2012 1:57, Adam Skutt wrote: > > On Apr 26, 6:34 pm, Kiuhnm wrote: > >>> If you > >> understand that your 'a' is not really an object but a reference to it, > >> everything becomes clear and you see that '==' always do the same thing. > > > Yes, object identity is implemented almost? everywhere by comparing > > the value of two pointers (references)[1]. I've already said I'm not > > really sure how else one would go about implementing it. > > >> You might tell me that that's just an implementation detail, but when an > >> implementation detail is easier to understand and makes more sense than > >> the whole abstraction which is built upon it, something is seriously wrong. > > > I'm not sure what abstraction is being built here. I think you have > > me confused for someone else, possibly Steven. > > The abstraction is this: > - There are primitives and objects. > - Primitives are not objects. The converse is also true. > - Primitives can become objects (boxing). > - Two primitives x and y are equal iff x == y. > - Two objects x and y are equal iff x.equals(y). > - Two objects are the same object iff x == y. > - If x is a primitive, then y = x is a deep copy. > - If x is an object, then y = x is a shallow copy. > - ... > This is not an abstraction at all, but merely a poor explanation of how things work in Java. Your last statement is totally incorrect, as no copying of the object occurs whatsoever. The reference is merely reseated to refer to the new object. If you're going to chide me for ignoring the difference between the reference and the referent object, then you shouldn't ignore it either, especially in the one case where it actually matters! If we try to extend this to other languages, then it breaks down completely. > The truth: > - Primitives can be references. > - Two primitives are equal iff x == y. > - Operator '.' automatically derefences references. > You have the first statement backwards. References are a primitive construct, not the other way around. While true, it's still a bad way to think about what's going on. It breaks down once we add C++ / Pascal reference types to the mix, for example. It's better to think about variables (names) and just recognize that not all variables have the same semantics. It avoids details that are irrelevant to writing actual programs and remains consistent. > Equality or equivalence is a relation which is: > - reflexive > - symmetric > - transitive > Everything else... is something else. Call it semi-equality, > tricky-equality or whatever, but not equality, please. Sure, but then it's illegal to allow the usage of '==' with floating point numbers, which will never have these properties in any usable implementation[1]. So we're back to what started this tangent, and we end up needing 'equals()' methods on our classes to distinguish between the different forms of equality. That's precisely what you want to avoid. Or we can just accept that '==' doesn't always possess those properties, which is what essentially every programming language does, and call it (value) equality. As long as we don't cross incompatible meanings, it's hard to believe that this isn't the right thing to do. > > > If anything, you have that backwards. Look at Python: all variables > > in Python have pointer semantics, not value semantics. > > When everything is "white", the word "white" becomes redundant. > So the fact that everything in Python have reference semantics means > that we can't stop thinking about value and reference semantics. Nope. The behavior of variables is absolutely essential to writing correct programs. If I write a program in Python that treats variables as if they were values, it will be incorrect. > > > In imperative > > languages, pointers have greater utility over value types because not > > all types can obey the rules for value types. For example, I don't > > know how to give value semantics to something like a I/O object (e.g, > > file, C++ fstream, C FILE), since I don't know how to create > > independent copies. > > By defining a copy constructor. Then write me a working one. I'll wait. To save yourself some time, you can start with std::fstream. > Python is already without pointers (*). > A world where everyone is a lawyer is a world without lawyers (really, > there isn't any other way we can get rid of them :) ). > > (*) By the way, some would argue that references are not pointers. They would be completely and utterly wrong, and probably imbuing pointers with properties they don't actually possess. Unless you're talking about C++ / Pascal references, which really aren't pointers and do possess a different set of semantics (alias might be a better term for them). Adam [1] Not in any fashion that's useful to the programmer, at any rate. -- http://mail.python.org/mailman/listinfo/python-list
Python id() does not return an address [was Re: why () is () and [] is [] work in other way?]
On Thu, 26 Apr 2012 04:42:36 -0700, Adam Skutt wrote: > On Apr 26, 5:10 am, Steven D'Aprano +comp.lang.pyt...@pearwood.info> wrote: >> Solution to *what problem*? >> > This confusion that many people have over what 'is' does, including > yourself. I have no confusion over what "is" does. The "is" operator returns True if and only if the two operands are the same object, otherwise it returns False. If you think that "is" does something different, you are wrong. >> > An address is an identifier: a number that I can use to access a >> > value[1]. >> >> Then by your own definition, Python's id() does not return an address, >> since you cannot use it to access a value. > > The fact Python lacks explicit dereferencing doesn't change the fact > that id() returns an address. Replace 'can' with 'could' or 'could > potentially' or the whole phrase with 'represents' if you wish. It's a > rather pointless thing to quibble over. You can't treat id() as an address. Did you miss my post when I demonstrated that Jython returns IDs generated on demand, starting from 1? In general, there is *no way even in principle* to go from a Python ID to the memory location (address) of the object with that ID, because in general objects *may not even have a fixed address*. Objects in Jython don't, because the Java virtual machine can move them in memory. The fact that CPython happens to use the memory address of objects, suitably converted to an int object, is a red-herring. It leads to nothing but confusion. > Would you call the result of casting a C pointer to an int an address? > If so, you must call the result of id() an address as well-- you can't > dereference either of them. If not, then you need to provide an > alternate name for the result of casting a C pointer to an int. I don't need to do anything of the sort. It was *your* definition, not mine. Don't put the responsibility on me for your definition being broken. (And for the record, in C you can cast an integer into a pointer, although the results are implementation-specific. There's no equivalent in Python.) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
HTML Code - Line Number
Hello, For scrapping purposes, I am having a bit of trouble writing a block of code to define, and find, the relative position (line number) of a string of HTML code. I can pull out one string that I want, and then there is always a line of code, directly beneath the one I can pull out, that begins with the following: However, because this string of HTML code above is not unique to just the information I need (which I cannot currently pull out), I was hoping there is a way to effectively say "if you find the html string _ in the line of HTML code above, and the string in the line immediately following, then pull everything that follows this second string. Any thoughts as to how to define a function to do this, or do this some other way? All insight is much appreciated! Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Direct vs indirect [was Re: why () is () and [] is [] work in other way?]
On Thu, 26 Apr 2012 04:42:36 -0700, Adam Skutt wrote: > On Apr 26, 5:10 am, Steven D'Aprano +comp.lang.pyt...@pearwood.info> wrote: >> But I was actually referring to something more fundamental than that. >> The statement "a is b" is a *direct* statement of identity. "John is my >> father." "id(a) == id(b)" is *indirect*: "The only child of John's >> grandfather is the parent of the mother-in-law of my sister-in-law" >> sort of thing. (Excuse me if I got the relationships mixed up.) > > Again, the fact that you somehow think this absurd family tree is > relevant only shows you're fundamentally confused about what object > oriented identity means. That's rather depressing, seeing as I've given > you a link to the definition. Perhaps you failed to notice that this "absurd" family tree, as you put it, consists of grandparent+parent+sibling+in-law. What sort of families are you familiar with that this seems absurd to you? I think you have inadvertently demonstrated the point I am clumsily trying to make. Even when two expressions are logically equivalent, the form of the expressions make a big difference to the comprehensibility of the text. Which would you rather read? for item in sequence[1:]: ... for item in sequence[sum(ord(c) for c in 'avocado') % 183:]: ... The two are logically equivalent, so logically you should have no preference between the two, yes? > In a mathematical sense, you're saying that given f(x) = x+2, using f(x) > is somehow more "direct" (whatever the hell that even means) I thought that the concept of direct and indirect statements would be self-evident. Let me try again. A statement is "direct" in the sense I mean if it explicitly states the thing you intend it to state. A statement is "indirect" if it requires one or more logical steps to go from the statement, as given, to the conclusion intended. "Queen Elizabeth II is the ruling monarch of the United Kingdom" is a direct statement of the fact that Queen Elizabeth II is the ruling monarch of the UK. (Do I really need to explain this?) "Queen Elizabeth II is the Commander-in-chief of the Canadian Armed Forces" is an *indirect* statement of the fact that Elizabeth is the ruling monarch of the UK. It is indirect because it doesn't explicitly say that she is monarch, but the Commander-in-Chief of the Canadian Armed Forces is always the ruling monarch of Canada, and the ruling monarch of Canada is always the ruling monarch of the UK. Hence, Elizabeth being Commander-in-Chief necessarily implies that she is ruling monarch of the United Kingdom (at least until there is change to Canadian law). "a is b" is a direct test of whether a is b. (Duh.) "id(a) == id(b)" is an indirect test of whether a is b, since it requires at least three indirect steps: the knowledge of what the id() function does, the knowledge of what the == operator does, and the knowledge that equal IDs imply identity. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
Adam Skutt wrote: [ ... ] > In the real world, if we were doing the math with pen and paper, we'd > stop as soon as we hit such an error. Equality is simply not defined > for the operations that can produce NaN, because we don't know to > perform those computations. So no, it doesn't conceptually follow > that NaN = NaN, what conceptually follows is the operation is > undefined because NaN causes a halt. > > This is what programming languages ought to do if NaN is compared to > anything other than a (floating-point) number: disallow the operation > in the first place or toss an exception. Any code that tries such an > operation has a logic error and must be fixed. There was a time when subtracting 5 from 3 would have been a logic error. Your phrase "if we were doing the math ..." lies behind most of the history of math, esp. as it concerns arithmetic. Mathematicians kept extending the definitions so that they wouldn't have to stop. Feynman's _Lectures on Physics_, chapter 22, "Algebra" gives a stellar account of the whole process. Mel. -- http://mail.python.org/mailman/listinfo/python-list
Borg identity [was Re: why () is () and [] is [] work in other way?]
On Thu, 26 Apr 2012 04:42:36 -0700, Adam Skutt wrote: > You're going to have to explain the value of an "ID" that's not 1:1 with > an object's identity, for at least the object's lifecycle, for a > programmer. If you can't come up with a useful case, then you haven't > said anything of merit. I gave an example earlier, but you seem to have misunderstood it, so I'll give more detail. In the Borg design pattern, every Borg instance shares state and are indistinguishable, with only one exception: object identity. We can distinguish two Borg instances by using "is". Since the whole point of the pattern is for Borg instances to be indistinguishable, the existence of a way to distinguish Borg instances is a flaw and may be undesirable. At least, it's exposing an implementation detail which some people argue should not be exposed. Why should the caller care whether they are dealing with a singleton object or an unspecified number of Borg objects all sharing state? A clever interpreter could make many Borg instances appear to be a singleton. A really clever one could also make a singleton appear to be many Borg instances. Note that this is virtually the same situation as that which John Nagle objects to, namely that the implementation detail of small ints being singletons is exposed. There is only ever one 0 instance, but potentially many 3579 instances. John's argument is that Python should raise an exception if you compare "2 is 2", or for that matter "3579 is 3579", which is foolish. If you're going to change the semantics of "is", why not do something useful and ensure that "3579 is 3579" returns True regardless of whether they actually are the same instance or not? That would be far more useful than raising an exception. It would complicate the definition of "is", but perhaps that's a price people are willing to pay for avoiding the (trivial) confusion about object identity. [...] >> identities. The Borg design pattern, for example, would be an excellent >> candidate for ID:identity being treated as many-to-one. > > How would inheritance work if I did that? You don't inherit from Borg instances, and instances inherit from their class the same as any other instance. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
On Fri, Apr 27, 2012 at 10:07 AM, Steven D'Aprano wrote: > How about you give an actual working example of what you mean by a code > block and how you use it? He wrote a full blog post about it last week: http://mtomassoli.wordpress.com/2012/04/20/code-blocks-in-python/ -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On Fri, Apr 27, 2012 at 9:39 AM, Adam Skutt wrote: > On Apr 27, 8:07 am, Kiuhnm wrote: >> Useful... maybe, conceptually sound... no. >> Conceptually, NaN is the class of all elements which are not numbers, >> therefore NaN = NaN. > > NaN isn't really the class of all elements which aren't numbers. NaN > is the result of a few specific IEEE 754 operations that cannot be > computed, like 0/0, and for which there's no other reasonable > substitute (e.g., infinity) for practical applications . > > In the real world, if we were doing the math with pen and paper, we'd > stop as soon as we hit such an error. Equality is simply not defined > for the operations that can produce NaN, because we don't know to > perform those computations. So no, it doesn't conceptually follow > that NaN = NaN, what conceptually follows is the operation is > undefined because NaN causes a halt. > > This is what programming languages ought to do if NaN is compared to > anything other than a (floating-point) number: disallow the operation > in the first place or toss an exception. Any code that tries such an > operation has a logic error and must be fixed. NaNs do not signify errors (for instance, a NaN could result simply from subtracting one Inf from another), and they do not necessarily imply that the calculation should halt. They are propagating values indicating the lack of a concrete value, and frequently they can reasonably be ignored. If you need to know about a NaN immediately, then you can trap them with fpectl. If you don't, then you let it propagate and check for it at the end of the calculation. If instead they raised exceptions by default, then we would need to wrap virtually every floating point operation in a try-except, which would quickly become a mess. See also: http://grouper.ieee.org/groups/754/faq.html#exceptions -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
On Fri, 27 Apr 2012 17:03:19 +0200, Kiuhnm wrote: > On 4/27/2012 16:09, Steven D'Aprano wrote: >> On Fri, 27 Apr 2012 13:24:35 +0200, Kiuhnm wrote: >> >>> I'd like to change the syntax of my module 'codeblocks' to make it >>> more pythonic. >>> >>> Current Syntax: >>> >>> with res << func(arg1) << 'x, y': >>> print(x, y) >>> >>> with res << func(arg1) << block_name << 'x, y': >>> print(x, y) >> >> >> I'm sorry, I don't see how this is a code block. Where is the code in >> the block, and how can you pass it to another object to execute it? > > Maybe if you read the entire post... No, I read the entire post. It made no sense to me. Let me give one example. You state: The full form is equivalent to def anon_func(x, y): print(x, y) res = func(arg1, block_name = anon_func) but this doesn't mean anything to me. What's func? Where does it come from? What's arg1? Why does something called block_NAME have a default value of a function instead of a NAME? How about you give an actual working example of what you mean by a code block and how you use it? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On Apr 27, 8:07 am, Kiuhnm wrote: > Useful... maybe, conceptually sound... no. > Conceptually, NaN is the class of all elements which are not numbers, > therefore NaN = NaN. NaN isn't really the class of all elements which aren't numbers. NaN is the result of a few specific IEEE 754 operations that cannot be computed, like 0/0, and for which there's no other reasonable substitute (e.g., infinity) for practical applications . In the real world, if we were doing the math with pen and paper, we'd stop as soon as we hit such an error. Equality is simply not defined for the operations that can produce NaN, because we don't know to perform those computations. So no, it doesn't conceptually follow that NaN = NaN, what conceptually follows is the operation is undefined because NaN causes a halt. This is what programming languages ought to do if NaN is compared to anything other than a (floating-point) number: disallow the operation in the first place or toss an exception. Any code that tries such an operation has a logic error and must be fixed. However, when comparing NaN against floating point numbers, I don't see why NaN == NaN returning false is any less conceptually correct than any other possible result. NaN's very existence implicitly declares that we're now making up the rules as we go along, so we might as well pick the simplest set of functional rules. Plus, floating point numbers violate our expectations of equality anyway, frequently in surprising ways. 0.1 + 0.1 + 0.1 == 0.3 is true with pen and paper, but likely false on your computer. It's even potentially possible to compare two floating point variables twice and get different results each time[1]! As such, we'd have this problem with defining equality even if NaN didn't exist. We must treat floating-point numbers as a special case in order to write useful working programs. This includes defining equality in a way that's different from what works for nearly every other data type. Adam [1] Due to register spilling causing intermediate rounding. This could happen with the x87 FPU since the registers were 80-bits wide but values were stored in RAM as 64-bits. This behavior is less common now, but hardly impossible. -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On 4/27/2012 16:07, Steven D'Aprano wrote: On Fri, 27 Apr 2012 14:17:48 +0200, Kiuhnm wrote: Define your terms: what do you mean by "equal"? a and b are equal iff Nope. What I meant is that we can talk of equality whenever... a = a a = b => b = a a = b and b = c => a = c If some of this properties are violated, we're talking of something else. Sorry, that won't do it. You haven't defined equality, or given any way of deciding whether two entities are equal. What you have listed are three *properties* of equality, namely: - reflexivity (a = a) - symmetry (if a = b then b = a) - transitivity (if a = b and b = c then a = c) But those three properties apply to any equivalence relation, not just equality. Examples: But that's what equality is in programming languages. You choose whatever you want. Just abide to those rules. "both are odd" (of integers) "have the same birthday" (of people) "is congruent to" (of triangles) "is in the same tax bracket" (of tax payers) "has the same length" (of pieces of string) "both contain chocolate" (of cakes) I can very well define a class Z_2 where 1 and 3 are equal (both are odd) and equality is defined as x % 2 == y % 2 This is why I keep saying that asking what 'equal' means lead us nowhere. Equality is whatever you see fit as long as you follows some rules. For example, if we define the operator "~" to mean "has the same genes" (to be precise: the same genotype), then if Fred and Barney are identical twins we have: Fred ~ Fred Fred ~ Barney and Barney ~ Fred Identical triplets are rare (at least among human beings), but if we clone Barney to get George, then we also have: Fred ~ Barney and Barney ~ George => Fred ~ George. So "have the same genes" meets all your conditions for equality, but isn't equality: the three brothers are very different. Fred lost his arm in a car crash, Barney is a hopeless alcoholic, and George is forty years younger than his two brothers. /You/ decide if '~' is a good definition for equality in your case. If it isn't, then define it another way. What I'm saying is that equality is in the eye of the beholder. To me, in some situations, all odd numbers are the same. What's wrong with that? Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: syntax for code blocks
On 4/27/2012 16:09, Steven D'Aprano wrote: On Fri, 27 Apr 2012 13:24:35 +0200, Kiuhnm wrote: I'd like to change the syntax of my module 'codeblocks' to make it more pythonic. Current Syntax: with res << func(arg1) << 'x, y': print(x, y) with res << func(arg1) << block_name << 'x, y': print(x, y) I'm sorry, I don't see how this is a code block. Where is the code in the block, and how can you pass it to another object to execute it? Maybe if you read the entire post... Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On 4/27/2012 1:57, Adam Skutt wrote: On Apr 26, 6:34 pm, Kiuhnm wrote: On 4/26/2012 20:54, Adam Skutt wrote: On Apr 26, 12:02 pm, Kiuhnmwrote: On 4/26/2012 16:00, Adam Skutt wrote: On Apr 26, 9:37 am, Kiuhnm wrote: The fact that you think that that's "differing behaviour" is what makes it a misfeature. The fact that you think that '==' can take objects as operands confirms that Java *does* confuse programmers. The equality operator can absolutely be used between two objects. Try it if you don't believe me. It always does identity comparison when given two objects. It can also be given two primitives, and in this case, it does value comparison. Despite performing different operations with the same symbol, there's little risk of confusion because I can trivially figure out if a variable is an object or an primitive. No, it can't be used between objects but only between primitives and references (which should be regarded as primitives, by the way). The only way to access an object is through a reference. If you understand that your 'a' is not really an object but a reference to it, everything becomes clear and you see that '==' always do the same thing. Yes, object identity is implemented almost? everywhere by comparing the value of two pointers (references)[1]. I've already said I'm not really sure how else one would go about implementing it. You might tell me that that's just an implementation detail, but when an implementation detail is easier to understand and makes more sense than the whole abstraction which is built upon it, something is seriously wrong. I'm not sure what abstraction is being built here. I think you have me confused for someone else, possibly Steven. The abstraction is this: - There are primitives and objects. - Primitives are not objects. The converse is also true. - Primitives can become objects (boxing). - Two primitives x and y are equal iff x == y. - Two objects x and y are equal iff x.equals(y). - Two objects are the same object iff x == y. - If x is a primitive, then y = x is a deep copy. - If x is an object, then y = x is a shallow copy. - ... The truth: - Primitives can be references. - Two primitives are equal iff x == y. - Operator '.' automatically derefences references. So Java is just C++ plus Garbage colletion plus some automatic dereferencing minus a lot of other things. On the other hand, Python regards everything as an object and that's the way we should go, in my opinion. Some objects are mutable and other are immutable. This distinction doesn't mess with references. Therefore, modifying the behavior of 'is' would be, in my opinion, a huge step back. You're missing the big picture. The two comparisons are asking different questions: Value equality asks if the operands 'have the same state' regardless of how they exist in memory. Identity equality asks if the two operands are the same block of memory. The two are distinct because not all types support both operations. If I write a function that does a value comparison, then it should do value comparison on _every type that can be passed to it_, regardless of whether the type is a primitive or an object, whether it has value or reference semantics, and regardless of how value comparison is actually implemented. If I write some function: f(x : T, y : U) => x == y where T and U are some unknown types, then I want the function to do a value comparison for every type pairing that allows the function to compile. Likewise, if I write a function that does identity comparison, then it logically wants to do identity comparison on _every type that can be passed to it_. "Value comparison" and "identity comparison" is a terminology that I don't agree on. Equality or equivalence is a relation which is: - reflexive - symmetric - transitive Everything else... is something else. Call it semi-equality, tricky-equality or whatever, but not equality, please. It's only natural that Python defaults to the Identity relation which is the (set-)intersection of all possible equality relations. To accomplish this, I must have a distinct way of asking each question. In Python we have '==' and 'is'[2]; in Java we have 'Object.equals()' and '=='; in C and C++ we distinguish by the types of the variables being compared (T and T*). Java gives '==' a different meaning for primitive types, but that turns out to be OK because I can't write a function that takes both a primitive type and a reference type at the same position. And as a side effect some integers are primitives and other are objects. Yes, the reason it does this is due to what I said above, but that doesn't have any bearing on why we pick one operation over the other as programmers. The distinction between primitives and objects is unfortunate. It is as if Java tried to get rid of pointers but never completely succeeded in doing that. It's the distinction between primitives and objects that shoul
Re: syntax for code blocks
On Fri, 27 Apr 2012 13:24:35 +0200, Kiuhnm wrote: > I'd like to change the syntax of my module 'codeblocks' to make it more > pythonic. > > Current Syntax: > > with res << func(arg1) << 'x, y': > print(x, y) > > with res << func(arg1) << block_name << 'x, y': > print(x, y) I'm sorry, I don't see how this is a code block. Where is the code in the block, and how can you pass it to another object to execute it? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On Fri, 27 Apr 2012 14:17:48 +0200, Kiuhnm wrote: >>> Define your terms: what do you mean by "equal"? >> >> a and b are equal iff > > Nope. What I meant is that we can talk of equality whenever... > >> a = a >> a = b => b = a >> a = b and b = c => a = c >> If some of this properties are violated, we're talking of something >> else. Sorry, that won't do it. You haven't defined equality, or given any way of deciding whether two entities are equal. What you have listed are three *properties* of equality, namely: - reflexivity (a = a) - symmetry (if a = b then b = a) - transitivity (if a = b and b = c then a = c) But those three properties apply to any equivalence relation, not just equality. Examples: "both are odd" (of integers) "have the same birthday" (of people) "is congruent to" (of triangles) "is in the same tax bracket" (of tax payers) "has the same length" (of pieces of string) "both contain chocolate" (of cakes) For example, if we define the operator "~" to mean "has the same genes" (to be precise: the same genotype), then if Fred and Barney are identical twins we have: Fred ~ Fred Fred ~ Barney and Barney ~ Fred Identical triplets are rare (at least among human beings), but if we clone Barney to get George, then we also have: Fred ~ Barney and Barney ~ George => Fred ~ George. So "have the same genes" meets all your conditions for equality, but isn't equality: the three brothers are very different. Fred lost his arm in a car crash, Barney is a hopeless alcoholic, and George is forty years younger than his two brothers. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Half-baked idea: list comprehensions with "while"
On 04/27/12 07:23, Chris Angelico wrote: On Fri, Apr 27, 2012 at 10:17 PM, John O'Hagan wrote: results = [x = expensive_call(i) for i in iterable if condition(x)] Nest it: results = [x for x in (expensive_call(i) for i in iterable) if condition(x)] While it's what I do in cases like this, the nesting is not nearly as readable as John's suggested syntax. Granted, the ability to do assignments in such a context opens up a whole can of unpythonic worms, so I'd suggest the following for clarity: temp = (expensive_call(i) for i in iterable) results = [x for x in temp if condition(x)] -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Half-baked idea: list comprehensions with "while"
On Fri, Apr 27, 2012 at 10:17 PM, John O'Hagan wrote: > results = [x = expensive_call(i) for i in iterable if condition(x)] Nest it: results = [x for x in (expensive_call(i) for i in iterable) if condition(x)] ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On 4/27/2012 14:07, Kiuhnm wrote: On 4/27/2012 13:09, Steven D'Aprano wrote: On Thu, 26 Apr 2012 18:02:31 +0200, Kiuhnm wrote: On 4/26/2012 16:00, Adam Skutt wrote: C# and Python do have a misfeature: '==' is identity comparison only if operator== / __eq__ is not overloaded. Identity comparison and value comparison are disjoint operations, so it's entirely inappropriate to combine them. They're not "disjoint", in fact one almost always implies the other (*). Python's idea is that, by default, any object is equal to itself and only itself. The fact that this is equivalent to "identity comparison" is just a coincidence, from a conceptual point of view. Define your terms: what do you mean by "equal"? a and b are equal iff Nope. What I meant is that we can talk of equality whenever... a = a a = b => b = a a = b and b = c => a = c If some of this properties are violated, we're talking of something else. Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: Half-baked idea: list comprehensions with "while"
On Fri, 27 Apr 2012 19:57:31 +1000 Chris Angelico wrote: > On Fri, Apr 27, 2012 at 7:49 PM, Miles Rout wrote: > > We have if inside list comprehensions? I didn't know that, could you provide > > an example? > > You mean like: > > [x*2+1 for x in range(10) if x%3] > Speaking of list comprehensions, if clauses and half-baked ideas, sometimes I'd like to be able to do something like: results = [x = expensive_call(i) for i in iterable if condition(x)] (or maybe "expensive_call(i) as x..." or whatever), instead of: results = [] for i in iterable: x = expensive_call(i) if condition(x): results.append(x) -- John -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On Fri, Apr 27, 2012 at 10:07 PM, Kiuhnm wrote: > Conceptually, NaN is the class of all elements which are not numbers, > therefore NaN = NaN. The conceptually correct way would be to check for > 'NaN' explicitly. Conceptually, "single-digit-numbers" is the class of all elements which are integers [0,10). Does that mean that SdN = SdN, and therefore that 2 = 5? ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: RuntimeWarning: Unable to load template engine entry point
Issue get solved by updating following packages $ easy_install -U DecoratorTools $ easy_install -U turbojson -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On 4/27/2012 13:09, Steven D'Aprano wrote: On Thu, 26 Apr 2012 18:02:31 +0200, Kiuhnm wrote: On 4/26/2012 16:00, Adam Skutt wrote: C# and Python do have a misfeature: '==' is identity comparison only if operator== / __eq__ is not overloaded. Identity comparison and value comparison are disjoint operations, so it's entirely inappropriate to combine them. They're not "disjoint", in fact one almost always implies the other (*). Python's idea is that, by default, any object is equal to itself and only itself. The fact that this is equivalent to "identity comparison" is just a coincidence, from a conceptual point of view. Define your terms: what do you mean by "equal"? a and b are equal iff a = a a = b => b = a a = b and b = c => a = c If some of this properties are violated, we're talking of something else. The fact that you can define '==' whatever way you want is irrelevant. I call that "calling 'equality' something which shouldn't be regarded as such and making Python comply with it anyway." The complication is that "equal" has many meanings. Does 1/2 equal 2/4? Well, yes, numerically, but numerical equality is not the only useful sense of equality -- not even for mathematicians! Although the convention to write "1/2 = 2/4" is too strong to discard, there are areas of mathematics where 1/2 and 2/4 are not treated as equal regardless of numerical equality. http://en.wikipedia.org/wiki/Mediant_%28mathematics%29 In Python, "equal" can have any meaning we like, because we can override __eq__. For most meaningful equality comparisons, we expect that X should always equal itself, even if it doesn't equal anything else, and so __eq__ defaulting to an identity comparison if you don't override it makes good sense. Some people (e.g. the creator of Eiffel, Bertrand Meyer) argue that identity should *always* imply equality (reflexivity). I disagree, but regardless, reflexivity is *almost always* the right thing to do. When it comes to equality, Python defaults to sensible behaviour. By default, any object supports equality. By default, "a == a" is true for any object a. If you want to define a non-reflexive type, you have to do so yourself. If you want to define a type that doesn't support equality at all, you have to do so yourself. But both use-cases are vanishingly rare, and rather troublesome to use. It would be stupid for Python to make them the default behaviour. After all, Python is a tool, not a philosophy. There's no need to force the user to start from a blank slate and define everything from first principles when you can start with the common tools you normally need, and add or subtract from it as needed. (*) nan == nan is false, but, at least conceptually, a 'NotComparable' exception should be raised instead. That wouldn't be very useful, though. NANs are comparable. By definition, NAN != x for every x. They're just not reflexive. As I said, I know they are but I think they shouldn't, at least conceptually. I don't necessarily mind if the two operations have the same symbol, as long as there's some other way in-context to determine which operation is occurring. This is the case in C and C++, for example. Python's way is much much cleaner. Nope. Automatically substituting identity equality for value equality is wrong. While rare, there are legitimate reasons for the former to be True while the latter is False. There shouldn't be, to be fair. I disagree. Violating reflexivity has its uses. NANs are the classic example. Useful... maybe, conceptually sound... no. Conceptually, NaN is the class of all elements which are not numbers, therefore NaN = NaN. The conceptually correct way would be to check for 'NaN' explicitly. Another example is if you redefine "==" to mean something other than "equals". If your class treats == as something other than equality, there is no need for a==a to necessarily return True. Then it wouldn't be equality anymore. I can always call a cat 'dog'. Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: Half-baked idea: list comprehensions with "while"
On Fri, Apr 27, 2012 at 8:37 PM, Tim Wintle wrote: > Or like: > print [ 0 if b%2==1 else 1 for b in range(10)] > [1, 0, 1, 0, 1, 0, 1, 0, 1, 0] That's nothing to do with the list comp, that's just the expression-if syntax that you can use anywhere. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
syntax for code blocks
I'd like to change the syntax of my module 'codeblocks' to make it more pythonic. Current Syntax: with res << func(arg1) << 'x, y': print(x, y) with res << func(arg1) << block_name << 'x, y': print(x, y) New Syntax: with res == func(arg1) .taking_block (x, y): print(x, y) with res == func(arg1) .taking_block (x, y) as block_name: print(x, y) The full form is equivalent to def anon_func(x, y): print(x, y) res = func(arg1, block_name = anon_func) Suggestions? Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On Thu, 26 Apr 2012 11:31:39 -0700, John Nagle wrote: > I would suggest that "is" raise ValueError for the ambiguous cases. > If both operands are immutable, "is" should raise ValueError. That's the > case where the internal representation of immutables shows through. You've already made this suggestion before. Unfortunately you failed to think it through: it would break *nearly all Python code*, and not just "broken" code. It would break code that relies on documented language features. It would break code that applies a standard Python idiom. I count at least 638 places where your suggestion would break the standard library. [steve@ando ~]$ cd /usr/local/lib/python3.2/ [steve@ando python3.2]$ grep "if .* is None:" *.py | wc -l 638 That's an average of four breakages per module. > If this breaks a program, it was broken anyway. Incorrect. Your suggestion breaks working code for no good reason. Astonishingly, your suggestion doesn't break code that actually is broken: def spam(arg=None): if arg == None: ... actually is broken, since it doesn't correctly test for the sentinel. You can break it by passing an object which compares equal to None but isn't actually None. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On 4/27/12 12:07 AM, Paul Rubin wrote: Nobody writes: All practical languages have some implementation-defined behaviour, often far more problematic than Python's. The usual reason for accepting implementation-defined behavior is to enable low-level efficiency hacks written for specific machines. C and C++ are used for that sort of purpose, so they leave many things implementation-defined. Python doesn't have the same goals and should leave less up to the implementation. Java, Ada, Standard ML, etc. all try to eliminate implementation-defined behavior in the language much more than Python does. I don't have any idea why you consider that to be "throwing the baby out with the bath water". I think there are two implementation-defined behaviors that are being discussed in this thread. One is that some immutable objects like the empty tuple may be interned such that all instances of them are the same object and have the same identity. This is allowed for efficiency reasons. CPython has used this freedom to good effect. The optimization is not theoretical. The other is that identities may be reused by different objects that do no coexist at the same time. This is to permit implementations where the ID is the address of the object in memory, like CPython. But other implementations with different memory models (including ones where "address in memory" doesn't make any sense) may forbid reuse of IDs. This allows alternative implementations like Jython and IronPython. There are specific, deliberate, practical consequences of those two implementation-defined behaviors. These are the babies that you would be throwing out. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Half-baked idea: list comprehensions with "while"
On Fri, 2012-04-27 at 19:57 +1000, Chris Angelico wrote: > On Fri, Apr 27, 2012 at 7:49 PM, Miles Rout wrote: > > We have if inside list comprehensions? I didn't know that, could you provide > > an example? > > You mean like: > > [x*2+1 for x in range(10) if x%3] Or like: >>> print [ 0 if b%2==1 else 1 for b in range(10)] [1, 0, 1, 0, 1, 0, 1, 0, 1, 0] -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On Thu, 26 Apr 2012 18:02:31 +0200, Kiuhnm wrote: > On 4/26/2012 16:00, Adam Skutt wrote: >> C# and Python do have a misfeature: '==' is identity comparison only if >> operator== / __eq__ is not overloaded. Identity comparison and value >> comparison are disjoint operations, so it's entirely inappropriate to >> combine them. > > They're not "disjoint", in fact one almost always implies the other (*). > Python's idea is that, by default, any object is equal to itself and > only itself. The fact that this is equivalent to "identity comparison" > is just a coincidence, from a conceptual point of view. Define your terms: what do you mean by "equal"? The complication is that "equal" has many meanings. Does 1/2 equal 2/4? Well, yes, numerically, but numerical equality is not the only useful sense of equality -- not even for mathematicians! Although the convention to write "1/2 = 2/4" is too strong to discard, there are areas of mathematics where 1/2 and 2/4 are not treated as equal regardless of numerical equality. http://en.wikipedia.org/wiki/Mediant_%28mathematics%29 In Python, "equal" can have any meaning we like, because we can override __eq__. For most meaningful equality comparisons, we expect that X should always equal itself, even if it doesn't equal anything else, and so __eq__ defaulting to an identity comparison if you don't override it makes good sense. Some people (e.g. the creator of Eiffel, Bertrand Meyer) argue that identity should *always* imply equality (reflexivity). I disagree, but regardless, reflexivity is *almost always* the right thing to do. When it comes to equality, Python defaults to sensible behaviour. By default, any object supports equality. By default, "a == a" is true for any object a. If you want to define a non-reflexive type, you have to do so yourself. If you want to define a type that doesn't support equality at all, you have to do so yourself. But both use-cases are vanishingly rare, and rather troublesome to use. It would be stupid for Python to make them the default behaviour. After all, Python is a tool, not a philosophy. There's no need to force the user to start from a blank slate and define everything from first principles when you can start with the common tools you normally need, and add or subtract from it as needed. > (*) nan == nan is false, but, at least conceptually, a 'NotComparable' > exception should be raised instead. That wouldn't be very useful, > though. NANs are comparable. By definition, NAN != x for every x. They're just not reflexive. >> I don't necessarily mind if the two operations have the same symbol, as >> long as there's some other way in-context to determine which operation >> is occurring. This is the case in C and C++, for example. >> >>> Python's way is much much cleaner. >> >> Nope. Automatically substituting identity equality for value equality >> is wrong. While rare, there are legitimate reasons for the former to >> be True while the latter is False. > > There shouldn't be, to be fair. I disagree. Violating reflexivity has its uses. NANs are the classic example. Another example is if you redefine "==" to mean something other than "equals". If your class treats == as something other than equality, there is no need for a==a to necessarily return True. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Half-baked idea: list comprehensions with "while"
On 4/27/2012 11:49, Miles Rout wrote: On 27/04/2012 5:57 a.m., Kiuhnm wrote: On 4/26/2012 19:48, Paul Rubin wrote: Roy Smith writes: x = [a for a in iterable while a] from itertools import takewhile x = takewhile(bool, a) I see that as a 'temporary' solution, otherwise we wouldn't need 'if' inside of list comprehensions either. Kiuhnm We have if inside list comprehensions? I didn't know that, could you provide an example? elems = [4, 1, 9, 6, 3, 8, 3, 9, 2] small_elems = [x for x in elems if x < 5] print str(small_elems) # 4, 1, 3, 3, 2 #leading_small_elems = [x for x in elems while x < 5] #print str(leading_small_elems) # 4, 1 Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: Half-baked idea: list comprehensions with "while"
On Fri, Apr 27, 2012 at 7:49 PM, Miles Rout wrote: > We have if inside list comprehensions? I didn't know that, could you provide > an example? You mean like: [x*2+1 for x in range(10) if x%3] ? ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Half-baked idea: list comprehensions with "while"
On 27/04/2012 5:57 a.m., Kiuhnm wrote: On 4/26/2012 19:48, Paul Rubin wrote: Roy Smith writes: x = [a for a in iterable while a] from itertools import takewhile x = takewhile(bool, a) I see that as a 'temporary' solution, otherwise we wouldn't need 'if' inside of list comprehensions either. Kiuhnm We have if inside list comprehensions? I didn't know that, could you provide an example? -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On Apr 26, 10:56 pm, "OKB (not okblacke)" wrote: > Adam Skutt wrote: > > If I write a function that does a value comparison, then it should > > do value comparison on _every type that can be passed to it_, > > regardless of whether the type is a primitive or an object, whether > > it has value or reference semantics, and regardless of how value > > comparison is actually implemented. If I write some function: > > f(x : T, y : U) => x == y > > where T and U are some unknown types, then I want the function to > > do a value comparison for every type pairing that allows the > > function to compile. Likewise, if I write a function that does > > identity comparison, then it logically wants to do identity > > comparison on _every type that can be passed to it_. > > What you say here makes perfect sense, but also shows that you > really shouldn't be using Python if you want stuff to work this way. In > Python any value of any type can be passed to any function. The claims > you are making about object identity and object equality are reasonable, > but as you show here, to really handle them requires dragging in a huge > amount of type-system baggage. So the check gets deferred to runtime, and the programmer may need to explictly throw 'NotImplemented' or something like that. Which is what happens in Python. Not type-checking arguments simply moves the burden from the language to the programmer, which is a standard consequence of moving from static to dynamic typing. Adam -- http://mail.python.org/mailman/listinfo/python-list
Re: (3.2) Overload print() using the C API?
Peter Faulks, 27.04.2012 10:36:
> On 27/04/2012 5:15 PM, Stefan Behnel wrote:
>> Peter Faulks, 26.04.2012 19:57:
>>> I want to extend an embedded interpreter so that calls to print() are
>>> automagically sent to a C++ gui (windows exe) via a callback function in
>>> the DLL.
>>>
>>> Then I'll be able to do this:
>>>
>>> test.py
>>> import printoverload
>>>
>>> printoverload.set_stdout()
>>> printoverload.set_stderr()
>>>
>>> print("this will be sent to a C function in printoverload.pyd")
>>> ---
>>
>> Why would you want to divert only "print" instead of changing sys.stdout in
>> general? Not all output comes from print calls.
>>
> Because I don't want to have to poll the stdout buffer.
You don't have to. It's delivered right at your door and even rings the
bell when it arrives to hand over the parcel in person.
> I want the script
> itself to update a window in the host application (via the extension) every
> time the script calls print().
Then replace sys.stdout (and maybe also sys.stderr) by another object that
does what you want whenever its write() method is called.
> But I guess that won't work if the script raises an exception...
Yep, you better catch those yourself. The C-API function you use for
executing the Python code in the first place will tell you when there was
an error.
BTW, my mention of Cython wasn't just a joke. You might want to look at it
because it makes these things essentially trivial compared to plain C-API
code in C.
Stefan
--
http://mail.python.org/mailman/listinfo/python-list
Re: (3.2) Overload print() using the C API?
On 27/04/2012 5:15 PM, Stefan Behnel wrote:
Peter Faulks, 26.04.2012 19:57:
I want to extend an embedded interpreter so that calls to print() are
automagically sent to a C++ gui (windows exe) via a callback function in
the DLL.
Then I'll be able to do this:
test.py
import printoverload
printoverload.set_stdout()
printoverload.set_stderr()
print("this will be sent to a C function in printoverload.pyd")
---
Why would you want to divert only "print" instead of changing sys.stdout in
general? Not all output comes from print calls.
Stefan
Because I don't want to have to poll the stdout buffer. I want the
script itself to update a window in the host application (via the
extension) every time the script calls print().
But I guess that won't work if the script raises an exception...
This is doing my head in. :)
--
http://mail.python.org/mailman/listinfo/python-list
Re: (3.2) Overload print() using the C API?
Peter Faulks, 26.04.2012 19:57:
> I want to extend an embedded interpreter so that calls to print() are
> automagically sent to a C++ gui (windows exe) via a callback function in
> the DLL.
>
> Then I'll be able to do this:
>
> test.py
> import printoverload
>
> printoverload.set_stdout()
> printoverload.set_stderr()
>
> print("this will be sent to a C function in printoverload.pyd")
> ---
Why would you want to divert only "print" instead of changing sys.stdout in
general? Not all output comes from print calls.
Stefan
--
http://mail.python.org/mailman/listinfo/python-list
Re: (3.2) Overload print() using the C API?
Peter Faulks, 26.04.2012 21:28: > "All you have to do is assign to print". Sounds great! Can some kind soul > hit me with a clue stick? Were do I look in the API? Here's the (Py3) Cython code for it: print = my_print_function Stefan -- http://mail.python.org/mailman/listinfo/python-list
