I confirm the same problem, except that it happens more often for me, about 50
to 60 times each time I run the test.
It smells like some sort of “unsynchronised acess by different threads to an
object problem ”.
One (very) hacky way round it until the real cause is established is to modify
__contains__() in _weakrefset.py line 68 to retry until it gets a proper result
def __contains__(self, item):
while 1:
try:
try:
wr = ref(item)
except TypeError:
return False
return wr in self.data
except Exception:
pass
Regards
Andy Graham
From: Andres Sommerhoff
Sent: Monday, April 13, 2015 6:39 AM
To: ironpython-users@python.org
Subject: [Ironpython-users] Fwd: weakref random "SystemError" and "ValueError"
exception (affecting _weakrefset.py and abc.py)
Dear IronPython gurus!
Hopping you can help me with a kind of ramdom bug (unexpected SystemError and
ValueError exception) in _weakrefset.py (it would be great if you can replicate
it by running my script below. Please let me know).
The error is random, but I managed to reproduce it by running the "ramdomly"
ofending code 100.000 times inside a loop (Script attached). Note that the test
takes only a few seconds, and in my PC it throws regularly between 5 to 30
exception for all those cycles). I see there are other people suffering for the
same, but without solution or workaround yet
(https://mail.python.org/pipermail/ironpython-users/2014-November/017332.html
and https://github.com/IronLanguages/main/issues/1187)
In my case, weakref error was related with an intensive use of isinstance()
inside "Pulp" library (an optimization python library). Just for your
reference: isintance() use internally a WeakSet as a cache for the class types,
which in turn use weakref (see ABCMeta.__instancecheck__() in the standard file
"abc.py").
In my test script I have isolated the problem to WeakSet only (I isolated it to
clean the bug from the "abc.py" and "Pulp" library stuff). The exception
happens inside WeakSet.__contains__() function (from _weakrefset.py file). As
stated, it happens randomly, but apparently only related with a GC collect
cycle into a memory "hungry" python script . I ran the test script in two PCs
with similar results: Windows 7 64bits and Windows 7 32bits. Both using ipy.exe
32bit version 2.7.4 (2.7.0.40). The .NET version is 4.0.30319.18444 (32-bit).
The test script does:
1.. It simulate a "memory intensive" python code (creating a weakref object
of 2kb to 12kb in each loop. If smaller, like 0.1kb objects, then the bug don't
show up)
2.. It manually runs GC.collect() every 1.000 cycles (should collect those
weakref objects)
3.. ... and it repeat (100.000 times) the following "offending" boolean test:
test = item in WeakSetObject #<- Repeated 100.000 times.
#-> it fails between 10 to 20 times with
an unexpected exception
NOTE 1: The "item" is an object added to the WeakSet at the beginning of the
script, but "item" should not GC collected as it also keeps a normal (not weak)
reference alive.
NOTE 2: The boolean test should give always True, which is the case 99.9% of
the time. Less than 0.01%, the boolean test fails raising an exception of the
type "ValueError" (Description:"Index was out of range") or a bit more frequent
"SystemError" (Description:"Handle is not initialized"). This happens 5 to 30
times in 100.000 test cycle (Seems very small, but it is important enough to
avoid a practical construction of a medium size optimization problem with
"Pulp" library).
Tracking down the error, the exception ValueError is raised in line 70 and the
exception "SystemError" in line 73 of "_weakrefset.py" .
On "Lib\_weakrefset.py"
35 :class WeakSet(object):
....
68 : def __contains__(self, item):
69 : try:
*70: wr = ref(item) # <- here is raised "ValueError"
Exception ("Index was out of range")
71 : except TypeError:
72 : return False
*73: return wr in self.data # <- here is raised "SystemError"
Exception ("Handle is not initialized")
Continuing after the exception, when executing the same boolean test again, it
works fine (same item, same WeakSetObject, same execution, local and global
context. Script was not started again!). I.e. if you catch the exception and
continue the execution, It is like as if the exception never happened before
(it's like a runtime lapsus!).
I believe to fix the source of the problem, Ironpython should trap or avoid the
error in weakref module itself (C# code). I don't know how... can someone
kindly help me to fix this?
Cheers,
Andres Sommerhoff
--------------------------------------------------------------------------------
_______________________________________________
Ironpython-users mailing list
Ironpython-users@python.org
https://mail.python.org/mailman/listinfo/ironpython-users
_______________________________________________
Ironpython-users mailing list
Ironpython-users@python.org
https://mail.python.org/mailman/listinfo/ironpython-users
