#2134: Create a general exit-safe interface to C libraries
--------------------------------------------------+-------------------------
Reporter: wenzeslaus | Owner: grass-dev@…
Type: enhancement | Status: new
Priority: normal | Milestone: 7.0.0
Component: Python ctypes | Version: svn-trunk
Keywords: G_fatal_error, exit, multiprocessing | Platform: All
Cpu: Unspecified |
--------------------------------------------------+-------------------------
Comment(by huhabla):
In my opinion the RPC approach is only meaningful for persistent
applications that need fast access to C-library functions, or that need
low level API access for data modification (like digitizing).
My intention to write the RPC server was to make the temporal framework
usable in persistent applications and to be as fast as possible.
Extending the PyGRASS framework with signal handling will not solve the
problem: that memory structures may be corrupted in case of a
G_fatal_error() call or a segfault, leading to incorrect computation
results or to segmentation faults in the persistent application.
Implementation of persistent applications that use the GRASS C-library
functions in their process memory is IMHO not possible. We have had this
discussion for years.
The PyGRASS interface is well designed for module programming not for
persistent applications. Otherwise each C-function call should be handled
via RPC. From my point of view and some tests that i made slows the RPC
approach the processing significantly down.
Example:
{{{
from multiprocessing import Process, Pipe, Lock
import time
num = 50000
def func(a):
b = a + 1
c = b + 1
d = c + b
return d
# DICT
def dict_func():
a = {}
for i in xrange(num):
a["%i%i%i%i"%(i,i,i,i)] = func
start = time.time()
for i in xrange(num):
a["%i%i%i%i"%(i,i,i,i)](5)
end = time.time()
print end - start
def target(lock, conn):
a = [func]*num
while True:
# Avoid busy waiting
conn.poll(4)
data = conn.recv()
lock.acquire()
ret = a[data[0]](data[1])
conn.send(ret)
lock.release()
def rpc_func():
client_conn, server_conn = Pipe()
lock = Lock()
server = Process(target=target, args=(lock, server_conn))
server.daemon = True
server.start()
start = time.time()
for i in xrange(num):
if server.is_alive() is True:
data = [i, 5]
client_conn.send(data)
client_conn.recv()
end = time.time()
print end - start
if __name__ == "__main__":
# Measuring the time to call of 50000 functions that are
# registered in a dictionary
dict_func()
# Measuring the time to call 50000 functions
# using an RPC approach
rpc_func()
}}}
Running the script will show that calling functions from a dict is 50 time
faster than using a pipe with a subprocess.
{{{
python rpc_test.py
0.0984511375427
4.59985399246
}}}
--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2134#comment:7>
GRASS GIS <http://grass.osgeo.org>
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
