Re: [Cocci] Data exchange over network interfaces by SmPL scripts
> Unfortunately, I observed during a few runs on my test system > that the displayed record sets can vary. Thus I guess that this approach > (which works together with Python multi-threading functionality) will need > further software adjustments. I stumbled on general software development challenges from inter-process communication over TCP connections. This programming interface supports reliable data transmissions. But the POSIX API does not directly support so far to determine how many of the sent data are still on the way for delivery to the receiving process. * Operating systems can provide additional functions for this purpose. I find then that Linux APIs could be improved for more efficient analysis of network connections. * Network protocols influence also corresponding data processing approaches. + Customised network communication is not needed if you can depend on system functionality by databases. + If you would occasionally like to experiment with related services, the application of the technology “Common Object Request Broker Architecture” can be another interesting design option. Example: http://omniorb.sourceforge.net/ Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Data exchange over network interfaces by SmPL scripts
> connecting
…
> File "", line 35, in store_statements
> socket.gaierror: [Errno -2] Name or service not known
> Error in Python script, line 56, file …
It seems that the attached adjusted data processing approach can produce
an usable analysis result.
elfring@Sonne:~/Projekte/Coccinelle/janitor> time /usr/bin/python3
list_duplicate_statement_pairs_from_if_branches-server4.py
statement1|statement2|"function name"|"source file"|incidence
dprintk ( "%s: readreg error (reg == 0x%02x, ret == %i)\n" , __func__ , reg ,
ret ) ;|return - 1
;|stv0297_readreg|/home/elfring/Projekte/Linux/next-patched/drivers/media/dvb-frontends/stv0297.c|3
dprintk ( "%s: readreg error (reg == 0x%02x, ret == %i)\n" , __func__ , reg1 ,
ret ) ;|return - 1
;|stv0297_readregs|/home/elfring/Projekte/Linux/next-patched/drivers/media/dvb-frontends/stv0297.c|3
real0m1,044s
user0m0,389s
sys 0m0,055s
Unfortunately, I observed during a few runs on my test system
that the displayed record sets can vary. Thus I guess that this approach
(which works together with Python multi-threading functionality) will need
further software adjustments.
Would you like to add any advices here?
Regards,
Markus
import threading, socket, socketserver, struct, subprocess
inputs = []
def receive_data(s, n):
d = b''
while len(d) < n:
p = s.recv(n - len(d))
if not p:
return None
d += p
return d
def receive_message(s):
expect = receive_data(s, 4)
if not expect:
return None
return receive_data(s, struct.unpack(">I", expect)[0])
class threaded_TCP_request_handler(socketserver.BaseRequestHandler):
def handle(self):
data = receive_message(self.request)
if data:
inputs.append(data.decode())
class threaded_TCP_server(socketserver.ThreadingMixIn, socketserver.TCPServer):
pass
if __name__ == "__main__":
server = threaded_TCP_server(("localhost", 1234), threaded_TCP_request_handler)
with server:
ip, port = server.server_address
server_thread = threading.Thread(target = server.serve_forever)
server_thread.daemon = True
server_thread.start()
cp = subprocess.run(["/usr/local/bin/spatch",
"--timeout",
"9",
"--python",
"/usr/bin/python3",
"-D",
"server_id=" + str(ip),
"-D",
"server_port=" + str(port),
"/home/elfring/Projekte/Coccinelle/janitor/list_duplicate_statement_pairs_from_if_branches-client3.cocci",
"/home/elfring/Projekte/Linux/next-patched/drivers/media/dvb-frontends/stv0297.c"],
capture_output = True, text = True)
server.shutdown()
import sys
if cp.returncode:
sys.stderr.write("%s\n===\nexit code: %d" % (cp.stderr, cp.returncode))
else:
if len(inputs) > 0:
def report():
mapping = {}
def insert(x):
"""Add data to an internal table."""
key = x["name"], x["file"], x["line"], x["column"]
if key in mapping:
sys.stderr.write("""A duplicate key was passed.
function: %s
file: %s
line: %s
column: %d
""" % key)
raise RuntimeError
else:
mapping[key] = x["s1"], x["s2"]
def data_import():
import json
for k in inputs:
for v in json.loads(k):
insert(v)
data_import()
from collections import Counter
counts = Counter()
for k, v in mapping.items():
counts[(v[0], v[1], k[0], k[1])] += 1
delimiter = "|"
duplicates = {}
for k, v in counts.items():
if v > 1:
duplicates[k] = v
if len(duplicates.keys()) > 0:
sys.stdout.write(delimiter.join(["statement1",
"statement2",
'"function name"',
'"source file"',
"incidence"])
+ "\r\n")
for k, v in duplicates.items():
sys.stdout.write(delimiter.join([k[0], k[1], k[2], k[3], str(v)])
+ "\r\n")
else:
sys.stderr.write("Duplicate statements were not determined"
" from the following records.\n"
Re: [Cocci] Data exchange over network interfaces by SmPL scripts
> File "", line 35, in store_statements > socket.gaierror: [Errno -2] Name or service not known > Error in Python script, line 56, file … It seems that the attached adjusted data processing approach can produce an usable analysis result. elfring@Sonne:~/Projekte/Coccinelle/janitor> time /usr/bin/python3 list_duplicate_statement_pairs_from_if_branches-server4.py statement1|statement2|"function name"|"source file"|incidence dprintk ( "%s: readreg error (reg == 0x%02x, ret == %i)\n" , __func__ , reg , ret ) ;|return - 1 ;|stv0297_readreg|/home/elfring/Projekte/Linux/next-patched/drivers/media/dvb-frontends/stv0297.c|3 dprintk ( "%s: readreg error (reg == 0x%02x, ret == %i)\n" , __func__ , reg1 , ret ) ;|return - 1 ;|stv0297_readregs|/home/elfring/Projekte/Linux/next-patched/drivers/media/dvb-frontends/stv0297.c|3 real0m1,044s user0m0,389s sys 0m0,055s Unfortunately, I observed during a few runs on my test system that the displayed record sets can vary. Thus I guess that this approach (which works together with Python multi-threading functionality) will need further software adjustments. Would you like to add any advices here? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Data exchange over network interfaces by SmPL scripts
>> File "", line 34, in store_statements >> AttributeError: __exit__ >> Error in Python script, line 55, file … > > I have no idea. It looks like a python problem. Partly, yes (of course). > If you want help, you will have to construct a script > that exhibits the error with print statements only. This suggestion will probably not work (because additional test output will not trigger the shown error response alone.) I see solution challenges like the following after the addition of another data type conversion for the network service port variable. elfring@Sonne:~/Projekte/Coccinelle/janitor> /usr/local/bin/spatch --python /usr/bin/python3 -D server_id=127.0.0.1 -D server_port=1234 list_duplicate_statement_pairs_from_if_branches-client2.cocci ~/Projekte/Linux/next-patched/drivers/media/dvb-frontends/stv0297.c … Using Python version: 3.7.2 (default, Dec 30 2018, 16:18:15) [GCC] … connecting Traceback (most recent call last): File "", line 4, in File "", line 35, in store_statements socket.gaierror: [Errno -2] Name or service not known Error in Python script, line 56, file … How should the system configuration details be improved for a simple test connection? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
