On 11/12/2011 05:12 PM, Burton Strauss III wrote:
You need the actual THREAD id
- check the man page on ps.
i know, but even if i can abviously list the created threads in many different
ways with 'ps',
i would not identity the data collection thread ID using simply ps output at
runtime.
But it clearly
shows both the child and parent PID.
please show me where it shows in the ntop output?
i still see only the parent PID. In the lines produced by ntop on the very end
of a few lines you can see [p1434] that in my case is the parent ID.
I cannot see any other number that can possible be related it the child threads.
After the 'THREADMGMT' string there are additional values like
't140309304494336'
do you mean this? This is not a thread PID.
What i need is the thread PID, obviously after it has been created by the
father process, to pass it as a 'taskset' argument, and set the CPU affinity
only on this thread.
I cannot do this, by passing the father PID to 'taskset', because for newly
created threads the system will assign the same cpu affinity.
There might be other methods of finding out the thread ID, for example looking
in detail at the IO for each thread using linux system monitoring tools,
but i need to find it automatically when ntop starts and after that apply the
cpu affinity.
It depends on your OS, some create threads from fork() some full processes.
That's why we show both.
i think this is not the problem here.
i'm on a RHEL 6.1 with the usual pthread library which ntop uses.
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Enrico
Sent: Friday, November 11, 2011 3:15 PM
To: [email protected]
Subject: Re: [Ntop] ntop - set cpu affinity for data collection thread
this is simply not true, at least in my system.
as i have written in my first post the PID showed is the one of the father
process.
grep -i threadmgmt /var/log/messages
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309304494336]: Now running
as a daemon
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309268899584]: SFP: Started
thread for fingerprinting
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309268899584]: SFP:
Fingerprint scan thread starting [p1434]
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309260506880]: SIH: Idle
host scan thread starting [p1434]
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309260506880]: SIH: Started
thread for idle hosts detection
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309252114176]: DNSAR(1):
Started thread for DNS address resolution
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309252114176]: DNSAR(1):
Address resolution thread running
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309243721472]: DNSAR(2):
Address resolution thread running
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309243721472]: DNSAR(2):
Started thread for DNS address resolution
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309235328768]: DNSAR(3):
Started thread for DNS address resolution
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309235328768]: DNSAR(3):
Address resolution thread running
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309226936064]: INITWEB:
Started thread for web server
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309226936064]: WEB: Server
connection thread starting [p1434]
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140309226936064]: WEB: Server
connection thread running [p1434]
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT: RRD: Started thread
(t140308864038656) for data collection
Nov 11 16:38:21 ids01 ntop[1434]: THREADMGMT[t140308864038656]: RRD: Data
collection thread starting [p1434]
Nov 11 16:38:22 ids01 ntop[1434]: THREADMGMT[t140309304494336]: ntop
RUNSTATE: INITNONROOT(3)
Nov 11 16:38:22 ids01 ntop[1434]: THREADMGMT[t140309304494336]: ntop
RUNSTATE: RUN(4)
Nov 11 16:38:22 ids01 ntop[1434]: THREADMGMT[t140308855645952]: NPS(1):
Started thread for network packet sniffing [eth0]
Nov 11 16:38:22 ids01 ntop[1434]: THREADMGMT[t140308855645952]: NPS(eth0):
pcapDispatch thread starting [p1434]
Nov 11 16:38:22 ids01 ntop[1434]: THREADMGMT[t140309268899584]: SFP:
Fingerprint scan thread running [p1434]
Nov 11 16:38:22 ids01 ntop[1434]: THREADMGMT[t140309260506880]: SIH: Idle
host scan thread running [p1434]
Nov 11 16:38:22 ids01 ntop[1434]: THREADMGMT[t140308855645952]: NPS(eth0):
pcapDispatch thread running [p1434]
Nov 11 16:38:32 ids01 ntop[1434]: THREADMGMT[t140308847253248]: RRD: Started
thread for throughput data collection
Nov 11 16:38:32 ids01 ntop[1434]: THREADMGMT[t140308864038656]: RRD: Data
collection thread running [p1434]
Nov 11 16:38:32 ids01 ntop[1434]: THREADMGMT[t140308847253248]: RRD:
Throughput data collection: Thread starting [p1434]
Nov 11 16:38:32 ids01 ntop[1434]: THREADMGMT[t140308847253248]: RRD:
Throughput data collection: Thread running [p1434]
ps -efL | grep 1434
ntop 1434 1 1434 0 10 16:38 ? 00:00:00 /usr/local/bin/ntop
@/usr/local/etc/ntop/eth0-start --protocols /usr/local/etc/ntop/eth0-proto -d
ntop 1434 1 1436 0 10 16:38 ? 00:00:00 /usr/local/bin/ntop
@/usr/local/etc/ntop/eth0-start --protocols /usr/local/etc/ntop/eth0-proto -d
ntop 1434 1 1437 0 10 16:38 ? 00:00:01 /usr/local/bin/ntop
@/usr/local/etc/ntop/eth0-start --protocols /usr/local/etc/ntop/eth0-proto -d
ntop 1434 1 1438 0 10 16:38 ? 00:00:00 /usr/local/bin/ntop
@/usr/local/etc/ntop/eth0-start --protocols /usr/local/etc/ntop/eth0-proto -d
ntop 1434 1 1439 0 10 16:38 ? 00:00:00 /usr/local/bin/ntop
@/usr/local/etc/ntop/eth0-start --protocols /usr/local/etc/ntop/eth0-proto -d
ntop 1434 1 1440 0 10 16:38 ? 00:00:00 /usr/local/bin/ntop
@/usr/local/etc/ntop/eth0-start --protocols /usr/local/etc/ntop/eth0-proto -d
ntop 1434 1 1443 0 10 16:38 ? 00:00:05 /usr/local/bin/ntop
@/usr/local/etc/ntop/eth0-start --protocols /usr/local/etc/ntop/eth0-proto -d
ntop 1434 1 1445 0 10 16:38 ? 00:00:00 /usr/local/bin/ntop
@/usr/local/etc/ntop/eth0-start --protocols /usr/local/etc/ntop/eth0-proto -d
ntop 1434 1 1448 7 10 16:38 ? 00:25:55 /usr/local/bin/ntop
@/usr/local/etc/ntop/eth0-start --protocols /usr/local/etc/ntop/eth0-proto -d
ntop 1434 1 1477 0 10 16:38 ? 00:00:00 /usr/local/bin/ntop
@/usr/local/etc/ntop/eth0-start --protocols /usr/local/etc/ntop/eth0-proto -d
root 4308 3626 4308 0 1 22:16 pts/0 00:00:00 grep 1434
still waiting for a reply. Eventually is accept suggestions for modifying
the code that forks this particular thread.
thanks.
On 11/11/2011 06:41 PM, Burton Strauss III wrote:
Read the log - look for the THREADMGMT messages. They show thread
creation and WHAT each thread is.
-----Burton
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Enrico
Sent: Thursday, November 10, 2011 8:26 AM
To: [email protected]
Subject: [Ntop] ntop - set cpu affinity for data collection thread
Hello
we have seen that the snort-daq module, that comes with pfring, can
bind the data acquistion process to a particular cpu subset with the
'sched_setaffinity (2)' function.
This is really useful for reducing interrupts handling time in our
system
We would like to be able to do the same thing also for ntop.
Unlike snort we see that it is an high parallelized application and
usually creates almost 10 threads when starts.
Using Taskset would bind all the 10 threads to a single cpu, but
brings to resources competition between ntop threads on the same core
which increases a lot the context switching time.
Is there a way to know what is the data acquisition/collection thread
for a ntop instance that actually handles the packets coming from the
kernel?
Looking at ntop logs for example only shows what is the father process
creating that thread.
Any suggestion?
Thanks in advance.
Enrico.
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
