So with the mutex only one thread at a time is in pcap_loop()? I'm surprised a mutex is faster than having a manager thread dispatch workers, but I can see how it's safer than I thought.
Do the thread's share the pcap_t structure? I think each thread has it's own. I followed up on tcpdump-workers asking if there's anything hidden in the pcap_t structure that this would cause problems for - AFAIK as long as we don't use pcap_next_ex() it's ok. -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Luca Deri Sent: Monday, May 30, 2005 10:58 AM To: [email protected] Subject: Re: [Ntop-dev] ntop packet loss & multithreading Burton, if you go on the ntop-ht cvs you will see that there's a mutex for serializing the access to pcap_XXX. The new ntop-ht is running since last week on a busy host and it seems to be stable. It losses -50% packets of the "normal" ntop. I haven't heard from users yet, but I think it's time to start thinking to merge the two subtrees. Cheers, Luca Burton Strauss wrote: >Luca: > >This whole thing bothers me and I finally figured out why. You are >depending upon undocumented behavior of libpcap. You are ASSUMING that >a multi-threaded application, making pcap_loop() calls from each thread >against the same pcap_t structure will see different packets. A >load-balancing scenario. > >However, that's not documented. pcap_loop is this (0.8.3): > >int pcap_loop(pcap_t *p, int cnt, pcap_handler callback, u_char *user) >{ ... > for (;;) { >... > do { > n = p->read_op(p, cnt, callback, user); > } while (n == 0); ... > } >} > >The actual read operation is different for EACH environment: > >pcap-dag.c:484: handle->read_op = dag_read; >pcap-dlpi.c:713: p->read_op = pcap_read_dlpi; >pcap-linux.c:406: handle->read_op = pcap_read_linux; >pcap-nit.c:288: p->read_op = pcap_read_nit; >pcap-pf.c:442: p->read_op = pcap_read_pf; >pcap-snit.c:347: p->read_op = pcap_read_snit; >pcap-snoop.c:331: p->read_op = pcap_read_snoop; >pcap-win32.c:302: p->read_op = pcap_read_win32; > > >It certainly seems to work. However internally: > >(1) There is NO serialization. Especially under SMP (dual core), it's >going to be possible for multiple threads for the same device to be >running and end up inside (for example under Linux) recvfrom(). pcap_t >is an opaque structure, but if you look at it in pcap_int.h, it's just >a bunch of counters, fields and pointers. > >This isn't something we can patch/work-around in ntop - it's below the >pcap_loop() call, where libpcap might be sleeping awaiting packets. > > >(2) Different behavior under different OSes. Linux ends up using >recvfrom(), bpf uses read(). Win32 and dag use the memory addresses of >their ring buffers directly. > > >(3) Different behavior under different libpcaps. Between 0.7.x and >0.8.x this underwent a major rewrite to make the OS/device specific >operations work more transparently. C++ style class overloading >(read_op), vs. direct calls. It shouldn't FUNCTIONALLY be different, >but even so, these areas get a lot of changes especially with quirks for new OSes. > > >(4) Thread safety. In the mailing list - at least as of a year ago - >there were still some questions as to whether the DAG, DLPI >(HP/Solaris) versions were really thread safe. >http://www.tcpdump.org/lists/workers/2004/04/msg00179.html. Even if >they've fixed this, I highly doubt that they were thinking of this kind >of thread safety. > > >I really think you need to create a manager thread which runs >pcap_loop() and then dispatches the received packets to a number of worker threads. >That way, we can use your new functions to serialize access to the ntop >structures and safely update HostTraffic. > > >I'm going to post over on tcpdump-workers and will follow up here with >what I find. But I think this is just too far a leap for 3.2. > >-----Burton > > > > > > > >_______________________________________________ >Ntop-dev mailing list >[email protected] >http://listgateway.unipi.it/mailman/listinfo/ntop-dev > > -- Luca Deri <[EMAIL PROTECTED]> http://luca.ntop.org/ Hacker: someone who loves to program and enjoys being clever about it - Richard Stallman _______________________________________________ Ntop-dev mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-dev _______________________________________________ Ntop-dev mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
