Hello hackers, while working on the "ringmap"-project I've faced a problem of "no coherency in the memory regions mapped from kernel into the user-space".
Details: While integrating ringmap with the ixgbe-driver, I've made some changes to the ixgbe: 1. The mbufs for received packets will be only allocated once. 2. Allocated mbufs will be reused as in ring-buffer one after the other (no new mbufs will be allocated again). 3. Packet buffers (mbuf->m_data) will mapped into the user-space. So, the user-space process has access to the packets after those DMA-transfer from the network adapter into the RAM Problem: Sometimes the user-space process sees not new DMAed data in the mapped packet-buffer, but the OLD data that was previously stored in the same packet buffer. If I try to monitor the received data in the kernel, the kernel sees the data correctly. But sometimes it is vice versa: the user-space process sees the correct new data and the kernel sees the old data in the buffer. It seems to be that the memory-buffer for packets is not synchronized with all CPU's caches. Probably [user|kernel]-thread tries sometimes to reads the old dirty data from the cache of the CPU the thread running on. (In the same time the other thread sees the new data in the same mapped buffer). Can you please provide me with some information that would be helpful for avoiding this unexpected coherence-problem. Alex P.S. Details about hardware and used software: 1. /var/run/dmesg.boot : ... CPU: Dual Core AMD Opteron(tm) Processor 865 (1800.01-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x20f10 Family = f Model = 21 Stepping = 0 Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> Features2=0x1<SSE3> AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow!+,3DNow!> AMD Features2=0x3<LAHF,CMP> real memory = 3758030848 (3583 MB) avail memory = 3677495296 (3507 MB) ACPI APIC Table: <A M I OEMAPIC > FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 4 package(s) x 2 core(s) ... 2. uname -v FreeBSD 9.0-CURRENT #3 3. sysctl kern.osreldate kern.osreldate: 900014 4. //depot/projects/soc2010/ringmap/ _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
