some more debug logs. #ifdef MAP_HUGETLB huge_sz = odp_sys_huge_page_size(); <--- 1 GB here need_huge_page = (huge_sz && alloc_size > page_sz); /* munmap for huge pages requires sizes round up by page */ alloc_hp_size = (size + align + (huge_sz - 1)) & (-huge_sz); #endif
*o/p: odp_shm_reserve : huge_sz 1073741824, need_huge_page 1, alloc_hp_size 1073741824* On Tue, May 10, 2016 at 6:23 PM, nousi <bnousi...@gmail.com> wrote: > value is zero in all the huge page parameters. > > root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# ls -l > total 0 > -r--r--r-- 1 root root 4096 May 10 18:15 free_hugepages > -rw-r--r-- 1 root root 4096 May 10 18:15 nr_hugepages > -rw-r--r-- 1 root root 4096 May 10 18:15 nr_hugepages_mempolicy > -rw-r--r-- 1 root root 4096 May 10 18:15 nr_overcommit_hugepages > -r--r--r-- 1 root root 4096 May 10 18:15 resv_hugepages > -r--r--r-- 1 root root 4096 May 10 18:15 surplus_hugepages > root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat > free_hugepages > 0 > root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat > nr_hugepages > 0 > root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat > nr_hugepages_mempolicy > 0 > root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat > nr_overcommit_hugepages > 0 > root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat > resv_hugepages > 0 > root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat > surplus_hugepages > 0 > root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# > > On Tue, May 10, 2016 at 6:19 PM, nousi <bnousi...@gmail.com> wrote: > >> yes, there are two directories under "/sys/kernel/mm/hugepages" >> >> root@ubuntu-15-10:/sys/kernel/mm/hugepages# ls -l >> total 0 >> drwxr-xr-x 2 root root 0 May 10 18:15 hugepages-1048576kB >> drwxr-xr-x 2 root root 0 May 10 18:15 hugepages-2048kB >> >> >> On Tue, May 10, 2016 at 6:13 PM, Maxim Uvarov <maxim.uva...@linaro.org> >> wrote: >> >>> In my case there is 1 Gb huge page in: >>> /sys/kernel/mm/hugepages >>> >>> and odp_shm_reserve() just rounds all allocations to 1 GB: >>> >>> #ifdef MAP_HUGETLB >>> huge_sz = odp_sys_huge_page_size(); <--- 1 GB here >>> need_huge_page = (huge_sz && alloc_size > page_sz); >>> /* munmap for huge pages requires sizes round up by page */ >>> alloc_hp_size = (size + align + (huge_sz - 1)) & (-huge_sz); >>> #endif >>> >>> Do you see the same thing? >>> >>> Maxim. >>> >>> On 05/10/16 15:07, nousi wrote: >>> >>>> Hi Maxim, >>>> >>>> Thanks for your support. >>>> >>>> mamp return below error; >>>> "mmap: Cannot allocate memory" >>>> "mount -t hugetlbfs none /mnt/hugetlbfs" also does not help. >>>> >>>> Huge page allocation success in below two calls after that it fails. >>>> 1) odp_thread_globals >>>> 2) odp_buffer_pools >>>> >>>> Please have a look at the console logs below. >>>> >>>> odp_shared_memory.c:299:odp_shm_reserve(): odp_thread_globals: huge >>>> page allocation success ! >>>> odp_shared_memory.c:299:odp_shm_reserve(): odp_buffer_pools: huge page >>>> allocation success ! >>>> odp_pool.c:104:odp_pool_init_global(): >>>> Pool init global >>>> odp_pool.c:105:odp_pool_init_global(): pool_entry_s size 8512 >>>> odp_pool.c:106:odp_pool_init_global(): pool_entry_t size 8512 >>>> odp_pool.c:107:odp_pool_init_global(): odp_buffer_hdr_t size 216 >>>> odp_pool.c:108:odp_pool_init_global(): >>>> *mmap: Cannot allocate memory* >>>> odp_queue.c:130:odp_queue_init_global():Queue init ... >>>> odp_shared_memory.c:297:odp_shm_reserve(): odp_queues: >>>> No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> >>>> >>>> Recently I Moved from Ubuntu 14.04 to 15.10, in Ubuntu 14.04 I had run >>>> calssfier example without any huge page allocation error. >>>> >>>> On Tue, May 10, 2016 at 3:50 PM, Maxim Uvarov <maxim.uva...@linaro.org >>>> <mailto:maxim.uva...@linaro.org>> wrote: >>>> >>>> Does: >>>> mount -t hugetlbfs none /mnt/hugetlbfs >>>> >>>> help? >>>> >>>> Maxim. >>>> >>>> >>>> On 10 May 2016 at 13:16, Maxim Uvarov <maxim.uva...@linaro.org >>>> <mailto:maxim.uva...@linaro.org>> wrote: >>>> >>>> looks like you have enough free HP. Which error returns >>>> mmap() on try with HP? >>>> >>>> On 10 May 2016 at 11:57, nousi <bnousi...@gmail.com >>>> <mailto:bnousi...@gmail.com>> wrote: >>>> >>>> linaro@ubuntu-15-10:~$ cat /proc/sys/vm/nr_hugepages >>>> 1024 >>>> linaro@ubuntu-15-10:~$ >>>> >>>> linaro@ubuntu-15-10:~$ cat /proc/meminfo >>>> MemTotal: 8061836 kB >>>> MemFree: 470516 kB >>>> MemAvailable: 1901932 kB >>>> Buffers: 92600 kB >>>> Cached: 1939696 kB >>>> SwapCached: 7516 kB >>>> Active: 3238960 kB >>>> Inactive: 1804492 kB >>>> Active(anon): 2712440 kB >>>> Inactive(anon): 1069492 kB >>>> Active(file): 526520 kB >>>> Inactive(file): 735000 kB >>>> Unevictable: 76 kB >>>> Mlocked: 76 kB >>>> SwapTotal: 16547836 kB >>>> SwapFree: 16370160 kB >>>> Dirty: 19816 kB >>>> Writeback: 0 kB >>>> AnonPages: 3004784 kB >>>> Mapped: 679960 kB >>>> Shmem: 770776 kB >>>> Slab: 264692 kB >>>> SReclaimable: 212172 kB >>>> SUnreclaim: 52520 kB >>>> KernelStack: 11952 kB >>>> PageTables: 65780 kB >>>> NFS_Unstable: 0 kB >>>> Bounce: 0 kB >>>> WritebackTmp: 0 kB >>>> CommitLimit: 19530176 kB >>>> Committed_AS: 11165432 kB >>>> VmallocTotal: 34359738367 kB >>>> VmallocUsed: 410416 kB >>>> VmallocChunk: 34358947836 kB >>>> HardwareCorrupted: 0 kB >>>> AnonHugePages: 583680 kB >>>> CmaTotal: 0 kB >>>> CmaFree: 0 kB >>>> HugePages_Total: 1024 >>>> HugePages_Free: 1022 >>>> HugePages_Rsvd: 1022 >>>> HugePages_Surp: 0 >>>> Hugepagesize: 2048 kB >>>> DirectMap4k: 234400 kB >>>> DirectMap2M: 6991872 kB >>>> DirectMap1G: 2097152 kB >>>> linaro@ubuntu-15-10:~$ >>>> >>>> >>>> On Tue, May 10, 2016 at 12:38 PM, Maxim Uvarov >>>> <maxim.uva...@linaro.org <mailto:maxim.uva...@linaro.org>> >>>> wrote: >>>> >>>> Please put output for: >>>> cat /proc/meminfo >>>> cat /proc/sys/vm/nr_hugepages >>>> >>>> Thank you, >>>> Maxim. >>>> >>>> >>>> >>>> On 10 May 2016 at 08:36, nousi <bnousi...@gmail.com >>>> <mailto:bnousi...@gmail.com>> wrote: >>>> >>>> mmap is failing in "odp_shm_reserve" function >>>> (odp_queue_init_global() ---> odp_shm_reserve() >>>> ---> odp_shm_reserve()) >>>> >>>> >>>> debug logs: >>>> >>>> root@ubuntu-15-10 >>>> :/home/linaro/linaro/odp/example/classifier# >>>> ./odp_classifier -i eno1 -m 0 -p >>>> "ODP_PMR_SIP_ADDR:192.168.10.11:FFFFFFFF:queue1" >>>> -p "ODP_PMR_SIP_ADDR:10.130.69.0:000000FF:queue2" >>>> -p "ODP_PMR_SIP_ADDR:10.130.68.0:FFFFFE00:queue3" >>>> odp_pool.c:104:odp_pool_init_global(): >>>> Pool init global >>>> odp_pool.c:105:odp_pool_init_global(): >>>> pool_entry_s size 8512 >>>> odp_pool.c:106:odp_pool_init_global(): >>>> pool_entry_t size 8512 >>>> odp_pool.c:107:odp_pool_init_global(): >>>> odp_buffer_hdr_t size 216 >>>> odp_pool.c:108:odp_pool_init_global(): >>>> odp_queue.c:130:odp_queue_init_global():Queue init >>>> ... >>>> >>>> odp_shared_memory.c:296:odp_shm_reserve():odp_queues: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> odp_queue.c:154:odp_queue_init_global():done >>>> odp_queue.c:155:odp_queue_init_global():Queue init >>>> global >>>> odp_queue.c:157:odp_queue_init_global(): struct >>>> queue_entry_s size 320 >>>> odp_queue.c:159:odp_queue_init_global(): >>>> queue_entry_t size 320 >>>> odp_queue.c:160:odp_queue_init_global(): >>>> >>>> odp_schedule.c:145:odp_schedule_init_global():Schedule >>>> init ... >>>> >>>> odp_shared_memory.c:296:odp_shm_reserve():odp_scheduler: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> >>>> odp_shared_memory.c:296:odp_shm_reserve():odp_sched_pool: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> odp_schedule.c:226:odp_schedule_init_global():done >>>> >>>> odp_shared_memory.c:296:odp_shm_reserve():odp_pktio_entries: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> >>>> odp_shared_memory.c:296:odp_shm_reserve():crypto_pool: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> >>>> odp_shared_memory.c:296:odp_shm_reserve():shm_odp_cos_tbl: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> >>>> odp_shared_memory.c:296:odp_shm_reserve():shm_odp_pmr_tbl: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> main :: odp_init_global done! >>>> odp_classifier.c:500:main():main :: odp_init_local >>>> done! >>>> odp_classifier.c:505:main():main :: >>>> odp_shm_reserve done! >>>> >>>> ODP system info >>>> --------------- >>>> ODP API version: 1.10.0 >>>> CPU model: Intel(R) Core(TM) i7-5600U CPU >>>> CPU freq (hz): 2600000000 >>>> Cache line size: 64 >>>> CPU count: 4 >>>> >>>> Running ODP appl: "odp_classifier" >>>> ----------------- >>>> Using IF:eno1 >>>> >>>> num worker threads: 2 >>>> first CPU: 2 >>>> cpu mask: 0xC >>>> >>>> odp_shared_memory.c:296:odp_shm_reserve():packet_pool: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> odp_pool.c:759:odp_pool_print():Pool info >>>> odp_pool.c:760:odp_pool_print():--------- >>>> odp_pool.c:762:odp_pool_print(): pool 1 >>>> odp_pool.c:764:odp_pool_print(): name packet_pool >>>> odp_pool.c:769:odp_pool_print(): pool type packet >>>> odp_pool.c:771:odp_pool_print(): pool storage ODP >>>> managed shm handle 11 >>>> odp_pool.c:773:odp_pool_print(): pool status active >>>> odp_pool.c:777:odp_pool_print(): pool opts >>>> segmented, non-zeroized, created >>>> odp_pool.c:778:odp_pool_print(): pool base >>>> 0x7f5091aab000 >>>> odp_pool.c:780:odp_pool_print(): pool size 1310720 >>>> (320 pages) >>>> odp_pool.c:781:odp_pool_print(): pool mdata base >>>> 0x7f5091bb5940 >>>> odp_pool.c:782:odp_pool_print(): udata size 0 >>>> odp_pool.c:783:odp_pool_print(): headroom 66 >>>> odp_pool.c:784:odp_pool_print(): tailroom 0 >>>> odp_pool.c:791:odp_pool_print(): seg length 1856 >>>> requested, 1936 used >>>> odp_pool.c:793:odp_pool_print(): pkt length 1856 >>>> requested, 1936 used >>>> odp_pool.c:795:odp_pool_print(): num bufs 564 >>>> odp_pool.c:797:odp_pool_print(): bufs available 564 >>>> odp_pool.c:798:odp_pool_print(): bufs in use 0 >>>> odp_pool.c:799:odp_pool_print(): buf allocs 0 >>>> odp_pool.c:800:odp_pool_print(): buf frees 0 >>>> odp_pool.c:801:odp_pool_print(): buf empty 0 >>>> odp_pool.c:803:odp_pool_print(): blk size 1936 >>>> odp_pool.c:805:odp_pool_print(): blks available 564 >>>> odp_pool.c:806:odp_pool_print(): blk allocs 0 >>>> odp_pool.c:807:odp_pool_print(): blk frees 0 >>>> odp_pool.c:808:odp_pool_print(): blk empty 0 >>>> odp_pool.c:809:odp_pool_print(): buf high wm >>>> value 282 >>>> odp_pool.c:810:odp_pool_print(): buf high wm count >>>> 0 >>>> odp_pool.c:811:odp_pool_print(): buf low wm >>>> value 141 >>>> odp_pool.c:812:odp_pool_print(): buf low wm count >>>> 0 >>>> odp_pool.c:813:odp_pool_print(): blk high wm >>>> value 282 >>>> odp_pool.c:814:odp_pool_print(): blk high wm count >>>> 0 >>>> odp_pool.c:815:odp_pool_print(): blk low wm >>>> value 141 >>>> odp_pool.c:816:odp_pool_print(): blk low wm count >>>> 0 >>>> main :: odp_pool_print done! >>>> odp_packet_io.c:230:setup_pktio_entry():eno1 uses >>>> socket_mmap >>>> created pktio:01, dev:eno1, queue mode (ATOMIC >>>> queues) >>>> default pktio01 >>>> odp_shared_memory.c:296:odp_shm_reserve(): >>>> DefaultPool: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> odp_shared_memory.c:296:odp_shm_reserve(): >>>> queue1Pool0: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> odp_shared_memory.c:296:odp_shm_reserve(): >>>> queue2Pool1: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> odp_shared_memory.c:296:odp_shm_reserve(): >>>> queue3Pool2: >>>> :: No huge pages, fall back to normal pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> >>>> ---------------------------------------- >>>> CLASSIFIER EXAMPLE STATISTICS >>>> ---------------------------------------- >>>> CONFIGURATION >>>> >>>> COS VALUE MASK >>>> ---------------------------------------- >>>> queue1 192.168.10.11 FFFFFFFF >>>> queue2 10.130.69.0 000000FF >>>> queue3 10.130.68.0 FFFFFE00 >>>> >>>> RECEIVED PACKETS >>>> ---------------------------------------- >>>> queue1 |queue2 |queue3 |DefaultCos |Total Packets >>>> queue pool |queue pool |queue pool |queue pool >>>> | >>>> 845 845 |0 0 |0 0 |38 38 |883 >>>> >>>> >>>> On Mon, May 9, 2016 at 9:00 PM, Bill Fischofer >>>> <bill.fischo...@linaro.org >>>> <mailto:bill.fischo...@linaro.org>> wrote: >>>> >>>> >>>> >>>> On Mon, May 9, 2016 at 6:57 AM, nousi >>>> <bnousi...@gmail.com >>>> <mailto:bnousi...@gmail.com>> wrote: >>>> >>>> >>>> Hi All, >>>> >>>> Pleas help me in running ODP classifier >>>> example with huge pages. >>>> In ubuntu 15.10 by default interface >>>> naming as "eno1" and value in >>>> "/proc/sys/vm/nr_hugepages." is 1024. >>>> classifier example program could not able >>>> to use huge pages even though nr_hugepages >>>> value is non zero. >>>> I could able to run the classier example, >>>> but it is not using huge pages. >>>> /* >>>> console log is pasted below for you >>>> reference. >>>> >>>> */root@odp/example/classifier# >>>> >>>> ./odp_classifier -i eno1 -m 0 -p >>>> "ODP_PMR_SIP_ADDR:192.168.10.11 >>>> :FFFFFFFF:queue1" >>>> -p >>>> "ODP_PMR_SIP_ADDR:10.130.69.0 >>>> :000000FF:queue2" >>>> -p >>>> "ODP_PMR_SIP_ADDR:10.130.68.0 >>>> :FFFFFE00:queue3" >>>> odp_pool.c:104:odp_pool_init_global(): >>>> Pool init global >>>> odp_pool.c:105:odp_pool_init_global(): >>>> pool_entry_s size 8512 >>>> odp_pool.c:106:odp_pool_init_global(): >>>> pool_entry_t size 8512 >>>> odp_pool.c:107:odp_pool_init_global(): >>>> odp_buffer_hdr_t size 216 >>>> odp_pool.c:108:odp_pool_init_global(): >>>> >>>> odp_queue.c:130:odp_queue_init_global():Queue >>>> init ... >>>> odp_shared_memory.c:296:odp_shm_reserve(): >>>> odp_queues: >>>> No huge pages, fall back to normal >>>> pages, >>>> check: /proc/sys/vm/nr_hugepages. >>>> >>>> >>>> This is an informational message saying that >>>> the linux-generic implementation was unable to >>>> allocate huge pages so it's falling back to >>>> normal pages. I'm not sure why you're seeing >>>> that except that it seems that some >>>> allocations may have been successful (those in >>>> odp_pool.c) while those for queue >>>> initialization were not. >>>> >>>> I'll let others who are more expert in this >>>> area chime in with some additional thoughts. >>>> >>>> >>>> >>>> >>>> Thanks & Regards, >>>> /B.Nousilal, >>>> / >>>> >>>> >>>> _______________________________________________ >>>> lng-odp mailing list >>>> lng-odp@lists.linaro.org >>>> <mailto:lng-odp@lists.linaro.org> >>>> >>>> https://lists.linaro.org/mailman/listinfo/lng-odp >>>> >>>> >>>> >>>> >>>> >>>> -- /Thanks & Regards, >>>> B.Nousilal, >>>> / >>>> >>>> _______________________________________________ >>>> lng-odp mailing list >>>> lng-odp@lists.linaro.org >>>> <mailto:lng-odp@lists.linaro.org> >>>> https://lists.linaro.org/mailman/listinfo/lng-odp >>>> >>>> >>>> >>>> >>>> >>>> -- /Thanks & Regards, >>>> B.Nousilal, >>>> / >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> /Thanks & Regards, >>>> B.Nousilal, >>>> / >>>> >>> >>> >> >> >> -- >> >> >> *Thanks & Regards,B.Nousilal,* >> > > > > -- > > > *Thanks & Regards,B.Nousilal,* > -- *Thanks & Regards,B.Nousilal,*
_______________________________________________ lng-odp mailing list lng-odp@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lng-odp