Re: netisr issues
Hi, is there any decent documentation what netisr is , what relevant sysctls/loader options are there, and in what cases it should/better be used, ie best practice? Thanks, Sami On Mon, Mar 11, 2013 at 6:48 PM, Freddie Cash wrote: > On Mon, Mar 11, 2013 at 9:40 AM, Mark Saad wrote: > > > Freddie > >So should I be adjusting the numbers of threads or is this > > determined somewhere ? > > > > I think it's supposed to be automatic, 1 thread per CPU, but I manually > set it via /boot/loader.conf: > net.isr.bindthreads="1" # Bind netisr threads to > CPU cores > net.isr.maxthreads="8" # Set number of threads to > number of CPU cores > net.isr.numthreads="8" #"" > > The net.isr.dispatch is automatically set to direct on our systems. Not > sure if that's the default or not. We used to set that via > /boot/loader.conf as well, but it was removed in the upgrade to 9-STABLE > something as it no longer did anything. > > -- > Freddie Cash > fjwc...@gmail.com > ___ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" > -- Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: netisr issues
On Mon, Mar 11, 2013 at 12:48 PM, Freddie Cash wrote: > On Mon, Mar 11, 2013 at 9:40 AM, Mark Saad wrote: >> >> Freddie >>So should I be adjusting the numbers of threads or is this >> determined somewhere ? >> > I think it's supposed to be automatic, 1 thread per CPU, but I manually set > it via /boot/loader.conf: > net.isr.bindthreads="1" # Bind netisr threads to CPU > cores > net.isr.maxthreads="8" # Set number of threads to > number of CPU cores > net.isr.numthreads="8" #"" > > The net.isr.dispatch is automatically set to direct on our systems. Not > sure if that's the default or not. We used to set that via > /boot/loader.conf as well, but it was removed in the upgrade to 9-STABLE > something as it no longer did anything. > > -- > Freddie Cash > fjwc...@gmail.com Just so I am clear , prior to r49 by rwatson net.isr.direct=1 enabled the same interaction as net.isr.dispatch=deferred after the commit , with the latter commit enabling netisr to have mulch-threaded support ? -- mark saad | nones...@longcount.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: netisr issues
On Mon, Mar 11, 2013 at 9:40 AM, Mark Saad wrote: > Freddie >So should I be adjusting the numbers of threads or is this > determined somewhere ? > > I think it's supposed to be automatic, 1 thread per CPU, but I manually set it via /boot/loader.conf: net.isr.bindthreads="1" # Bind netisr threads to CPU cores net.isr.maxthreads="8" # Set number of threads to number of CPU cores net.isr.numthreads="8" #"" The net.isr.dispatch is automatically set to direct on our systems. Not sure if that's the default or not. We used to set that via /boot/loader.conf as well, but it was removed in the upgrade to 9-STABLE something as it no longer did anything. -- Freddie Cash fjwc...@gmail.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: netisr issues
On Mon, Mar 11, 2013 at 12:26 PM, Freddie Cash wrote: > I seem to recall that the method for setting "direct" changed so that it's > not a binary option (net.isr.direct), but instead is a policy setting now > (net.isr.dispatch). Try: > > net.isr.dispatch="direct" > > That's what's set on our 9.1-STABLE systems: > # sysctl net.isr > net.isr.numthreads: 8 > net.isr.maxprot: 16 > net.isr.defaultqlimit: 256 > net.isr.maxqlimit: 10240 > net.isr.bindthreads: 1 > net.isr.maxthreads: 8 > net.isr.direct: 0 > net.isr.direct_force: 0 > net.isr.dispatch: direct > > # netstat -Q | head > Configuration: > SettingCurrentLimit > Thread count 88 > > Default queue limit25610240 > Dispatch policy direct n/a > Threads bound to CPUs enabled n/a > > > > On Mon, Mar 11, 2013 at 9:22 AM, Freddie Cash wrote: >> >> You're right. I was looking at different net.isr oids, not the _direct >> ones. My bad. >> >> >> On Mon, Mar 11, 2013 at 9:17 AM, Mark Saad wrote: >>> >>> On Mon, Mar 11, 2013 at 12:11 PM, Freddie Cash wrote: >>> > Works if you set them in /etc/sysctl.conf. Haven't looked into it, but >>> > I >>> > think there's something in the startup that sets them to 0 after the >>> > kernel >>> > is loaded, so the loader.conf settings are overwritten. >>> > >>> > >>> > On Mon, Mar 11, 2013 at 7:42 AM, Mark Saad >>> > wrote: >>> >> >>> >> All >>> >> I am looking for some guidance on how to turn netisr back on, on a >>> >> 9.1-RELEASE and 9.1-STABLE box. It looks it stopped working as it did >>> >> in >>> >> prior versions of FreeBSD . >>> >> I tested this on 9.1-RELEASE and 9.1-STABLE #0 r247804 built last >>> >> monday. >>> >> >>> >> My question is this. If I enable the direct option in boot/loader.conf >>> >> via >>> >> this >>> >> >>> >> net.isr.direct="1" >>> >> net.isr.direct_force="1" >>> >> >>> >> I do not get any expected result. >>> >> >>> >> root@chambers:~ # sysctl net.isr.direct >>> >> net.isr.direct: 0 >>> >> root@chambers:~ # sysctl net.isr.direct_force >>> >> net.isr.direct_force: 0 >>> >> >>> >> root@chambers:~ # netstat -Q >>> >> Configuration: >>> >> SettingCurrentLimit >>> >> Thread count 11 >>> >> Default queue limit25610240 >>> >> Dispatch policy direct n/a >>> >> Threads bound to CPUs disabled n/a >>> >> >>> >> >>> >> >>> >> >>> >> Am I missing something ? >>> >> >>> >> >>> >> -- >>> >> mark saad | nones...@longcount.org >>> >> ___ >>> >> freebsd-stable@freebsd.org mailing list >>> >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >>> >> To unsubscribe, send any mail to >>> >> "freebsd-stable-unsubscr...@freebsd.org" >>> > >>> > >>> > >>> > >>> > -- >>> > Freddie Cash >>> > fjwc...@gmail.com >>> >>> Freddie >>> When I tried to set them in /etc/sysctl.conf , sysctl stated they >>> were read-only . >>> >>> [root@mkr2 /etc]# sysctl -w net.isr.direct=1 >>> sysctl: oid 'net.isr.direct' is read only >>> >>> >>> [root@mkr2 /etc]# sysctl -w net.isr.direct_force=1 >>> sysctl: oid 'net.isr.direct_force' is read only >>> >>> -- >>> mark saad | nones...@longcount.org >>> ___ >>> freebsd-stable@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >>> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" >> >> >> >> >> -- >> Freddie Cash >> fjwc...@gmail.com > > > > > -- > Freddie Cash > fjwc...@gmail.com Freddie So should I be adjusting the numbers of threads or is this determined somewhere ? -- mark saad | nones...@longcount.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: netisr issues
I seem to recall that the method for setting "direct" changed so that it's not a binary option (net.isr.direct), but instead is a policy setting now (net.isr.dispatch). Try: net.isr.dispatch="direct" That's what's set on our 9.1-STABLE systems: # sysctl net.isr net.isr.numthreads: 8 net.isr.maxprot: 16 net.isr.defaultqlimit: 256 net.isr.maxqlimit: 10240 net.isr.bindthreads: 1 net.isr.maxthreads: 8 net.isr.direct: 0 net.isr.direct_force: 0 net.isr.dispatch: direct # netstat -Q | head Configuration: SettingCurrentLimit Thread count 88 Default queue limit25610240 Dispatch policy direct n/a Threads bound to CPUs enabled n/a On Mon, Mar 11, 2013 at 9:22 AM, Freddie Cash wrote: > You're right. I was looking at different net.isr oids, not the _direct > ones. My bad. > > > On Mon, Mar 11, 2013 at 9:17 AM, Mark Saad wrote: > >> On Mon, Mar 11, 2013 at 12:11 PM, Freddie Cash wrote: >> > Works if you set them in /etc/sysctl.conf. Haven't looked into it, but >> I >> > think there's something in the startup that sets them to 0 after the >> kernel >> > is loaded, so the loader.conf settings are overwritten. >> > >> > >> > On Mon, Mar 11, 2013 at 7:42 AM, Mark Saad >> wrote: >> >> >> >> All >> >> I am looking for some guidance on how to turn netisr back on, on a >> >> 9.1-RELEASE and 9.1-STABLE box. It looks it stopped working as it did >> in >> >> prior versions of FreeBSD . >> >> I tested this on 9.1-RELEASE and 9.1-STABLE #0 r247804 built last >> monday. >> >> >> >> My question is this. If I enable the direct option in boot/loader.conf >> via >> >> this >> >> >> >> net.isr.direct="1" >> >> net.isr.direct_force="1" >> >> >> >> I do not get any expected result. >> >> >> >> root@chambers:~ # sysctl net.isr.direct >> >> net.isr.direct: 0 >> >> root@chambers:~ # sysctl net.isr.direct_force >> >> net.isr.direct_force: 0 >> >> >> >> root@chambers:~ # netstat -Q >> >> Configuration: >> >> SettingCurrentLimit >> >> Thread count 11 >> >> Default queue limit25610240 >> >> Dispatch policy direct n/a >> >> Threads bound to CPUs disabled n/a >> >> >> >> >> >> >> >> >> >> Am I missing something ? >> >> >> >> >> >> -- >> >> mark saad | nones...@longcount.org >> >> ___ >> >> freebsd-stable@freebsd.org mailing list >> >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> >> To unsubscribe, send any mail to " >> freebsd-stable-unsubscr...@freebsd.org" >> > >> > >> > >> > >> > -- >> > Freddie Cash >> > fjwc...@gmail.com >> >> Freddie >> When I tried to set them in /etc/sysctl.conf , sysctl stated they >> were read-only . >> >> [root@mkr2 /etc]# sysctl -w net.isr.direct=1 >> sysctl: oid 'net.isr.direct' is read only >> >> >> [root@mkr2 /etc]# sysctl -w net.isr.direct_force=1 >> sysctl: oid 'net.isr.direct_force' is read only >> >> -- >> mark saad | nones...@longcount.org >> ___ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" >> > > > > -- > Freddie Cash > fjwc...@gmail.com > -- Freddie Cash fjwc...@gmail.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: netisr issues
You're right. I was looking at different net.isr oids, not the _direct ones. My bad. On Mon, Mar 11, 2013 at 9:17 AM, Mark Saad wrote: > On Mon, Mar 11, 2013 at 12:11 PM, Freddie Cash wrote: > > Works if you set them in /etc/sysctl.conf. Haven't looked into it, but I > > think there's something in the startup that sets them to 0 after the > kernel > > is loaded, so the loader.conf settings are overwritten. > > > > > > On Mon, Mar 11, 2013 at 7:42 AM, Mark Saad > wrote: > >> > >> All > >> I am looking for some guidance on how to turn netisr back on, on a > >> 9.1-RELEASE and 9.1-STABLE box. It looks it stopped working as it did in > >> prior versions of FreeBSD . > >> I tested this on 9.1-RELEASE and 9.1-STABLE #0 r247804 built last > monday. > >> > >> My question is this. If I enable the direct option in boot/loader.conf > via > >> this > >> > >> net.isr.direct="1" > >> net.isr.direct_force="1" > >> > >> I do not get any expected result. > >> > >> root@chambers:~ # sysctl net.isr.direct > >> net.isr.direct: 0 > >> root@chambers:~ # sysctl net.isr.direct_force > >> net.isr.direct_force: 0 > >> > >> root@chambers:~ # netstat -Q > >> Configuration: > >> SettingCurrentLimit > >> Thread count 11 > >> Default queue limit25610240 > >> Dispatch policy direct n/a > >> Threads bound to CPUs disabled n/a > >> > >> > >> > >> > >> Am I missing something ? > >> > >> > >> -- > >> mark saad | nones...@longcount.org > >> ___ > >> freebsd-stable@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable > >> To unsubscribe, send any mail to " > freebsd-stable-unsubscr...@freebsd.org" > > > > > > > > > > -- > > Freddie Cash > > fjwc...@gmail.com > > Freddie > When I tried to set them in /etc/sysctl.conf , sysctl stated they > were read-only . > > [root@mkr2 /etc]# sysctl -w net.isr.direct=1 > sysctl: oid 'net.isr.direct' is read only > > > [root@mkr2 /etc]# sysctl -w net.isr.direct_force=1 > sysctl: oid 'net.isr.direct_force' is read only > > -- > mark saad | nones...@longcount.org > ___ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" > -- Freddie Cash fjwc...@gmail.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: netisr issues
On Mon, Mar 11, 2013 at 12:11 PM, Freddie Cash wrote: > Works if you set them in /etc/sysctl.conf. Haven't looked into it, but I > think there's something in the startup that sets them to 0 after the kernel > is loaded, so the loader.conf settings are overwritten. > > > On Mon, Mar 11, 2013 at 7:42 AM, Mark Saad wrote: >> >> All >> I am looking for some guidance on how to turn netisr back on, on a >> 9.1-RELEASE and 9.1-STABLE box. It looks it stopped working as it did in >> prior versions of FreeBSD . >> I tested this on 9.1-RELEASE and 9.1-STABLE #0 r247804 built last monday. >> >> My question is this. If I enable the direct option in boot/loader.conf via >> this >> >> net.isr.direct="1" >> net.isr.direct_force="1" >> >> I do not get any expected result. >> >> root@chambers:~ # sysctl net.isr.direct >> net.isr.direct: 0 >> root@chambers:~ # sysctl net.isr.direct_force >> net.isr.direct_force: 0 >> >> root@chambers:~ # netstat -Q >> Configuration: >> SettingCurrentLimit >> Thread count 11 >> Default queue limit25610240 >> Dispatch policy direct n/a >> Threads bound to CPUs disabled n/a >> >> >> >> >> Am I missing something ? >> >> >> -- >> mark saad | nones...@longcount.org >> ___ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" > > > > > -- > Freddie Cash > fjwc...@gmail.com Freddie When I tried to set them in /etc/sysctl.conf , sysctl stated they were read-only . [root@mkr2 /etc]# sysctl -w net.isr.direct=1 sysctl: oid 'net.isr.direct' is read only [root@mkr2 /etc]# sysctl -w net.isr.direct_force=1 sysctl: oid 'net.isr.direct_force' is read only -- mark saad | nones...@longcount.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: netisr issues
Works if you set them in /etc/sysctl.conf. Haven't looked into it, but I think there's something in the startup that sets them to 0 after the kernel is loaded, so the loader.conf settings are overwritten. On Mon, Mar 11, 2013 at 7:42 AM, Mark Saad wrote: > All > I am looking for some guidance on how to turn netisr back on, on a > 9.1-RELEASE and 9.1-STABLE box. It looks it stopped working as it did in > prior versions of FreeBSD . > I tested this on 9.1-RELEASE and 9.1-STABLE #0 r247804 built last monday. > > My question is this. If I enable the direct option in boot/loader.conf via > this > > net.isr.direct="1" > net.isr.direct_force="1" > > I do not get any expected result. > > root@chambers:~ # sysctl net.isr.direct > net.isr.direct: 0 > root@chambers:~ # sysctl net.isr.direct_force > net.isr.direct_force: 0 > > root@chambers:~ # netstat -Q > Configuration: > SettingCurrentLimit > Thread count 11 > Default queue limit25610240 > Dispatch policy direct n/a > Threads bound to CPUs disabled n/a > > > > > Am I missing something ? > > > -- > mark saad | nones...@longcount.org > ___ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" > -- Freddie Cash fjwc...@gmail.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
