Re: asymmetric smp
On Mon, 5 May 2014 01:10:24 -0400 Matthew Mondor wrote: > which some CPUs might have trouble with (i.e. RAS)... I think that what I meant was CAS -- Matt
Re: asymmetric smp
On Wed, 02 Apr 2014 17:21:02 +0200 Johnny Billquist wrote: > On 2014-04-02 16:10, John Nemeth wrote: > > On Apr 2, 1:55pm, Johnny Billquist wrote: > > } The root fs in on nfs, as I'm running the machine diskless. Disk is > > } served from a -current NetBSD/alpha system sitting right next to it. And > > } I have changed the Alpha to run at 10 MB/s half duplex, and I have 2k > > } block size for NFS. Login is obviously already running, since that is > > } what also prompts for the username, and doing it twice should even put > > } some stuff in local cache. > > > > Uh, actually getty does the initial prompt for username on > > the console. After collecting the username, getty execs login. > > Hmm. My mistake in that case. So we have image activation at that point. > Hmm... Possibly other things to verify would be /etc/passwd.conf (you'll likely need to also regenerate passwords if you change those settings), and if VAX has specialized lock code or uses the new generic atomic operations which some CPUs might have trouble with (i.e. RAS)... -- Matt
Re: asymmetric smp
On 2014-04-02 16:10, John Nemeth wrote: On Apr 2, 1:55pm, Johnny Billquist wrote: } The root fs in on nfs, as I'm running the machine diskless. Disk is } served from a -current NetBSD/alpha system sitting right next to it. And } I have changed the Alpha to run at 10 MB/s half duplex, and I have 2k } block size for NFS. Login is obviously already running, since that is } what also prompts for the username, and doing it twice should even put } some stuff in local cache. Uh, actually getty does the initial prompt for username on the console. After collecting the username, getty execs login. Hmm. My mistake in that case. So we have image activation at that point. Hmm... Thanks for pointing it out. Johnny
Re: asymmetric smp
On 2014-04-02 15:38, Anders Magnusson wrote: Martin Husemann skrev 2014-04-02 15:33: On Wed, Apr 02, 2014 at 03:13:19PM +0200, Johnny Billquist wrote: What model of VAX do you have, and how long does it take to boot, to the point where you get the login prompt on the console? VS4000/M96 with 128 MB, and local scsi disk - very nice machine ;-) Didn't measure exactly right now, but on the order of 30s. Heh, that machines is like 30 times faster than Johnny's :-) Indeed. You can't get much faster VAXen than that. If it takes about 3s from pressing enter after the username until the password prompt appears, then 30s on this 3500 seems very reasonable. Johnny
Re: asymmetric smp
On Apr 2, 1:55pm, Johnny Billquist wrote: } On 2014-04-01 23:04, Warner Losh wrote: } > On Apr 1, 2014, at 5:49 AM, Johnny Billquist wrote: } > } >> Good points. } >> Is this the right time to ask why booting NetBSD on a VAX (a 3500) now takes more than 15 minutes? What is the system doing all that time??? } > } > FreeBSD used to take forever to boot on certain low-end ARM CPUs with /etc/rc.d after it was imported from NetBSD. This was due to crappy root-device performance (100kB/s is enough for anybody, right?) and crappy, at the time, pmap code that caused excess page traffic in the /etc/rc.d environment. Perhaps those areas would be fruitful to profile? Also, there were some inefficiencies that were either the result of a botched port, or were basic to the system that got fixed. Between fixing all these things, the boot time went from 10 minutes down to ~20s. } } Always nice with some ideas. The problem here is that this used to be } way faster in the past, but have slowed down recently. } } The time between entering a username and getting the password prompt in } the same 3500 with the latest release is something like 30 seconds. } } This is on an otherwise idle system, where boot has completed. 30 } seconds (approximately, I should time it) just from pressing enter after } the username, until I just get the "Password:" prompt seems incredible } to me. } } The root fs in on nfs, as I'm running the machine diskless. Disk is } served from a -current NetBSD/alpha system sitting right next to it. And } I have changed the Alpha to run at 10 MB/s half duplex, and I have 2k } block size for NFS. Login is obviously already running, since that is } what also prompts for the username, and doing it twice should even put } some stuff in local cache. Uh, actually getty does the initial prompt for username on the console. After collecting the username, getty execs login. }-- End of excerpt from Johnny Billquist
Re: asymmetric smp
Martin Husemann skrev 2014-04-02 15:33: On Wed, Apr 02, 2014 at 03:13:19PM +0200, Johnny Billquist wrote: What model of VAX do you have, and how long does it take to boot, to the point where you get the login prompt on the console? VS4000/M96 with 128 MB, and local scsi disk - very nice machine ;-) Didn't measure exactly right now, but on the order of 30s. Heh, that machines is like 30 times faster than Johnny's :-) -- R
Re: asymmetric smp
On Wed, Apr 02, 2014 at 03:13:19PM +0200, Johnny Billquist wrote: > What model of VAX do you have, and how long does it take to boot, to the > point where you get the login prompt on the console? VS4000/M96 with 128 MB, and local scsi disk - very nice machine ;-) Didn't measure exactly right now, but on the order of 30s. Martin
Re: asymmetric smp
On 2014-04-02 14:00, Martin Husemann wrote: On Wed, Apr 02, 2014 at 01:55:04PM +0200, Johnny Billquist wrote: The time between entering a username and getting the password prompt in the same 3500 with the latest release is something like 30 seconds. Mostly it needs someone with an affected machine to debug it. My bet would be on PAM being related to this issue. (FWIW, it takes < 3 seconds on my vax) Hmm. This is with a very default installation that I have not even started trying to customize. But thanks, PAM is definitely an interesting point. I should look. And even more thanks in saying that it's not the same for you. That implies that there is something bad in the config, and not something totally generic. What model of VAX do you have, and how long does it take to boot, to the point where you get the login prompt on the console? Johnny
Re: asymmetric smp
On Wed, Apr 02, 2014 at 01:55:04PM +0200, Johnny Billquist wrote: > The time between entering a username and getting the password prompt in > the same 3500 with the latest release is something like 30 seconds. Mostly it needs someone with an affected machine to debug it. My bet would be on PAM being related to this issue. (FWIW, it takes < 3 seconds on my vax) Martin
Re: asymmetric smp
On 2014-04-01 23:04, Warner Losh wrote: On Apr 1, 2014, at 5:49 AM, Johnny Billquist wrote: Good points. Is this the right time to ask why booting NetBSD on a VAX (a 3500) now takes more than 15 minutes? What is the system doing all that time??? FreeBSD used to take forever to boot on certain low-end ARM CPUs with /etc/rc.d after it was imported from NetBSD. This was due to crappy root-device performance (100kB/s is enough for anybody, right?) and crappy, at the time, pmap code that caused excess page traffic in the /etc/rc.d environment. Perhaps those areas would be fruitful to profile? Also, there were some inefficiencies that were either the result of a botched port, or were basic to the system that got fixed. Between fixing all these things, the boot time went from 10 minutes down to ~20s. Always nice with some ideas. The problem here is that this used to be way faster in the past, but have slowed down recently. The time between entering a username and getting the password prompt in the same 3500 with the latest release is something like 30 seconds. This is on an otherwise idle system, where boot has completed. 30 seconds (approximately, I should time it) just from pressing enter after the username, until I just get the "Password:" prompt seems incredible to me. The root fs in on nfs, as I'm running the machine diskless. Disk is served from a -current NetBSD/alpha system sitting right next to it. And I have changed the Alpha to run at 10 MB/s half duplex, and I have 2k block size for NFS. Login is obviously already running, since that is what also prompts for the username, and doing it twice should even put some stuff in local cache. The 3500 have 16 megs of memory, and looking at the system once logged in, I'm not using much CPU, nor is all memory committed yet. I have (on and off) complained about perceived slowing down of NetBSD/vax over the years, but it's been a little while since I last tried, and my experience now on the 3500 is so horrible that I'd say it is very close to impossible to run NetBSD on it anymore. And actually, the Alpha is pretty saggy with -current as well. No idea why, but occasionally it spends a whole lot of time in system mode. Might be related. Or not... Johnny
Re: asymmetric smp
On Apr 1, 2014, at 5:49 AM, Johnny Billquist wrote: > Good points. > Is this the right time to ask why booting NetBSD on a VAX (a 3500) now takes > more than 15 minutes? What is the system doing all that time??? FreeBSD used to take forever to boot on certain low-end ARM CPUs with /etc/rc.d after it was imported from NetBSD. This was due to crappy root-device performance (100kB/s is enough for anybody, right?) and crappy, at the time, pmap code that caused excess page traffic in the /etc/rc.d environment. Perhaps those areas would be fruitful to profile? Also, there were some inefficiencies that were either the result of a botched port, or were basic to the system that got fixed. Between fixing all these things, the boot time went from 10 minutes down to ~20s. Warner
Re: asymmetric smp
On 2014-04-01 01:47, Erik Fair wrote: On Mar 27, 2014, at 11:29 , matthew green wrote: it certainly can be improved for this situation, but i've got an SS10 with mis-matched cpus (2x100mhz, 1x150mhz, the latter with a bigger cache and thus significantly faster than the other cpus) and it works pretty fine. so - my guess is that some of the power and performance issues will still need work, but the system will boot and run normally without special work. I believe the point is to more nearly optimally schedule processes according to their needs on the processor(s) whose characteristics better match. Mr. Thomas also noted that this has energy use (power budget) implications. Apple started a push in the "energy efficient software" direction last year at WWDC, clearly to prod third-party software developers into ligher impact on the energy budgets of both iOS mobile devices and MacOS X based laptops, but this sort of thing also has implications for HVAC and power bills in large scale data centers too. As I have been wont to say for a decade or two, "the hardware giveth and the software taketh away" and it would be a good thing for NetBSD's ongoing relevance if we paid some attention to efficiency of our software's algorithms and execution efficiency with the explicit goal of minimizing energy use of the computers our system runs on. There are limits to what we can do; if users choose to run inefficient applications, or combine the software we provide in less-than-optimal ways with shell scripts, that's on them - we've (hopefully) done all we can do to provide them with the best tools. Good points. Is this the right time to ask why booting NetBSD on a VAX (a 3500) now takes more than 15 minutes? What is the system doing all that time??? Johnny
Re: asymmetric smp
On Mar 27, 2014, at 11:29 , matthew green wrote: > it certainly can be improved for this situation, but i've > got an SS10 with mis-matched cpus (2x100mhz, 1x150mhz, > the latter with a bigger cache and thus significantly > faster than the other cpus) and it works pretty fine. > > so - my guess is that some of the power and performance > issues will still need work, but the system will boot > and run normally without special work. I believe the point is to more nearly optimally schedule processes according to their needs on the processor(s) whose characteristics better match. Mr. Thomas also noted that this has energy use (power budget) implications. Apple started a push in the "energy efficient software" direction last year at WWDC, clearly to prod third-party software developers into ligher impact on the energy budgets of both iOS mobile devices and MacOS X based laptops, but this sort of thing also has implications for HVAC and power bills in large scale data centers too. As I have been wont to say for a decade or two, "the hardware giveth and the software taketh away" and it would be a good thing for NetBSD's ongoing relevance if we paid some attention to efficiency of our software's algorithms and execution efficiency with the explicit goal of minimizing energy use of the computers our system runs on. There are limits to what we can do; if users choose to run inefficient applications, or combine the software we provide in less-than-optimal ways with shell scripts, that's on them - we've (hopefully) done all we can do to provide them with the best tools. Erik
re: asymmetric smp
> However, it has a quirk that I don't think our scheduler will deal with. > > It has 4 Cortex-A15 cores @ 1.4Ghz and 4 Cortex-A7 cores @ 1.2Ghz. Even > if the frequencies weren't different, the A15 cores at least twice as > fast per cycle than the A7. That asymmetry is going to cause havoc with > the scheduler. In terms of power, the A7s use a lot less than the A15s why do you think the scheduler will have a major problem? it certainly can be improved for this situation, but i've got an SS10 with mis-matched cpus (2x100mhz, 1x150mhz, the latter with a bigger cache and thus significantly faster than the other cpus) and it works pretty fine. so - my guess is that some of the power and performance issues will still need work, but the system will boot and run normally without special work. .mrg.
Re: asymmetric smp
On Mar 27, 2014 2:32 AM, "Matt Thomas" wrote: > > > I recently ordered an ODROID-XU Lite to help beat on the my ARM MP code. > > However, it has a quirk that I don't think our scheduler will deal with. > > It has 4 Cortex-A15 cores @ 1.4Ghz and 4 Cortex-A7 cores @ 1.2Ghz. Even if the frequencies weren't different, the A15 cores at least twice as fast per cycle than the A7. That asymmetry is going to cause havoc with the scheduler. In terms of power, the A7s use a lot less than the A15s so if you can keep the work on the A7s and leave the A15s sleeping, you'll extend your battery life a lot. Yes these things are odd. Haven't got one yet. I did see that GCC now has a dual optimization mode for code that nerds to run on both. There are a few research documents around on what works for scheduling. Justin
Re: asymmetric smp
On Wed, Mar 26, 2014 at 07:32:24PM -0700, Matt Thomas wrote: > > I recently ordered an ODROID-XU Lite to help beat on the my ARM MP code. Ah... for a second I thought you want to add support for the VAX 11/782. -is
asymmetric smp
I recently ordered an ODROID-XU Lite to help beat on the my ARM MP code. However, it has a quirk that I don't think our scheduler will deal with. It has 4 Cortex-A15 cores @ 1.4Ghz and 4 Cortex-A7 cores @ 1.2Ghz. Even if the frequencies weren't different, the A15 cores at least twice as fast per cycle than the A7. That asymmetry is going to cause havoc with the scheduler. In terms of power, the A7s use a lot less than the A15s so if you can keep the work on the A7s and leave the A15s sleeping, you'll extend your battery life a lot. It should also be noted the A15s have different cache structure than the A7s as well. There is no hyperthreading so that headache. Imagine doing a build, you might want to keep the shells and the like on the A7 while letting A15s get the compiler/assembler/loader (more compute intensive).
