Re: 8.0 performance issue when running build.sh?
On Thu, 12 Jul 2018, Thor Lancelot Simon wrote: On Thu, Jul 12, 2018 at 07:39:10PM +0200, Martin Husemann wrote: On Thu, Jul 12, 2018 at 12:30:39PM -0400, Thor Lancelot Simon wrote: Are you running the builds from tmpfs to tmpfs, like the build cluster does? No, I don't have enough ram to test it that way. So if 8.0 has a serious tmpfs regression... we don't yet know. I have a system with (probably) enough RAM - amd64, 8/16 core/threads, with 128GB, running 8.99.18 - to test if someone wants to provide an explicit test scenario that can run on top of an existing "production" environment. +--+--++ | Paul Goyette | PGP Key fingerprint: | E-mail addresses: | | (Retired)| FA29 0E3B 35AF E8AE 6651 | paul at whooppee dot com | | Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd dot org | +--+--++
Re: 8.0 performance issue when running build.sh?
Date:Thu, 12 Jul 2018 15:18:08 -0400 From:Thor Lancelot Simon Message-ID: <20180712191808.ga2...@panix.com> | So if 8.0 has a serious tmpfs regression... we don't yet know. If the most serious problem is something in the basic infrastructure then it ought to be affecting all of the builds. But it is only really serious on the netbsd-7(-*) builds, -6 -8 and HEAD are not affected (or not nearly as much). (That is, the -7 builds end up taking about twice as long as HEAD and -8 builds, and much longer than -6 (which is smaller, and uses a much simpler gcc)) I believe the most likely cause is something peculiar to -7, when it runs on a -8 base, itself, perhaps related to the gcc version, or make, of binutilos, or ... and might be something that only affects some of the architectures (I don't have enough info to know). It could be almost anything given that scope limitation, but I don't see how something basic like tmpfs on -8 can be a possible cause. kre
Re: 8.0 performance issue when running build.sh?
On Thu, Jul 12, 2018 at 07:39:10PM +0200, Martin Husemann wrote: > On Thu, Jul 12, 2018 at 12:30:39PM -0400, Thor Lancelot Simon wrote: > > Are you running the builds from tmpfs to tmpfs, like the build cluster > > does? > > No, I don't have enough ram to test it that way. So if 8.0 has a serious tmpfs regression... we don't yet know. -- Thor Lancelot Simont...@panix.com "The two most common variations translate as follows: illegitimi non carborundum = the unlawful are not silicon carbide illegitimis non carborundum = the unlawful don't have silicon carbide."
Re: 8.0 performance issue when running build.sh?
On Thu, Jul 12, 2018 at 12:30:39PM -0400, Thor Lancelot Simon wrote: > Are you running the builds from tmpfs to tmpfs, like the build cluster > does? No, I don't have enough ram to test it that way. Martin
Re: 8.0 performance issue when running build.sh?
On Thu, Jul 12, 2018 at 05:29:57PM +0200, Martin Husemann wrote: > On Tue, Jul 10, 2018 at 11:01:01AM +0200, Martin Husemann wrote: > > So stay tuned, maybe only Intel to blame ;-) > > Nope, that wasn't it. > > I failed to reproduce *any* slowdown on an AMD CPU locally, maya@ kindly > repeated the same test on an affected Intel CPU and also could see no > slowdown. Are you running the builds from tmpfs to tmpfs, like the build cluster does? Thor
Re: 8.0 performance issue when running build.sh?
On Tue, Jul 10, 2018 at 11:01:01AM +0200, Martin Husemann wrote: > So stay tuned, maybe only Intel to blame ;-) Nope, that wasn't it. I failed to reproduce *any* slowdown on an AMD CPU locally, maya@ kindly repeated the same test on an affected Intel CPU and also could see no slowdown. So this seems to not be a generic NetBSD 8.0 issue, but something strange with the concrete hardware (maybe a driver bug). It is never easy - but at least this should not block the remaining 8.0 release cycle. Martin
Re: usb/xhci lock issue on HEAD
On Thursday 12 July 2018 00:54:45 Martin Husemann wrote: > You commented out a bit too much and it does not find your boot device? > > /* > * If wildcarded root and we the boot device wasn't determined, > * ask the user. > */ > if (rootspec == NULL && bootdv == NULL) > boothowto |= RB_ASKNAME; > > (kern_subr.c:257) > > You normally would see something like: > > [..] > boot device: wd0 > root on wd0a dumps on wd0b > root file system type: ffs > > but if it can not identify the device, it tries to ask you for it (and > runs into an unrelated usb bug). > No, I didn't comment out anything to do with the boot and root devices. It is true I ran into an unrelated usb bug, but there must also be a bug somewhere where the bootdv ends up NULL. I got past this problem by fixing the root device in the kernel config. The problems are the same on my wifi branch and head. So this is not my bug. The kernel (from both branches) says: boot device: I copied GENERIC and then removed a bunch of drivers and changed options, but left in all the 802.11 devices. It works and I have been running it for a while. I then removed all the 802.11 devices except urtwn and it can't find the boot device. That is the only difference between the working kernel and the kernel that doesn't know about the boot device. This is on HEAD. --Phil
Re: Too many PMC implementations
On 12.07.2018 08:48, Maxime Villard wrote: > Le 11/07/2018 à 19:49, Kamil Rytarowski a écrit : >> I'm not familiar with the internals myself, but from API point of view, >> something usable for porting rr (https://github.com/mozilla/rr) or even >> Linux perf-top is highly desirable. I treat personally perf-top as a >> gold standard. > > Well, yes, but right now let's first try to have functional internals... I fully understand and appreciate the option to garbage/collect redundant implementations. From a fuzzing point of view, we are researching during GSoC honggfuzz (vs libFuzzer) and it can use aid from performance counters: https://github.com/google/honggfuzz/blob/master/docs/FeedbackDrivenFuzzing.md signature.asc Description: OpenPGP digital signature
Re: Too many PMC implementations
Le 11/07/2018 à 18:22, Maxime Villard a écrit : Right now we have three (or more?) different implementations for Performance Monitoring Counters: * PMC: this one is MI. It is used only on one ARM model (xscale I think). There used to be an x86 code for it, but it was broken, and I removed it. The implementation comes with libpmc, a library we provide. The code hasn't moved these last 15 years. I don't like this implementation, it is really invasive (see the numerous pmc.h files that are all empty). * X86PMC: this one is MD, and only available for x86. I wrote it myself. The code is small (x86/pmc.c), and functional. The PMCs are system-wide, and retrieved on a per-cpu basis. But this implementation does not support tracking, that is, we get numbers (about the cache misses for example), but we don't know where they happened. * TPROF: this one is MI, but only x86 support is present. TPROF provides the backend needed to support tracking: via a device, that userland can read from, in order to absorb the event samples produced by the kernel. The backend is pretty good, but the frontend (where the user chooses which PMC etc) is inexistent - the CPU/event detection is not there either. The backend is MI (/dev/tprof/tprof.c), and can be used on other architectures. The module already exists to dynamically modload. I think it would be good to: * Remove PMC entirely. Then remove libpmc too. * Merge X86PMC into the x86 part of TPROF. That is to say, into x86/tprof_*. Then remove X86PMC. * Later, maybe, someone will want to add other architectures in TPROF, like all the recent ARMs. Maxime So, I've prepared a patch. It removes "options PERFCTRS", all the pmc.h files, the kernel sys_pmc.c, the man pages, and the PMC code of ARM XSCALE. Other ARMs have their own small PMC code, but it is used in the MI code, and not from the outside. These ones are obviously not removed. The x86 code is reordered not to rely on the legacy pmc.h file (which I recycled to put the definitions for X86PMC). Will commit soon...
Re: usb/xhci lock issue on HEAD
On Wed, Jul 11, 2018 at 09:34:20PM -0700, Phil Nelson wrote: > I'm not sure why this kernel is calling cngetsn() at setroot() time. You commented out a bit too much and it does not find your boot device? /* * If wildcarded root and we the boot device wasn't determined, * ask the user. */ if (rootspec == NULL && bootdv == NULL) boothowto |= RB_ASKNAME; (kern_subr.c:257) You normally would see something like: [..] boot device: wd0 root on wd0a dumps on wd0b root file system type: ffs but if it can not identify the device, it tries to ask you for it (and runs into an unrelated usb bug). Martin
