[9fans] 9vx patches [was: 9vx bootimage build instructions?]
On Thu, 26 May 2011 09:40:31 +0200, yy wrote:
2011/5/26 erik quanstrom quans...@quanstro.net:
9vx uses plan9.ini? last i checked, that assumption was false.
That depends where you checked. Ron's version (or mine, they are the
same now) has some support for plan9.ini files with the -f flag, as
is
documented in the man page:
http://bytebucket.org/yiyus/vx32/wiki/9vx.html. However, although
plan9.ini is used to set some config values (as the root file system
or the memory limit mentioned in this thread), 9vx cannot load a
kernel file. So:
There was a minor inconsistency. There is a comment in 9vx/mmu.c which
states that memsize can be overwritten with the '-m' option, but no -m
switch was provided in main. The following diff adds that (tested):
--- main.c 2011-05-26 08:34:02.687051389 -0500
+++ main.c 2011-05-26 08:34:10.830709411 -0500
@@ -170,6 +170,9 @@
username = EARGF(usage());
break;
+ case 'm':
+ memsize = atoi(EARGF(usage()));
+ break;
default:
usage();
}ARGEND
Also, thanks for the pointer. Setting memsize in a plan9.ini also
works.
i think you need to modify the 9vx kernel directly.
is right.
Modifying the 9vx kernel is not funny because of the bad state of the
.ed files in a/. I hope newer versions of gcc with plan9 extensions
make most of these scripts unnecessary, but as far as I know nobody
has tried.
The following command line switch (-n) adds an initial hack to
conf.nproc to override the 2000 hard coded limit. While this allws me
to now run over 64 threads, running it to high gives me a warning that
there are to many procs for devproc. I'm providing the patch here as
is, but will likely take a poke at it again over the weekend. As a
note, I need to be able to spawn over 3000 procs to stress test some
code. I would like to get this working under 9vx as well...
--- main.c 2011-05-26 08:40:00.286043255 -0500
+++ main.c 2011-05-26 08:43:42.369712636 -0500
@@ -64,6 +64,8 @@
static char* getuser(void);
+int nproclimit = 2000; // allow override of the hardlimit set to the
number of procs
+
void
usage(void)
{
@@ -173,6 +175,9 @@
case 'm':
memsize = atoi(EARGF(usage()));
break;
+ case 'n':
+ nproclimit = atoi(EARGF(usage()));
+ break;
default:
usage();
}ARGEND
@@ -284,8 +289,8 @@
conf.npage += conf.mem[i].npage;
conf.upages = conf.npage;
conf.nproc = 100 + ((conf.npage*BY2PG)/MB)*5;
- if(conf.nproc 2000)
- conf.nproc = 2000;
+ if(conf.nproc nproclimit)
+ conf.nproc = nproclimit;
conf.nimage = 200;
conf.nswap = 0;
conf.nswppo = 0;
I do not think I have write access to your 9vx repository, and I do not
know if you want the nproc limit hack. Let me know if you find these
acceptable.
EBo --
Re: [9fans] 9vx patches [was: 9vx bootimage build instructions?]
2011/5/26 EBo e...@sandien.com:
On Thu, 26 May 2011 09:40:31 +0200, yy wrote:
2011/5/26 erik quanstrom quans...@quanstro.net:
9vx uses plan9.ini? last i checked, that assumption was false.
That depends where you checked. Ron's version (or mine, they are the
same now) has some support for plan9.ini files with the -f flag, as is
documented in the man page:
http://bytebucket.org/yiyus/vx32/wiki/9vx.html. However, although
plan9.ini is used to set some config values (as the root file system
or the memory limit mentioned in this thread), 9vx cannot load a
kernel file. So:
There was a minor inconsistency. There is a comment in 9vx/mmu.c which
states that memsize can be overwritten with the '-m' option, but no -m
switch was provided in main. The following diff adds that (tested):
Yeah, I added this option years ago but it must have been lost in some
of the SoC work plus merges at some point.
--- main.c 2011-05-26 08:34:02.687051389 -0500
+++ main.c 2011-05-26 08:34:10.830709411 -0500
@@ -170,6 +170,9 @@
username = EARGF(usage());
break;
+ case 'm':
+ memsize = atoi(EARGF(usage()));
+ break;
default:
usage();
}ARGEND
Also, thanks for the pointer. Setting memsize in a plan9.ini also works.
Or maybe yiyus added it here in an attempt to get things out of CLI
opts when the plan9.ini stuff happened.
i think you need to modify the 9vx kernel directly.
is right.
Modifying the 9vx kernel is not funny because of the bad state of the
.ed files in a/. I hope newer versions of gcc with plan9 extensions
make most of these scripts unnecessary, but as far as I know nobody
has tried.
The following command line switch (-n) adds an initial hack to conf.nproc to
override the 2000 hard coded limit. While this allws me to now run over 64
threads, running it to high gives me a warning that there are to many procs
for devproc. I'm providing the patch here as is, but will likely take a
poke at it again over the weekend. As a note, I need to be able to spawn
over 3000 procs to stress test some code. I would like to get this working
under 9vx as well...
--- main.c 2011-05-26 08:40:00.286043255 -0500
+++ main.c 2011-05-26 08:43:42.369712636 -0500
@@ -64,6 +64,8 @@
static char* getuser(void);
+int nproclimit = 2000; // allow override of the hardlimit set to the number
of procs
+
void
usage(void)
{
@@ -173,6 +175,9 @@
case 'm':
memsize = atoi(EARGF(usage()));
break;
+ case 'n':
+ nproclimit = atoi(EARGF(usage()));
+ break;
default:
usage();
}ARGEND
@@ -284,8 +289,8 @@
conf.npage += conf.mem[i].npage;
conf.upages = conf.npage;
conf.nproc = 100 + ((conf.npage*BY2PG)/MB)*5;
- if(conf.nproc 2000)
- conf.nproc = 2000;
+ if(conf.nproc nproclimit)
+ conf.nproc = nproclimit;
conf.nimage = 200;
conf.nswap = 0;
conf.nswppo = 0;
I do not think I have write access to your 9vx repository, and I do not know
if you want the nproc limit hack. Let me know if you find these acceptable.
Typically the way to do this is to create your own public fork, and
then send a pull request to the maintainer of whoever you forked from
since hg has the distributed model.
EBo --
Re: [9fans] 9vx patches [was: 9vx bootimage build instructions?]
The following command line switch (-n) adds an initial hack to
conf.nproc to override the 2000 hard coded limit. While this allws me
to now run over 64 threads, running it to high gives me a warning that
there are to many procs for devproc. I'm providing the patch here as
this is a bug in the warning message.
; diffy -c devproc.c
/n/dump/2011/0526/sys/src/9/port/devproc.c:139,147 - devproc.c:139,149
* If notepg, c-pgrpid.path is pgrp slot, .vers is noteid.
*/
#define QSHIFT 5 /* location in qid of proc slot # */
+ #define SLOTBITS 23 /* number of bits in the slot */
+ #define SLOTMASK((1SLOTBITS)-1 QSHIFT)
#define QID(q) ulong)(q).path)0x001F)0)
- #define SLOT(q) (ulong)(q).path)0x07FE0)QSHIFT)-1)
+ #define SLOT(q) (ulong)(q).path)SLOTMASK)QSHIFT)-1)
#define PID(q) ((q).vers)
#define NOTEID(q) ((q).vers)
/n/dump/2011/0526/sys/src/9/port/devproc.c:288,294 - devproc.c:290,296
static void
procinit(void)
{
- if(conf.nproc = (1(16-QSHIFT))-1)
+ if(conf.nproc = (SLOTMASKQSHIFT) - 1)
print(warning: too many procs for devproc\n);
addclock0link((void (*)(void))profclock, 113); /* Relative prime to HZ
*/
}
- erik
Re: [9fans] 9vx patches [was: 9vx bootimage build instructions?]
On Thu, 26 May 2011 11:34:29 -0400, erik quanstrom wrote: ... it to high gives me a warning that there are to many procs for devproc... this is a bug in the warning message. ; diffy -c devproc.c ... + #define SLOTBITS 23 /* number of bits in the slot */ + #define SLOTMASK((1SLOTBITS)-1 QSHIFT) ... Thanks, I have added this patch to my ebuilds. The system still crashes with very large numbers of procs (I think it is overrunning the static definition of libvx32/proc.c vxproc *procs[VXPROCSMAX]), but for now spawning 64 tasks is ok for development, but all the tests are shooting for 512. It would be nice to get all the systems running consistently. Any suggestions are greatly appreciated. EBo --
Re: [9fans] 9vx patches [was: 9vx bootimage build instructions?]
The system still crashes with very large numbers of procs (I think it
is overrunning the static definition of libvx32/proc.c vxproc
*procs[VXPROCSMAX]), but for now spawning 64 tasks is ok for
development, but all the tests are shooting for 512. It would be nice
to get all the systems running consistently. Any suggestions are
greatly appreciated.
unless someone has broken 9vx since i last pulled, the code has
that case handled.
for (pno = 0; ; pno++) {
if (pno == VXPROCSMAX) {
errno = EAGAIN;
return NULL;
}
if (procs[pno] == 0)
break;
}
if you follow vxproc_alloc calls, you'll see that each
caller checks the return value and does something reasonable.
given this,
main.c:318: conf.nproc = 100 + ((conf.npage*BY2PG)/MB)*5;
main.c:319: if(conf.nproc 2000)
main.c:320: conf.nproc = 2000;
i'd suspect that the real problem is you're not giving 9vx enough
kernel memory. try setting kernelpercent to something largeish.
but of course, i'm just guessing. it would be helpful to have some
debugging output.
- erik
Re: [9fans] 9vx patches [was: 9vx bootimage build instructions?]
On Thu, 26 May 2011 14:58:11 -0400, erik quanstrom wrote: The system still crashes with very large numbers of procs (I think it is overrunning the static definition of libvx32/proc.c vxproc *procs[VXPROCSMAX]), but for now spawning 64 tasks is ok for development, but all the tests are shooting for 512. It would be nice to get all the systems running consistently. Any suggestions are greatly appreciated. ... main.c:318: conf.nproc = 100 + ((conf.npage*BY2PG)/MB)*5; main.c:319: if(conf.nproc 2000) main.c:320: conf.nproc = 2000; The problem here is that I had to override this test to pass in a limit. On plan9 (running in kvm) it took over 3000 procs to run the 512 tasks, and I set the memsize to 2047 (it crashed with 2048). The behavior is interesting. It is moroting along and either aborts with a panic: sigsegv on cpu5, a panic: mmap address space 0, or it goes off into never never land (if I look at the CPU usage at this time it is essentially 0 where throwing 64 tasks at it causes one of the cores to go full tilt). i'd suspect that the real problem is you're not giving 9vx enough kernel memory. try setting kernelpercent to something largeish. I've played around with setting it from 70 to 100. but of course, i'm just guessing. it would be helpful to have some debugging output. I'll see what I can get you after I put out the immediate fires. Thanks for the suggestions. EBo --
[9fans] 9vx Patches
Hey guys, Back in my ``let's have fun with Plan 9'' yearly phase. Here I come back and I see some really cool work. 9vx definitely is the coolest thing I've seen so far, and devtrace looks pretty nifty too. What's really cool to me about them is that they're available and there is open work to be done with them. So I've started hacking away. My 9vx has a couple changes that I found useful / thought would be fun to implement / thought others might find useful. A brief synopsis: * Memory bounds -- gives 9vx an upper bound on resident memory usage as it limits the size of the pagefile. Set by running 9vx with -m [MB] * CPU bounds -- only works on FreeBSD so far. I've got a Linux port of that management code, but it doesn't seem to actually do the right thing (and it hasn't been a priority to fix since I rarely run Linux). I do plan on fixing it, but if someone else wants to fix it first, that'd be fine by me, too. Darwin support looks unlikely because, although the libkvm stuff is there and likely would work after just changing some of the member names of where things look, it has neither a clone(2) or rfork(2) implementation. If anybody has an idea about how to spawn and disassociate a child process in Darwin so it doesn't receive its parents' signals, let me know. Set by running 9vx with -l [% CPU]. * Set window width/height on the console. Set with -w [width] and -h [height], respectively. * Lock tracing. I needed this when debugging some of the netstack stuff because I screwed up a qunlock which deadlocked the system. Set by toggling -L. Things in the works: * Native Plan 9 netstack. It's ported, it compiles, but my virtual ethernet device doesn't seem to be transmitting anything (though I haven't done any extensive tests). See also: http://testbed.dh0.us/~dho/fuxyes.png * Porting devtrace. Needs some changes to compile with gas; the #pragmas are useless, but I think should work after a little more tweaking. * FreeBSD/amd64 support. I don't think it works yet, but it compiles. I don't rightly understand what all the LDT modification does on Linux x86_64 in relation to what I'm actually doing with %gs (hints from people with knowledge welcome). Other stuff * Automatic provisioning via a web browser. Yeah, Plan 9 in the web browser (if you have Java, I guess -- otherwise it's just VNC). http://testbed.dh0.us/ -- this code isn't version controlled, but I could probably put it in hg with little effort. The code for the rest of the stuff is at http://testbed.dh0.us:8000/ --dho
Re: [9fans] 9vx Patches
so here's a potentially interesting idea. Since you are running plan 9 under Linux with 9vx, consider using the TAU toolkit to measure it. http://www.cs.uoregon.edu/research/tau/home.php we've used these tools to optmize an MPI library and they are quite powerful. See what you think. ron
Re: [9fans] 9vx Patches
On Wed, Dec 17, 2008 at 4:10 PM, Devon H. O'Dell devon.od...@gmail.com wrote: * Porting devtrace. Needs some changes to compile with gas; the #pragmas are useless, but I think should work after a little more tweaking. note that those pragmas are the tip of the iceberg. gcc inserts the profiling code; on Plan 9, the linker does it. There will be a few things to fix here. neat stuff you are doing. ron
Re: [9fans] 9vx Patches
On Dec 17, 2008, at 4:33 PM, ron minnich wrote: so here's a potentially interesting idea. Since you are running plan 9 under Linux with 9vx, consider using the TAU toolkit to measure it. http://www.cs.uoregon.edu/research/tau/home.php we've used these tools to optmize an MPI library and they are quite powerful. TAU is good. There's also a Sun Studio Performance Analyzer that, to my taste, is a bit more tightly integrated than TAU and seems to be more scalable: http://developers.sun.com/sunstudio/overview/topics/analyzer_index.html In fact, Ron, now that you've mentioned it -- I'm going to try the PerfAn on 9vx myself ;-) And here's the question for you: how representative the behavior of 9vx is supposed to be of the regular Plan9 kernel? Thanks, Roman.
Re: [9fans] 9vx Patches
On Wed, Dec 17, 2008 at 4:43 PM, Roman Shaposhnik r...@sun.com wrote: In fact, Ron, now that you've mentioned it -- I'm going to try the PerfAn on 9vx myself ;-) And here's the question for you: how representative the behavior of 9vx is supposed to be of the regular Plan9 kernel? I don't know but the code sure looked like a plan 9 kernel to me. ron
Re: [9fans] 9vx Patches
On Dec 17, 2008, at 4:10 PM, Devon H. O'Dell wrote: * Automatic provisioning via a web browser. Yeah, Plan 9 in the web browser (if you have Java, I guess -- otherwise it's just VNC). http://testbed.dh0.us/ -- this code isn't version controlled, but I could probably put it in hg with little effort. Really nice bunch of things! I'm looking forward to seeing the implementations. As for the above service -- is it currently off-line? I can log-in and create an instance, but when I try to connect to it here's what I get: snazvx: dashboard control id namestate activateview 10 rvs-testrunning stopview (or vncviewer testbed.dh0.us:5810) vncviewer: ConnectToTcpAddr: connect: Connection refused Unable to connect to VNC server Thanks, Roman. P.S. Has anybody experimented with running 9vx on Amazon's EC2 yet?
Re: [9fans] 9vx Patches
2008/12/17 Roman Shaposhnik r...@sun.com: On Dec 17, 2008, at 4:10 PM, Devon H. O'Dell wrote: * Automatic provisioning via a web browser. Yeah, Plan 9 in the web browser (if you have Java, I guess -- otherwise it's just VNC). http://testbed.dh0.us/ -- this code isn't version controlled, but I could probably put it in hg with little effort. Really nice bunch of things! I'm looking forward to seeing the implementations. As for the above service -- is it currently off-line? I can log-in and create an instance, but when I try to connect to it here's what I get: Thanks! The 9vx that is installed by this though is the one from Russ's tip... mine's still not perfect and currently doesn't do any networking. snazvx: dashboard control id namestate activateview 10 rvs-testrunning stopview (or vncviewer testbed.dh0.us:5810) The vncviewer text is wrong; it's just vncviewer testbed.dh0.us:10 -- I haven't fixed this yet :( vncviewer: ConnectToTcpAddr: connect: Connection refused Unable to connect to VNC server Thanks, Roman. --dho P.S. Has anybody experimented with running 9vx on Amazon's EC2 yet? I would if it was free and I didn't have a server :\ --dho
