Re: [9fans] Plan 9 image for Google Compute Engine?
On Wed, May 28, 2014 at 12:34 PM, Aram Hăvărneanu wrote: > Plan 9 doesn't have virtio ethernet driver (except mischief's, which I > don't think is ready for public consumption yet). You will also need a virtio-scsi driver; GCE does not offer virtio-blk. But it should work just fine when both drivers are available. -- vs;
Re: [9fans] 3e
How was 3E different from 2E and from 4E? -- vs;
Re: [9fans] 9atom-nix
> the pae kernel deserves a word or so. the pae kernel brings > with it a streamlined kernel, supporting only very modern > machines, such as the pentium pro. If it requires PAE, it will not run on Banias and Dothan B-stepping and C-stepping (on 400MHz FSB) CPUs (most Pentium M's).
Re: [9fans] tcl, 9p
Hi, > Also, this is somewhat unrelated, but I was wondering whether each Go > executable contains the garbage collector. (It must, it seems, but just > checking). It does. -- vs
Re: [9fans] OT: how do 9fans backup their Mac(book)?
When my home directory is less than 2 gigabytes in size, I use dump9660 from Plan 9 port (or a standalone relative). Otherwise I rsync my home to a DragonFly BSD system and take a HAMMER snapshot. -- vs
Re: [9fans] 9ttp
On Thu, Sep 8, 2011 at 7:32 PM, erik quanstrom wrote: > On Thu Sep 8 19:31:19 EDT 2011, rminn...@gmail.com wrote: >> On Thu, Sep 8, 2011 at 4:19 PM, Venkatesh Srinivas wrote: >> > Just asynchronous TClunk is enough to improve 9P's performance over >> > high-latency links dramatically. >> >> not my experience, but I'm willing to be convinced. > > iirc, async clunks are dangerous. > > -erik I never agreed with that conclusion from the oct 2010 discussion here. -- vs
Re: [9fans] 9ttp
Using the hacky inferno-npe async clunk implementation, from october 2010; from a Linux server running inferno-npe to a Linux client running inferno-npe; latency ~15ms. Getting the sources of cwfs from the server fell from 5.6 sec to 4.5 sec. For the 9 kernel sources, 51 sec fell to 41 sec. Got better results with sources (40ms away) but don't have the numbers handy. -- vs
Re: [9fans] 9ttp
Just asynchronous TClunk is enough to improve 9P's performance over high-latency links dramatically. -- vs
Re: [9fans] why not halt in x86 multicore
On i386 systems with MONITOR/MWAIT, you could use them; simpler than adding support for IPIs, I imagine. -- vs
Re: [9fans] pic
I use xcip on a modern Unix; http://plan9.bell-labs.com/sources/contrib/vsrinivas/xcip-modern.jpg There is a tarball in my contrib, xcip.tar; it is a copy of 1995-era xcip, made to build on recent linux systems. It needs Plan9port to build (9's lex and yacc, iirc). -- vs
Re: [9fans] _xinc vs ainc
On Sat, May 7, 2011 at 9:24 PM, Bakul Shah wrote: > On Sat, 07 May 2011 20:25:25 EDT erik quanstrom > wrote: >> > > the difference, and my main point is that the loop in ainc means >> > > that it is not a wait-free algorithm. this is not only sub optimal, >> > > but also could lead to incorrect behavior. >> > >> > I think a more likely possibility for the change is to have a >> > *copy* of what was incremented. lock incl 0(ax) won't tell you >> > what the value was when it was incremented. >> >> you can read the code. that value is not used by the thread library. > > If you want to use the value being atomically incremented, > there is no more efficient way on x86. May not be used now but > may be it can be used to make some synchronization primitive > more efficient? XADD -- vs
Re: [9fans] lenovo or other laptop support
>> >at coraid, we're considering asking new hires use ed for >> >a week exclusively. sam -d allowed? :) -- vs
Re: [9fans] non GUI editor for plan9
sam -d -- vs
[9fans] Listing a directory
Hi, In 9P, if I wish to list a directory, I need to TWalk to the directory, TOpen the directory fid from the walk, and then TRead till I have all of the contents of the directory. If the directory's contents do not fit in a single read, I imagine I need to loop around TOpen / Tread / / Tread / TOpen, till I get the whole contents and see the same QIDs in each open, to get a coherent listing. (is this accurate?) Unfortunately, a TOpened FID to a directory cannot be used for walking it (why?); so the only use of a TOpened fid to a directory is to list it or to get a QID. Would it be reasonable for a TOpened fid to a directory to create a copy of the directory listing at the time of the RPC? -- vs
Re: [9fans] realemu
Don't forget platforms != x86; an emulator would be needed on ARM (say) if a PC-like graphics chip found its way over there. I believe X has had such a beast (x86emu?) and used it heavily on PPCen. -- vs
Re: [9fans] sleep/wakeup bug?
On Fri, Feb 25, 2011 at 1:51 AM, Russ Cox wrote: > > your layout in your first email (i think) assumes that wakeup > > can be called twice. > > it doesn't. the scenario in my first email has exactly one > sleep and one wakeup. > > the canonical problem you have to avoid when implementing > sleep and wakeup is that the wakeup might happen before > the sleep has gotten around to sleeping. > There is a pattern in DragonFly BSD for use with tsleep (tsleep is evolutionarily related to p9's sleep, though the descent is tortuous), tsleep_interlock. tsleep_interlock(WAITCHAN); /* Kick off some operation or release some higher level lock or w/e */ tsleep(WAITCHAN, ...); tsleep_interlock queues a process but does not actually go to sleep. If a wakeup happens after the tsleep_interlock but before tsleep has fully slept, the process does not sleep in tsleep(). I do not know if a similar design would be feasible given Plan 9's sleep/wakeup; at first glance, it looks to be so - sleep_interlock would look similar to sleep's front-end, but it would not 'gotolabel(&m->sched)'. -- vs
Re: [9fans] Cute plan9/inferno client?
On Sat, Jan 29, 2011 at 5:51 AM, Lawrence E. Bakst wrote: > At 4:31 PM -0500 1/28/11, Venkatesh Srinivas wrote: > >The Tegra is probably one of the best ARM chips out there... > > > > > >No it isn't; the Tegra lacks NEON. > > > >-- vs > > 1. I should have said: > > "The Tegra 2 is probably one of the best ARM based SOCs out there ..." > > 2. It lacks NEON because it doesn't need that ISA. The chip has an > integrated GPU which does the same thing a different way. > 1. Oh, huh, the Tegra 2 is available now? I wasn't aware, sorry about that. Looks like it is A9-based, yay. 2. If your task is amenable to the integrated GPU, sure; but its not fair to say that NEON and GPUs (or the OMAP's DSP) do the same thing. -- vs
Re: [9fans] Cute plan9/inferno client?
On Fri, Jan 28, 2011 at 4:56 PM, erik quanstrom wrote: > On Fri Jan 28 16:33:03 EST 2011, m...@acm.jhu.edu wrote: > > > > > > The Tegra is probably one of the best ARM chips out there... > > > > > > > No it isn't; the Tegra lacks NEON. > > why do you think simd support is key? we have simd on x86 > and ignore it. > - erik > Doesn't matter for Plan 9, its true. I was thinking more generally. For media stuff or signals stuff, NEON is totally key for these ARM widgets, even more than x86en, which can muscle by on OOE and chutzpah. But even disregarding SIMD... the Tegra was based on the ARM11; the ARM11 isn't a terribly competitive core compared to the Cortex A8 in use by many of the ARMed chips; the A8 was a wider (dual-issue) design with a lower-power sleep, bigger/faster cache, and a much better branch predictor. So I'm not sure sure by what metric you'd say "the Tegra is one of the best ARM chips out there...". -- vs
Re: [9fans] Cute plan9/inferno client?
> > The Tegra is probably one of the best ARM chips out there... > No it isn't; the Tegra lacks NEON. -- vs
Re: [9fans] sound, graphics, tuner
There exist two different AC97 drivers; look at the port of Doom to plan9 for pointers to one of them. -- vs
Re: [9fans] ape/make - which one
On Wed, Jan 12, 2011 at 11:53 AM, Steve Simon wrote: > anyone know the lineage of plan9's ape/make, > I would like to find a manual page and understand what > clever features it does / doesn't support. > > -Steve > > http://plan9.bell-labs.com/sources/plan9/sys/src/ape/cmd/make/ident.c -- vs
Re: [9fans] Video mode problem after installation on QEMU
Use -vga std at the qemu command line. All of the modes work okay then. -- vs
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, Nov 17, 2010 at 12:23 PM, dexen deVries wrote: > On Wednesday 17 November 2010 18:14:35 Venkatesh Srinivas wrote: >> (...) >> I'd be very careful with vac -m and -a on Unix; both have been at the >> root of considerable data-loss on a unix venti for me. I'd recommend >> vac-ing tarballs, rather than using vac's on unix trees directly. But >> your mileage may vary... > > > could you please elaborate a bit about that data loss? > traversing symlinks breaks? some files not getting read by vac at all? I have a number of vac archives that were made by vac-ing my homedir that cannot be mounted (venti can't find some of the scores in the tree). However, they did pass scrubbing (scrub was a tool i wrote that is sorta like half of venti/copy -- it traverses a tree, checks hashes, but doesn't write them out to anything). I have a number of trees in which somehow the SHA1 of the empty string was inserted: 'da39a3ee...'. And I have a number of trees that were venti/copied from one venti to another but somehow damaged (scores missing) after the copy, without seeing errors. vac -a in particular was good at producing archives which either don't mount or are filled with missing scores or zero-scores vs sha1(empty string) problems. Unvac is not useful for anything involving directories, since it outputs dirs which cannot be modified. http://groups.google.com/group/plan9port-dev/browse_thread/thread/8b0e2bc4b45d12b0 is a reasonably common bug from vac, seen on deep-ish directory trees. Two threads i can't find on the google groups interface, from p9p-dev as well: 1) 8/12/09 'venti/vac/vacfs trouble' and 2) 8/17/09 'vac errors'. Both bugs still exist afaik. I've seen this: cache block 501: type 8 score 1f6 iostate 1 addr 502 ref 0 nlock 1 cache block 502: type 8 score 1f7 iostate 1 addr 503 ref 0 nlock 1 cache block 503: type 8 score 1f8 iostate 1 addr 504 ref 0 nlock 1 (output starts at 0x000...000 ;; goes up to score 0x000...200). vtcachebumpblock: no free blocks in vtCachezsh: IOT instruction ./vac -s -v -h tcp!acm.jhu.edu!17034 /export/home happen on deep-ish directory trees as well; I never managaed to track it down, though. Most recently, I've run two venti servers; vac targets the first one, every night the score is venti/copied to the second server. After two weeks, the two servers (with identical initial configuration) had different size and block counts from the http://.../storage. -- vs
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, Nov 17, 2010 at 11:51 AM, David Leimbach wrote: > I'm giving consideration to maintaining a venti-based setup for my house for > all the digital media we have (since getting our Apple TV, we've had more > stuff to stream around the house). > I've just now started playing with things like vac/unvac, to backup and > extract trees of my HFS+ file system and I wonder about a few things. > What do people do if they ever lose their venti scores? Seems like this is > "handle-less" data now, and fairly useless. > I figure I could keep a vac archive of venti scores, then I'd only need to > "remember" one, and name the files I store the scores in reasonably, but if > that's lost... what are my options? > Dave There is a script floating around (dumpvacroots or somesuch) that lets you recover vac scores given access to the venti arenas. I'd be very careful with vac -m and -a on Unix; both have been at the root of considerable data-loss on a unix venti for me. I'd recommend vac-ing tarballs, rather than using vac's on unix trees directly. But your mileage may vary... -- vs
Re: [9fans] 9p vs http
> Under certain situations, 9p can do some forms of pipelining. The tagged > requests don't have to be waited on in order, for the next outgoing request > to be sent, unless there's a dependency of one completing before the other, > or the evaluation of completion of a previous one on another. Only if the file & file server involved are decent. -- vs
[9fans] Plan 9 libc locks and semacquire?
Hi, In the paper 'Semaphores in Plan 9' by Sape and Russ Cox, there was this note: "The performance of the semaphore-based lock implementation is sometimes much better and never noticeably worse than the spin locks. We will replace the spin lock implementation in the Plan 9 distribution soon." As far as I can tell, this has not happened; is there any reason why? Are there any objections of the sem*-based locks? Thanks, -- vs
Re: [9fans] Passing a file descriptor between processes
On Fri, Nov 5, 2010 at 7:41 AM, erik quanstrom wrote: > On Fri Nov 5 07:36:42 EDT 2010, rogpe...@gmail.com wrote: > > see srv(3) > > http://plan9.bell-labs.com/magic/man2html/3/srv > > > > On 5 November 2010 10:29, Kirill A. Shutemov > wrote: > > > One of the ugliest interface in Unix is passing a file descriptor > between > > > processes [1]. Does Plan9 provide any mechanism for it? > > > > > > [1] > http://book.chinaunix.net/special/ebook/addisonWesley/APUE2/0201433079/ch17lev1sec4.html > > the op may have seen this: http://lwn.net/Articles/411845/ > —which i found hard to read because the author's changing > definition of "unix". > > - erik > > There was a discussion here a while ago about it too: search for 'sendfd on native Plan 9' from december 2008. Start here: http://9fans.net/archive/2008/12/515, follow till January 2009. Currently, if your processes have a common parent, you can use rfork; if not, you must resort to #s. '#s' is a pretty unfortunate interface, though... -- vs
Re: [9fans] Plan9 development
> > one interesting thing about that example is that if it were done again > for the Plan 9 environment, mk might well be even smaller, since > some of the existing functionality isn't really used, > or might be achieved by simpler mechanisms, or with functionality > added instead by further composition with other programs; > alternatively, it might be redone in a radically different way. > > How would you rewrite mk to be simpler? Thanks, -- vs
Re: [9fans] Plan9 development
In the ~8 years since the 4th edition release, there has been pretty continuous work on Plan 9, both at Bell Labs and in the 9fans community; nightly an ISO is constructed and uploaded. Changes have been incremental -- a tool appears, bugs are fixed, etc.. http://acm.jhu.edu/git/plan9 has a git tree of the sources from December 2002 through February 2009, if you like to view changes in that form. http://bitbucket.org/rminnich/sysfromiso has a mercurial tree of more recent changes, if that's what you're looking for. -- vs
Re: [9fans] A little more ado about async Tclunk
erik quanstrom wrote: > what's a reasonable definition of "standard"? I've been using 'decent' in much the same way 'standard' or 'disk' is being used; I'd actually prefer nemo's idea of a QTDECENT qidtype to marking the file server. The original QTDECENT proposal (actually originally inverted logic, in the form of QTCTL) said this about indecent files: "this file does not behave like a regular file, do not cache and handle with care". I think stating it in the form: "indecent files require in-order issue _and_ completion of a 9p request stream from the perspective of the application making the system calls/generating 9p messages to the file service." There is a loose convention now of qid.version == 0 meaning something like 'indecent'. > roger, charles describe races in both async TClunk issue and async RClunk wait: r1 := A: sekrit B: TOpen(name, ...) spawn TClunk TOpen(name, ...) TClunk(fid..) RClunk(yep! / nope!) ... Perhaps I'm being really thick, but this is the same scenario as a current: A:explicit process B: ->TOpen(name, ...) <- ROpen(yea) -> TOpen(name, ...) <- ROpen(zzz) -> TClunk(fid) <- RClunk(fid) ... zzz := Yes, of course, on a 'decent' fileserver; 'no' on an indecent one, or with OEXCL in A's open. All the asynchronous issue opens is this concurrency issue with yourself (A against itself)... but it already exists if you don't use OEXCL in the (A against B) form. On Fri, Oct 29, 2010 at 11:59 AM, Bruce Ellis wrote: > this discussion was more interesting in thev UNIX room. froggie hasn't > hung up yet thru a serious thrashing this evening - and all the FSs > are synthetic - it has no disk. > > as much as i like philosophizing that's not my way. > > brucee > When you get a chance, could you describe your approach in more detail? Bruce Ellis also wrote: >> Roger Peppe wrote: >> even sending Tclunk synchronously is still problematic in quite a few scenarios, >> for the reasons i outlined above. > gee i thought i was the first to say deadly-embrace on this thread. > it's not only problematic it's wrong. just reiterating what little > shaun said circa 1999. Sorry, I don't see the embrace... when the original close() caller doesn't wait on the TClunk/RClunk pair, what would it stall on? What would the close process stall on? Thanks! -- vs
Re: [9fans] A little more ado about async Tclunk
On Fri, Oct 29, 2010 at 12:01 PM, Charles Forsyth wrote: > >Do you do completely asynch clunks or just the wait for the response?. > > it uses `completely' async clunks, which is why it can be broken. > > having the original process send the Tclunk and not wait > for the Rclunk is different. i think it was mentioned last time this > matter came up, and that's probably why i didn't pursue this discussion > further then, since that change is less of a problem. (at least, if you > don't mind > close not returning an error from the clunk, but since the current > implementations > suppress errors from clunk, it's a trickier position to sustain.) > > I think that this is a fine approach as well -- the vast majority of the performance improvement is in eliminating the wait for the Rclunk, not in the asynchronous issue. I didn't use this approach here not because I wanted to issue the clunk out-of-line, but because it was harder to code (I'd have needed to split devmnt's mountio()). This was actually the approach Wes and I did last year when commenting on the desirability of asynchronous clunks, in a purely "user mode" process. The performance gains were very similar to this work. -- vs
Re: [9fans] A little more ado about async Tclunk
On Fri, Oct 29, 2010 at 5:01 AM, Charles Forsyth wrote: > >What you are saying is that the problem could be something like: > > >-> Tclunk > >(do not wait for response) > >-> Topen (the file is exclusive) > > no, because what actually happens is closer to >A: Topen >... >queue request to *another process* to send Tclunk >... >A: Topen > > leading to either case 1 >A: Topen >B: Tclunk >A: Topen > > or case 2 >A: Topen >A: Topen >B: Tclunk > > depending on the scheduling of the two processes A and B > > introducing that other process introduces a new race, that simply did not > exist before, because > that other process didn't exist before. you always got case 1. > > and indeed, with the test i ran on the modified system usually it gets case > 1 > because of timing, but eventually a case 2 shows up and the second open > fails. > > Correct. Either ordering is reasonable and not-a-problem when dealing with a decent fileserver. There are certainly file servers and open options that force single-open, but this approach is not for them. -- vs
Re: [9fans] A little more ado about async Tclunk
On Thu, Oct 28, 2010 at 4:55 PM, Nemo wrote: > i have decent servers that wait for clunk to operate on written data once > it's complete. all octopus spoolers do that. > > Heh; when I wrote 'decent', I was recalling the old proposed QTDECENT qid type. I didn't mean to impugn your file servers; they're probably very nice people and if I met them, I'd should buy them a drink. Let's try to define 'decent' for this thread -- a decent fileserver is one on which close()s do not have any client-visible or semantic effect other than to invalidate the Fid that was passed to them. Lets see how many file servers we can think of that are 'decent': fossil, kfs, ken, memfs, ... -- vs
Re: [9fans] A little more ado about async Tclunk
On Thu, Oct 28, 2010 at 4:18 PM, Charles Forsyth wrote: > the race is that there's nothing to say that the clunk completes before the > process continues on to do something more, including some action that > depends on the clunk completing, > such as simply repeating the open. that open can fail if the mode or the > name > imposes particular constraints; constraints that depend on the order of > events as > expressed by the process. > Of course, we don't want the process to wait. On a 'decent' file server, TClunk doesn't do anything that a process would have to wait for, except invalidate the FID; (I exclude ORCLOSE and OEXCL files here). Guarding this with MCACHE means that there was already a statement from the user "I think that this file server is a decent character.". You can definitely write file servers on which TClunk does things -- ones in which the right combination of opens and clunks fires the missiles, for example. This is not for those file servers. On a decent server, since clunk cannot fail and won't launch the missiles, you can't really do anything that would depend on the result anyway... Seem reasonable? -- vs
Re: [9fans] A little more ado about async Tclunk
On Thu, Oct 28, 2010 at 8:12 AM, Charles Forsyth wrote: > you're essentially replacing >f := open(name, ...) >... >close(f) > > which runs as a sequential process, and subject to the usual rules for > sequential > composition, by > >f := open(name, ...) >... >spawn clunk(f) > > which introduces a race with an invisible anonymous process with no means > of synchronisation. > > what could possibly go wrong with that? > > (actually, it's more complex than spawning a new process, but the added > complexity of a set of service processes changes the details but not the > existence of the race, and doesn't affect the anonymity.) > Hmm, I don't see the race there; perhaps I'm missing something or being blind; could you explain? Also, if that is racy, isn't this at least analogous? f := Sys->open(name, ...); ... # put f in a cyclic structure f = nil; evil_cyclic = nil; ... # wait until an invisible service process closes it, perhaps later? Thanks, -- vs
Re: [9fans] A little more ado about async Tclunk
On Thu, Oct 28, 2010 at 11:53 AM, Francisco J Ballesteros wrote: > Yes, in general, agree, but this was about clunk usage and > semantics, and I think it's important to have the I'm done message > synchronous when you need that. > > Btw, I think you mean Tstat, don't you? > I'm not sure Twstat would work that way in practice. > > No, that was actually Twstat. -- vs
[9fans] A little more ado about async Tclunk
Hi folks, I just committed a very simple implementation of asynchronous TClunk to inferno-npe. The implementation will only defer sending clunks when MCACHE is specified on the mount (via the new -j option to Inferno's mount, analogous to Plan 9's mount -C) and when the file is not marked ORCLOSE. The implementation is very simple -- I queue the RPCs devmnt would issue synchronously and send them from a single kproc, one-after-another. There is no attempt to overlap the TClunks. There is no logic to handle flushing a deferred clunks on an unmount yet. The diff is not as clean as I would like, but it works. I connected from an inferno-npe client to an inferno-os server, sharing the Plan9 source tree from August 16th and measured the time required for fcp -R 16 -W 16 to get the sources to /plan9/sys/src/cmd/cwfs and to get /plan9/sys/src/9. The server is approximately 15ms away from the client; both are running Linux. For the cwfs sources, it takes me an average of 5.68sec (stdev 0.57s, 6 tests) issuing synchronous clunks (the default) and an average of 4.54sec (stdev 0.31s, 6 tests) issuing asynchronous clunks. For the 9 kernel sources, it took about 51s with synchronous clunks, about 41s with asynchronous clunks on. I only did 3 tries with the 9 sources. Thought this might be interesting... -- vs
Re: [9fans] πp
> > 2) you can't pipeline requests if the result of one request depends on the > result of a previous. for instance: walk to file, open it, read it, close > it. > if the first operation fails, then subsequent operations will be invalid. > Given careful allocation of FIDs by a client, that can be dealt with - operations on an invalid FID just get RErrors. -- vs
[9fans] iwp9 video?
Hi, For those of us who can't be at IWP9, are there any plans to videotape the talks? Thanks, -- vs
Re: [9fans] Load spikes on intervals in qemu
virtio is available in vanilla qemu nowadays. Why do you want to run your file server in qemu anyway, though? For snapshots, I use dump9660 on my host, coupled with inferno; u9fs would work just as well. -- vs
Re: [9fans] Load spikes on intervals in qemu
On Tue, Aug 17, 2010 at 9:50 AM, Brad Frank wrote: > Hi, I recently did a clean install of plan9 on qemu on linux. I've noticed > that the load is spiking on an interval every 30 seconds or something like > that. I looked at suggestions that it might be venti and timesync. But it > couldn't be venti because I didn't install venti, I have a fossil only > install. I looked at timesync, and killed it, and the load was still > spiking. It was suggested I try zwansch's gtop, which I did do, and I found > that fossil was using the most utime/stime. But it seemed like two different > threads? Fossil [disk] and Fossil [flush]. When I killed fossil [disk] my > load dropped, but obviously I lost disk access. What could possibly be > wrong, and why is it doing this? Another interesting thing, is that when the > load spikes like that, the emulator seems to temporarily lag or not respond, > until the load drops again, so this also effects the performance of plan9. > Any suggestions would be quite helpful. > I run Plan 9 in qemu, but I run neither fossil nor any other (major) disk file server in qemu. Instead, I have Inferno on my host serve files to Plan 9. To accomplish this: 1) I installed Plan 9, as normal, into a qemu disk image, with fossil as my fileserver 2) I tarred up my root and copied it to my host 3) I patched the 9 boot program to accept a file server on port rather than 564 (styx instead of 9fs) 4) I untarred the root fs and had Inferno: 'styxlisten -A tcp!*! export /plan9' 5) Start the both (qemu and inferno) together. 6) Profit! -- vs
Re: [9fans] GSoC Segments: What have I been doing, anyway?
> > typically, accessing memory above the stack is not a problem, so a single > red zone below the stack is enough. this red zone need not be an actual > segment, if the thread library picks segment addresses. although this > would force the thread library to keep tabs on the top of the heap. > > Yep! http://plan9.bell-labs.com/sources/contrib/vsrinivas/stackalloc.c is the thread stack allocator, passed through cb. -- vs
[9fans] GSoC Segments: What have I been doing, anyway?
Hi 9fans, So GSoC is more or less over! First, I really need to thank David Eckhardt and Erik Quanstrom for putting up with me this summer; dealing with me can be as frustrating as pulling teeth with a screwdriver when a patient only speaks another language. Next time I see either/both of them, I owe them beer/non-alcohol-beverage or pizza [1]. Also, thanks to Devon for everything, including waiting over an hour for me downtown. I have been working on the segment code and making forays into libthread. Let me first talk about what exactly I did, without talking about the motivations. In Plan 9 fresh-out-of-the-box, a process's address space is constructed from a series of segments, contiguous ranges of address space backed by the same object. By default, a process has a small number of segments: a Text segment, backed by the image, a Stack segment, backed by anonymous memory, a Data segment to back the heap structure, and a BSS segment for the usual purpose. Each process also has a small series of slots, currently 4, for other segments, obtained via the segattach() system call and released via the segdetach() syscall. When a process calls rfork(RFPROC), segments from the "shared" class are shared across the fork and "memory" class segments are copy-on-write across the fork; each process gets its own stack. When a process calls rfork(RFMEM | RFPROC), all segments except the Stack segment are maintained across the fork except the Stack segment. When a process class exec(), segments marked with SG_CEXEC are detached; the rest are inherited across the exec(). The Stack segment can never be inherited. Across an rfork(RFMEM | RFPROC), new segattach()es and segdetach()es are not visible - in Ron Minnich's terminology, we have shared memory, but not shared address spaces. First, I modified the segment slot structures, to lift the limit on four user segments. I made the segment array dynamic, resized in segattach(). The first few elements of the array are as in the current system, the special Text, Data, BSS, and Stack segments. The rest of the segment array is address-ordered, and searched via binary searches. The user/system interface doesn't change, except that the limit on segment attaches is now from the kernel memory allocator, rather than a fixed per-process limit. I further changed segattach() to add more flags: SG_NONE: A segment with the SG_NONE flag set does not have a backing store. Any accesses, read or write, cause a fault. This segment flag is useful for placing red zones at user-desired addresses. It is an error to combine the SG_NONE and SG_COMMIT flags. SG_COMMIT: A segment with the SG_COMMIT flag set is fully pre-faulted and its pages are not considered by the swapper. An SG_COMMIT segment is maintained at commit status across and exec() and rfork(RFMEM | RFPROC). If we are unable to satisfy pre-faults for all of the pages of the segment in segattach(), we cancel the attach. It is an error to combine the SG_COMMIT flag with SG_NONE. SG_SAS: A segment attached with the SG_SAS flag appears in the address space of all processes related to the current one by rfork(RFPROC | RFMEM). An SG_SAS segment will not overlap a segment in any process related via rfork(RFMEM | RFPROC). I finally changed libthread. Currently, libthread allocates thread stacks via malloc()/free(). I converted libthread to allocate thread stacks via segattach() - each thread stack consists of three segments, an anonymous segment flanked by two SG_NONE redzones. Currently I have posted a prototype (very generously called 'prototype') implementation of the above interface to sources; the prototype kernel omits a number of the checks claimed above. SG_SAS faults are not handled; SG_SAS segments must be SG_COMMIT. SG_COMMIT has no limit, which makes it very easy to crash a system by draining the page queue readily. The prototype libthread is of considerably higher quality, I think, and would be usable a production-grade implementation of these interfaces. The prototype kernel is usable though - I have run it alone on my terminal for approximately a month. However, the prototype kernel shows us that the interface can be implemented efficiently - even when using three segattach()es per thread stack, creating 1024 threads took 2.25s real time on a 400MHz AMD K6, versus 0.87s realtime with the original libthread and 9 kernel. Creating processes with thousands of segments is not incredibly speedy, but it is workable and there is a lot of low-hanging fruit that can improve performance. The SG_SAS work is fairly unusual for Plan9 - each process originally had a single, fixed-size segment slot array. Now, a process has a per-process segment array and a second shared segment array. The shared array is referenced by all processes created by rfork(RFMEM | RFPROC); the shares are unlinked on exec() or rfork(RFPROC). The SG_SAS logic was added to match the current semantics of thread stacks - as they are allocated by malloc() and free()
Re: [9fans] Async clunk?
On Sun, Aug 8, 2010 at 3:10 AM, Charles Forsyth wrote: > >Perhaps we could turn on async clunk for other files when the chan has > >CCACHE set (mount -C iirc). We already believe that the fileserver is > >'decent' then... > > that's more plausible, since you've declared that you're not interested > in certain details, but then the larger question arises: why pick on clunk? > in Op, the thing that reduces network latency is combining the sequence > open, (read|write), close > into a single operation. > When we tried out async Clunk in the Journal Callbacks work, it offered some fairly great improvements; and the details that it varies in are very minor compared to uncached 9p, particularly for decent fileservers. Since Clunk cannot fail, this also admits a simple (perhaps) approximation - issue the clunk as expected, but do not wait for RClunk. > services that wait for a close to commit, Hmm. Does this mean that any modification to devmnt, to keep a larger pool of FDs active for example, rather than clunking eagerly while walking, is unsafe? That would hold around directory FDs, which would make services like the above unhappy, or at least behave differently. Perhaps those file servers are assuming something not specified in the 9p definitions? Also, cyclic structures in Limbo with file descriptors embedded will not release the FD immediately after the outer structure is discarded. In a sense, that is also a delayed Clunk... -- vs
Re: [9fans] Async clunk?
On Sat, Aug 7, 2010 at 5:33 PM, Charles Forsyth wrote: > >close must be synchronous, unless you aim for NFS semantics. > > to be more explicit: think of exclusive-use services, services that wait > for a close to commit, > files with DMEXCL, and files opened ORCLOSE > For DMEXCL and ORCLOSE, we'd always issue synchronously. Perhaps we could turn on async clunk for other files when the chan has CCACHE set (mount -C iirc). We already believe that the fileserver is 'decent' then... -- vs
[9fans] Async clunk?
Hi,
At IWP9 4e, we suggested that asynchronous TClunks would do well for 9p
performance on moderate to high latency links; yesterday I came across a
thread from 2006/01 on 9fans discussing asynchronous TClunks for other
reasons ('clunk clunk' is the title; http://9fans.net/archive/2006/01/105).
Did anyone do any further work along this? As far as I can see, cclose()
does a synchronous clunk...
Thanks,
-- vs
[9fans] Auth-ed mount of sources from 9?
Hi, How do you mount sources auth-ed from 9? I must confess I've never done this with 9 by itself, I've always used Inferno's "mount -9" (even on plan9)... -- vs
Re: [9fans] coraid in the news
> clearly, coraid won't send commandos to make sure that you don't use zfs with coraid storage. If they do, will they be from outer space? -- vs
Re: [9fans] interesting timing tests
On Mon, Jun 21, 2010 at 10:40 AM, erik quanstrom wrote:
> void
> lock(ulong *l)
> {
>ulong old;
>ushort next, owner;
>
>old = _xadd(l, 1);
>for(;;){
>next = old;
>owner = old>>16;
>old = *l;
>if(next == owner)
>break;
>}
> }
>
> void
> unlock(ulong *l)
> {
>_xadd(l, 1<<16);
> }
Sure, that's reasonable in C; (i wasn't sure how to do it in asm for 8_a_,
that was what I was asking about). Just also remember to provide xadd; the
distribution 8a and 8l didn't support it last I checked.
Just another observation, we can bypass the load of old in the uncontended
case if we reverse old = *l and the compare/break in lock.
Anyway, thoughts on this lock?
-- vs
[9fans] lock question?
Hi, I've asked about this before, but I still don't see the reason; In 9/port/sysproc.c:sysrfork(), after a new process has been created, dupseg is called on each of the donor process's segments; why is the new process's p->seglock locked for this duration? The new process hasn't been published yet, no one else could serialize on that lock... Am I missing something? -- vs
[9fans] A little ado about taslock
Hi, Erik's thread about a 16-processor x86 machine convinced me to try something related to spinlocks. The current 9 spinlocks are portable code, calling an arch-provided tas() in a loop to do their thing. On i386, Intel recommends 'PAUSE' in the core of a spin-lock loop; I modified tas to PAUSE (0xF3 0x90 if you prefer) if the lock-acquire attempt failed. In a crude test on a 1.5GHz p4 willamette with a local fossil/venti and 256mb of ram, 'time mk 'CONF=pcf' > /dev/null' in /sys/src/9/pc, on a fully-built source tree, adding the PAUSE reduced times from an average of 18.97s to 18.84s (across ten runs). I tinkered a bit further. Removing the increments of glare, inglare and lockstat.locks, coupled with the PAUSE addition, reduced the average real time to 18.16s, again across 10 runs. If taslock.c were arch-specific, we could almost certainly do better - i386 doesn't need the coherence() call in unlock, we could safely test-and-tas rather than than raw tas(). There're also other places to look at too, wrt to application of arch-specific bits; see: http://code.google.com/p/inferno-npe/source/detail?r=b83540e1e77e62a19cbd21d2eb54d43d338716a5for what XADD can do for incref/decref. Similarly, pc/l.s:_xdec could be much shorter, again using XADD. None of these are a huge deal; just thought they might be interesting. Take care, -- vs
Re: [9fans] interesting timing tests
knowing which locks would rock.
i imagine the easiest way to find out would be modify lock() to bump a
per-lock ctr on failure-to-acquire. on i386 lock add would be the easiest
way to do that, i think. add an 'inited' field to the spinlock and a list
linkage as well, to allow for easy examination when you hit the system with
acid.
also if the locks in question need to be locked and the resources they
protect cannot be split, we can do much better than our current spinlocks:
void lock(int *l) {
int old = __sync_fetch_and_add(l, 1);
short next,owner;
do {
next = old & 0x;
owner = (old >> 16) & 0x;
old = *l;
} while(next != owner);
}
void unlock(int *l) {
__sync_fetch_and_add(l, (1 << 16));
}
(this is in gcc-C, but porting wouldn't be bad; the unlock
__sync_fetch_and_add would be LOCK ADD on i386. the __sync_fetch_and_add in
lock would be LOCK XADD on i386. i don't know 8a's syntax well enough to do
this right, in particular how 8a's pseudoregs work).
(many credits to nick piggin for this lock design. its totally rad.)
-- vs
Re: [9fans] 9atom hw support
Are the canonical sources to 9atom online/web-viewable someplace? -- vs
Re: [9fans] ctrl radeon: not working for ATI Radeon 9600 Mobility?
> [...] the main slowdown > is reading from the hardware framebuffer, which plan 9 does a lot. Where in the draw path does Plan 9 read back from the framebuffer? Why so much? Also, there is apparently a fast way to read-back on radeon series h/w - map some system memory on the card via the GART and do the same dance that drm's r100_copy_blit does. If the formats are amenable, it might be nice to expose cards' fast download-from-screen features, by adding something like Ctrl->dfs() or the like... -- vs
Re: [9fans] ctrl radeon: not working for ATI Radeon 9600 Mobility?
On Sat, May 1, 2010 at 10:08 AM, erik quanstrom wrote: >> erik quanstrom wrote: >> > i think it's mainly a question of getting the best graphics >> > performance. but these days the 2d accelleration (especially >> > for radeon) is fairly poor. >> >> Do you happen to have any workloads where this is a problem? There is >> a lot more we can do for the radeon, particularly the r100-r300 chips, >> which could improve draw performance. > > i didn't make it clear that i'm using the vesa > interface. and it's very slow for every day > use with acme, but it sure displays pictures quickly. > > ; pci | grep vid | grep 1002 > 1.5.0: vid 03.00.00 1002/791e 10 0:d80c 134217728 1: 16 > 2:fbff0004 65536 3: 16 4:a001 256 5:fbe0 1048576 > ; pci 1002/791e > 1002/791e > ATI Technologies Inc RS690 [Radeon X1200 Series] > >> To ask the radeon gpu to do anything interesting for you, you must >> carry out a series of register writes - for example, to fill a >> rectangle with a solid color, you write the rectangle's color to >> DP_BRUSH_FGRD_CLR (register 0x147c), the start to DST_X_Y, and the >> width and height to DST_WIDTH_HEIGHT. Everything you can ask the >> radeon to do is ultimately done via register writes. > > i wonder if this pio doesn't explain a lot of the slowness. > i would have thought that they would have used a ring. There is a command processor on the radeon, which we do not initialize; it can share a ring of commands with the host; it can also accept indirect buffers. If the radeon driver (on supported h/w) was too slow for general work, then we could use it, but if its fine right now, there is no need. If we really needed speed, we could go further - currently plan9 video drivers can only accelerate fill and scroll, but in principle the draw path could use any other functionality the card was willing to do for it; for the earlier radeons, the 2D engine can do a lot... for the later ones, we could use the 3d engine in the same style as X's EXA. -- vs
Re: [9fans] ctrl radeon: not working for ATI Radeon 9600 Mobility?
erik quanstrom wrote: > /sys/src/cmd/aux/vga/radeon.c uses radeon_pciids in radeon.h. > it ignores the vids and dids in /lib/vgadb. this is probablly a > mistake. Agreed. erik quanstrom wrote: > i think it's mainly a question of getting the best graphics > performance. but these days the 2d accelleration (especially > for radeon) is fairly poor. Do you happen to have any workloads where this is a problem? There is a lot more we can do for the radeon, particularly the r100-r300 chips, which could improve draw performance. Have you ever tried enabling HW_ACCEL in the Radeon driver? It accelerates fill/scroll to some degree, which would certainly be a first step. Gabriel Díaz wrote: > I guess the radeon driver is unable to work with modern ati cards no? Right now the Radeon driver can deal with r100-r300; it is missing PCI VID/DIDs for a number of r200 cards, mostly the later 9200 and 9250 series. It is also missing a number of r300s (9600, 9800) and the PCIe X1050, X3xx, X5xx, and X6xx. I don't know how well the later cards will work, but they are nominally the same as the 9500/9700, which are supported. Sean Stangl at CMU started working on support for more modern (R500/R600 iirc) cards; I believe he got most of ATOMBIOS support and snarf() routines for the r600 done. > I would love a book recommendation to learn something about graphics > programming, so at least i can guess what or where to look to try to debug > the problem. I don't know of a book that describes any modern graphics hardware, sorry. However, here is a bit of an overview of what is going on with the radeon, at least: The radeon exposes a pair of bus-accessible memory regions (BARs), the framebuffer and the MMIO region; these regions appear in physical memory and are mapped by vgaradeon.c (take a look at radeonenable()). On the Radeon, BAR 0 holds the framebuffer, IIRC; vgalinearpci() searches the card for the largest region and guesses that that is the framebuffer. BAR 2 holds the MMIO region. We ask that the framebuffer segment is set to write-combining via the MTRRs; we cannot do the same for the MMIO region. To just draw, you can write to the framebuffer region. To ask the radeon gpu to do anything interesting for you, you must carry out a series of register writes - for example, to fill a rectangle with a solid color, you write the rectangle's color to DP_BRUSH_FGRD_CLR (register 0x147c), the start to DST_X_Y, and the width and height to DST_WIDTH_HEIGHT. Everything you can ask the radeon to do is ultimately done via register writes. agd5f's blog has two entries that are pretty useful: http://www.botchco.com/agd5f/?p=50 and http://www.botchco.com/agd5f/?p=16. -- vs
Re: [9fans] ctrl radeon: not working for ATI Radeon 9600 Mobility?
On Thu, Apr 29, 2010 at 7:17 AM, erik quanstrom wrote: >> I added two lines of code to /sys/src/cmd/aux/vga/radeon.h; compiled >> vga and kernel; added line to /lib/vgadb. >> Now kernel sees the device (and I see it too: cat /dev/vgactl), but >> attempt to start framebuffer gives blank screen. Pointers are >> appreciated! > > it didn't work for me, either. i added a bit of > code to support the newer "atom bios" but > ran out of patience and the vesa driver works. ATOM BIOS should be unnecessary for the R9600; it is an r300-era card. I've not ever tried with a mobility (is it an m10?) r9600, but the desktop variant works when its PCI IDs are added... I forget whether I had to enable r300hack, though, for it... -- vs
[9fans] LOCK XADD for i386 incref/decref
Hi, A few months ago, I added a patch to inferno-npe to use LOCK XADD instead of the current lock/add/unlock sequence for incref and decref: (http://code.google.com/p/inferno-npe/source/detail?r=b83540e1e77e62a19cbd21d2eb54d43d338716a5 and http://code.google.com/p/inferno-npe/source/detail?r=82f13e6755218ecb7dec0f1392b2eb8bfe0bb2c7). On a 2.66 GHz Core 2 Duo, we noticed a full 14% reduction in runtime (9.6s vs 11.2s) of the thread-ring test. Similar performance gains were noticed on a Core i7 machine, but I no longer have the numbers handy. Perhaps this change is interesting for Plan 9... -- vs
Re: [9fans] missing machs unearthed
Not that this is a great answer, but the way I've done SMP boot without parsing either the ACPI or MPS tables was to issue broadcast init and startup IPIs, rather than targeted ones. All the CPUs in an i7-based machine came up, fwiw... -- vs
Re: [9fans] Semaphores for libc Lock?
Not quite what I was asking; I was asking about lock() / unlock() in libc being changed to use semacquire/semrelease (the syscalls)... -- vs
[9fans] /sys/src/cmd/fossil/xx?
What is /sys/src/cmd/fossil/xx? -- vs
[9fans] Semaphores for libc Lock?
Hi, In Russ Cox's and Sape Mullender's 2008 IWP9 paper on semaphores in Plan 9, they suggest that semaphore (semacquire/semrelease)-based locks in libc were better than the current spinning ones; they note that the locks would be replaced in the distribution soon. Is that still in the cards? -- vs
Re: [9fans] gsoc2010 + plan9
On Wed, Mar 3, 2010 at 6:40 PM, David Leimbach wrote: > What's the current state of the following: ... > 3. Cpu from Inferno to Plan 9? (i think this might be in npe's inferno > fork) ... Yep, its there. :D -- vs
Re: [9fans] vtcache exhaustion
On Mon, Feb 22, 2010 at 7:15 PM, Russ Cox wrote: > I am not sure, but I have seen things like this since > I tried to expand the venti protocol to >16-bit blocks. > I have since decided that was a mistake, too intrusive > a change, but I have not yet gotten around to rolling > back to the old versions of the files. > > That might make the bug go away, whatever it is. > Somehow it seems unlikely you have that many blocks > in your cache. I doubt it. I saw this bug in 2/2009, before the v4 protocol was deployed... -- vs
Re: [9fans] kenfs question?
On Sat, Feb 6, 2010 at 11:16 AM, erik quanstrom wrote: >> ilock() (pc/lock.c) calls splhi() and then calls lock(). If that lock were >> contended, >> how would the system not stop? And on a UP system, if you're inside an >> splhi() block, >> why would you need to take an uncontended lock? > > good question. on a up system, splhi() is sufficient. on a mp > system, the losing process will spin until the winning process > is finished. > > ken fs does need to return to its mp roots. > Did kenfs ever run on MP systems and use >1 CPU? -- vs
Re: [9fans] Speed of 9pfuse on Linux
Perhaps the time to talk about QTDECENT is at hand? -- vs
[9fans] procdata() question?
Hi, If I'm writing a library and I'd like to use procdata, is there any way to safely do so, considering that applications using the library might be using procdata as well? Perhaps it should take a key, like pthread_getspecific/_setspecific? Thanks, -- vs
Re: [9fans] kenfs question?
On Sun, Feb 7, 2010 at 5:42 AM, Charles Forsyth wrote: >> I guess I wasn't clear; what I was asking was why it was safe to >> attempt to take a lock when splhi() at all. > > because such a lock is always taken with splhi, using ilock. > you might find in older code the use of lock in interrupt handlers, > protected by the implicit splhi of interrupt handling, and in > a few cases explicit splhi calls before lock. Ah, okay. I didn't realize that. Perhaps it'd be clearer if ilock/iunlock took a different type than lock/unlock? Thanks! -- vs
Re: [9fans] kenfs question?
Hi, I guess I wasn't clear; what I was asking was why it was safe to attempt to take a lock when splhi() at all. On a UP, if that lock is contended and interrupts are off, you'll lockup, no? (The timer interrupt handler will never run, reschedule will never happen, the lock holder will never run). -- vs
[9fans] kenfs question?
Hi, I was looking at the sources to kenfs and saw something that confused me - ilock() (pc/lock.c) calls splhi() and then calls lock(). If that lock were contended, how would the system not stop? And on a UP system, if you're inside an splhi() block, why would you need to take an uncontended lock? Thanks, -- vs
[9fans] Retargeting 8{al}?
Hi,
I am working on a 32-bit microprocessor-on-an-FPGA; anyone know
what'd be involved in retargeting 8{al} (or friends) to a new
CPU? Where is the line between what 8a and 8l do? (which is
responsible for the pseudoregisters? instruction selection?)
Thanks,
-- vs
[9fans] hg mirror of /sys/src and /sys/include
Hi, I started mirroring /sys/src and /sys/include into an hg repository regularly, usually daily. The tree is available at http://code.google.com/p/plan9os. hg clone https://plan9os.googlecode.org/hg/ plan9 should do the trick to clone a copy. Hopefully this will be useful, -- vs
[9fans] vgaradeon (was Re: Problem of last update?)
On Wed, Jan 27, 2010 at 08:07:12AM -0500, erik quanstrom wrote: aux/vga: vgactlw: : bad VGA control message "type radeon" ... rio: can't open display: initdisplay: /dev/draw/new: no frame buffer init: rc exit status: rio 23: display open you need to add vgaradeon +cur to your plan9.ini. I guess this brings up something else. On 1/19, the Radeon driver made it into the Plan 9 distribution; anyone with a Radeon R100 (7000, 7500, Mobility M6, M7), some R200 (8500, 9000) or some R300s (9500, 9700) is encouraged to try them out, see if they feel faster or work for you or whatnot. The driver also has, disabled-by-default, hardware-accelerated fill and scroll routines. If you wish to try them out, modify /sys/src/9/pc/vgaradeon.c to define HW_ACCEL; I'd like to hear if they work on your card and if you can feel any difference. In the short term future, I hope to test the driver on a number of other cards and add some PCI IDs/anything-else-needed, such as the R9600, 9800, and 9200/9250. Longer-term, my goal is always support for more cards. Take care, -- vs
Re: [9fans] Hardware for Plan9
On Mon, Jan 11, 2010 at 12:24 PM, erik quanstrom wrote: >> Venti deals with incompletely written blocks; the arenas and index structures >> are still workable. The situation is even recoverable - a proxy could notice >> that one of the backends failed to return a read, so it rewrite the data from >> an other copy (which it can verify) to the failed one. > > what are you using as a venti proxy? or is this more theoretical? When I ran this on real h/w, I used Inferno's vcache, which works as-is. I have a binary (but have lost the source) for a Plan 9 equivalent, but cooking one up again should take only a few hours; its job is really simple. > how do you protect against lossage in the fossil cache? I don't, any more than fossil+fs does. > - erik -- vs
Re: [9fans] Hardware for Plan9
On Mon, Jan 11, 2010 at 10:22:49AM -0500, erik quanstrom wrote: An alternate configuration, which takes more memory, but might offer a bit more in the way of survivability, would be to not use fs for venti. Instead, run one venti daemon per disk, with independent arenas/indexes. Insert a little Venti proxy between fossil and your daemons; it should try reads on each venti until one returns a block; it should issue writes to all of them. why would that be more survivable? When an fs mirror is out of sync, which mirror holds the right data? Fs has no way of knowing. Venti at least has the block hashes. Imagine cutting power after a first disk in a mirror has data written but subsequent ones don't? With fs, disaster. (well, sorta. devfs always reads from the first device in a mirror first, and writes to the devices in order as well. you might get lucky, but you wouldn't know about errors until you have to hit the second device. at which point its too late.) Venti deals with incompletely written blocks; the arenas and index structures are still workable. The situation is even recoverable - a proxy could notice that one of the backends failed to return a read, so it rewrite the data from an other copy (which it can verify) to the failed one. Also, the isolation granted by writing data to two venti daemons is nicer than scribbling blocks to both disks; you can bring down either back-end venti while the system is running. You can even move one of the pairs to a remote system. If disks are removable in your configuration, you can even grow the available space live. -- vs
Re: [9fans] Hardware for Plan9
On Mon, Jan 11, 2010 at 01:01:43PM +, Steve Simon wrote: Interessting idea. Do you use a fossil only configuration and mirroring with fs(3)? I use fossil and venti with everything mirrored with fs(3) - each partition. Hi! An alternate configuration, which takes more memory, but might offer a bit more in the way of survivability, would be to not use fs for venti. Instead, run one venti daemon per disk, with independent arenas/indexes. Insert a little Venti proxy between fossil and your daemons; it should try reads on each venti until one returns a block; it should issue writes to all of them. Good luck, -- vs
[9fans] port/allocb.c question
Hi, In _allocb() in port/allocb.c, there is an _xinc(&b->ref) on a long inside the Block header. No one else ought have a pointer to this block yet, seeing as its not done being allocated. Is there any reason a locked increment is being used? -- vs
Re: [9fans] vga vesa works on mp pc
On Tue, Dec 8, 2009 at 7:05 PM, wrote: > I've just pushed out a small change to /sys/src/9/pc/realmode.c that > allows monitor=vesa to work on multiprocessor pcs without *nomp being > defined in plan9.ini (i.e., you can use all available processors [or > cores] and still use vesa mode). I've also pushed out new kernels, so > the distribution built tonight should include this fix. Cool, thanks! If I read this right, you turn off interrupts on just the CPU executing in realmode. Is the really all that had to be done for vesa to support MP systems? -- vs
[9fans] etherigbe.c using _xinc?
Hi, I noticed etherigbe.c (in igberballoc) was recently changed to increment the refcount on the block it allocates. Any reason it uses _xinc rather than incref? -- vs
[9fans] A trace of poolalloc in-kernel
Hi, I've gotten and graphed a quick trace of all requests in-kernel to poolalloc during a session with my terminal, where I booted up, created a few windows, built a kernel, and shutdown. http://grex.org/~vsrinivas/p9malloc-traces/ (There's even some eye-candy!) Quick summary: 50% of all requests to poolalloc in kernel during this session were under 50 bytes. Sadness. Hopefully this is of some interest. -- vs
[9fans] Standalone package of command-line sam for unix
Hi, I have constructed a standalone package of Sam (command-line half only) for UNIX, for systems that you don't want to / can't build Plan 9 Ports for and wish to connect to with samterm via ssh. http://grex.org/~vsrinivas/src/sam.tar.bz2 I extracted enough of lib9/libkern, libbio, and libutf from Go and Inferno to get this working; its a very hacky package atm, but I hope its useful to somebody; I use it already. -- vs
[9fans] Its not Venti...
http://www.peereboom.us/epitome/ -- vs
Re: [9fans] Practical issue on using rc shell
The other (Byron Rakitzis) unix port of rc can be linked against either readline or editline. -- vs
[9fans] Thanks for a neat IWP9!
Hi, Thanks to everyone who attended and to Erik Quanstrom and Coraid for a rockin' IWP9. -- vs
[9fans] iwp9: getting from Atlanta to Athens
Hi, For those of us traveling to IWP9, what are recommended ways to get from Atlanta to Athens? We were likely going to Atlanta by train... Thanks, -- vs
Re: [9fans] remedial sources auth, connect, and mount in plan9port
Hi, In order to construct an authenticated mount of sources, you will need to start factotum, use srv -a to create an auth-ed connection to the server and to "post" it, and to mount the "posted" connection. (assuming you have a working plan9port install and are on a unix): $ 9 factotum (start factotum in the current 'namespace'. p9p's current namespace is the value of the NAMESPACE environmental variable). $ srv -a sources.cs.bell-labs.com (I am prompted for my userid and password on the server) user[venkatesh]: [mount step; I use v9fs.You will have to adapt if you're using 9pfuse or 9mount] $ sudo mount -t 9p $NAMESPACE/sources.cs.bell-labs.com -o trans=unix Good luck, -- vs
Re: [9fans] segattach off-by-one?
So in my example, va = 0x10001001, len = 0x1000. I understood that to mean [0x10001001, 0x10002001) was the newly-valid interval, which would mean 0x10002000 was a valid address... The segattach manpage says va+len is 'rounded up'; wouldn't that mean the expanded interval was [0x10001000, 0x10003000)? Or am I misreading the manpage? Thanks, -- vs
[9fans] segattach off-by-one?
Hi,
This little program:
#include
#include
#define SEGBASE ((char *) 0x10001001)
#define SEGSIZE 0x1000
void main(void) {
segattach(0, "shared", SEGBASE, SEGSIZE);
// Works fine (writing to 0x10001fff)
*(char *) (SEGBASE + SEGSIZE - 2) = 'a';
// Suicide! (writing to 0x10002000)
*(char *) (SEGBASE + SEGSIZE - 1) = 'a';
}
However, segattach's manpage claims: "... and va+len is rounded up."
Shouldn't the second page here be mapped?
I propose this patch to /sys/src/9/port/segment.c::
--- segment.c.orig 2009-09-24 22:41:59.0 -0400
+++ segment.c 2009-09-24 22:38:25.0 -0400
@@ -641,6 +641,11 @@
int sno;
Segment *s, *os;
Physseg *ps;
+ ulong ova;
+
+ ova = va;
+ va = va&~(BY2PG-1);
+ len += (ova - va);
if(va != 0 && va >= USTKTOP)
error(Ebadarg);
This patch also prevents segattaching to the zero page, which I think
was worth doing...
Could people try this? Comments?
Thanks,
-- vs
Re: [9fans] fossil 'sync' lockup?
On Sun, Sep 6, 2009 at 11:54 AM, erik quanstrom wrote: >> I executed the fossil 'sync' command when my terminal was under >> moderate I/O load earlier today. Fossil seems to have locked up. Has >> anyone seen anything like this? The locking I can see along the path of 'sync' is vtlock (exclusive lock) of the Epoch lock in fsSync, the per-file locks in fileMetaFlush, and the cache's dirty lock and main lock in cacheFlush. Luckily VtLocks know their holder... Why are VtLocks in liboventi? And why does liboventi have an implementation of a subset of the thread library (rwlocks and rendezvous?) >> I have a dump of the first 256mb of kernel memory and all physical >> memory. Are there any structures that are kept at reasonably constant >> places in memory, so that I can start trawling them to see what fossil >> was doing when it died? > > how did you generate this dump? i've never done that. I was running my terminal in qemu. Switch to the monitor, pmemsave and memsave. > the short answer is that any non-automatic variables have > fixed address. for example, the process table is here > port/proc.c:/^procalloc in .arena Okay, thanks! Luckily nm worked with 9pcf and I found procalloc; I was going to start at the mach structure (always at KZERO + 0x15000) , find the current process structure's address, and then trawl memory for processes around it till I found fossil. > i hope i'm missing something. I hope so too. grep and seeking to the right offsets in a dump was not my least favorite thing ever, but its close. Thanks, -- vs
[9fans] fossil 'sync' lockup?
Hi, I executed the fossil 'sync' command when my terminal was under moderate I/O load earlier today. Fossil seems to have locked up. Has anyone seen anything like this? I have a dump of the first 256mb of kernel memory and all physical memory. Are there any structures that are kept at reasonably constant places in memory, so that I can start trawling them to see what fossil was doing when it died? Thanks, -- vs
Re: [9fans] replacing sources
Thanks! How long has the current sources server been running? -- vs
Re: [9fans] Interested in improving networking in Plan 9
On Sun, Aug 30, 2009 at 1:07 PM, erik quanstrom wrote: >> Simple example: file systems on Plan 9 are slow. Why is that? How do >> they work? How would you go about finding out how to make them faster? > > which ones? there are quite a number to choose from. > i've found that ken's fs beats the pants off nfs on similar > hardware with literally one cpu tied behind its back. Try this - build the source to charon over a 200ms link over 9p. Then try again over sshfs. Also, look at a single terminal with a local fossil install. Trace the path of an 'ls /'. Count the number of copies and context switches. Having the fastest file server in the world means nothing for file system performance when your path to it is a maze of twisty passages, all alike. -- vs
Re: [9fans] new 9atom.iso
Did you use the plan 9 kencc to build that inferno kernel? As I understand, inferno's 8c doesn't have the H5 option... -- vs
Re: [9fans] new 9atom.iso
On Fri, Aug 28, 2009 at 4:54 AM, hiro<23h...@googlemail.com> wrote: > perhaps we should try to boot plan 9 from a linux kernel? Sounds great to > me... > > this probably makes me a troll... > I hear ron minnich did that with his lguest port. Does that make him a troll? -- vs
Re: [9fans] Possible bug in p9p venti?
On Tue, Aug 25, 2009 at 8:08 PM, erik quanstrom wrote: >> >> Any chance this is related to the issues we discussed on #plan9? >> No, I'm pretty sure not. This dealt with a relatively rare case, a filesystem with an optimal read size larger than a constant in the Venti source. The constant was 64K, which covered every FS/OS cross I know of except ZFS on FreeBSD. > > since not everyone who reads the list does irc, why > don't you fill us in on the issues discussed on #plan9? > The problem discussed in IRC relates to p9p vac's -a option, which chains together vac trees to construct something like a dump filesystem on unix. The first problem was that a vac -a hadn't be run in a while, and the newly added tree has some problem with it (either stored incorrectly or being interpreted incorrectly...), which led to it being impossible to traverse or base further vacs on. The person in irc also had a problem where they blew away their Venti index, rebuilt the index, and had checkindex still report errors. That problem is pretty exciting too... -- vs
[9fans] Possible bug in p9p venti?
Hi,
I think I've found a bug in p9p's Venti, if anyone were to take a look
at this code or tell me if I'm on the right track, it'd be pretty
neat.
When trying to start Venti on a FreeBSD 8-BETA2 system and a ZFS
filesystem, we got this error:
2009/0822 23:12:30 venti: conf...
2009/0822 23:12:30 err 4: read arena0 offset 0x6 count 65536 buf
803f2 returned 65536: No such file or directory
2009/0822 23:12:30 err 4: can't read arena0: read arena0 offset
0x6 count 65536 buf 803f2 returned 65536: No such file or
directory
venti: can't init server: can't initialize venti: arena0: can't read
arena0: read arena0 offset 0x6 count 65536 buf 803f2 returned
65536: No such
In venti/srv/part.c:325, opsize is clamped at MaxIo, which is 64KB by
default. Later, on :329, we issue a pread for opsize bytes from our
partition, but check that the read returned some multiple of
blocksize. (Blocksize comes from part->fsblocksize, which is gotten by
fstatfs()'s .f_bsize on unix.) This is a problem when MaxIo <
blocksize ; on ZFS on FreeBSD, f_bsize = 128K.
A workaround is to clamp fsblocksize at MaxIo as well:
--- part.c.orig 2009-08-23 11:07:56.0 -0400
+++ part.c 2009-08-23 11:06:46.0 -0400
@@ -166,6 +166,9 @@
part->fsblocksize = sfs.f_bsize;
}
#endif
+
+ part->fsblocksize = min(part->fsblocksize, MaxIo);
+
if(subname && findsubpart(part, subname) < 0){
werrstr("cannot find subpartition %s", subname);
freepart(part);
Thanks,
-- vs
[9fans] trouble with venti tools and large arenapart...
I'm having trouble with venti's tools, creating an arena partition greater than 89.6GB... anyone seen anything like this before? m...@centaur:~/u% dd if=/dev/zero of=arenas bs=1024k count=1 seek=89500 m...@centaur:~/u% dd if=/dev/zero of=isect bs=1024k count=1 seek=4900 m...@centaur:~/u% fmtisect isect isect fmtisect isect: 627,231 buckets of 215 entries, 524,288 bytes for index map m...@centaur:~/u% fmtarenas arenas arenas fmtarenas arenas: 175 arenas, 93,847,805,952 bytes storage, 524,288 bytes for index map m...@centaur:~/u% fmtindex venti.conf fmtindex: 175 arenas, 627,186 index buckets, 93,844,938,752 bytes storage m...@centaur:~/u% m...@centaur:~/u% rm arenas isect m...@centaur:~/u% dd if=/dev/zero of=arenas bs=1024k count=1 seek=89700 m...@centaur:~/u% dd if=/dev/zero of=isect bs=1024k count=1 seek=4900 m...@centaur:~/u% fmtisect isect isect fmtisect isect: 627,231 buckets of 215 entries, 524,288 bytes for index map m...@centaur:~/u% fmtarenas arenas arenas fmtarenas arenas: 176 arenas, 94,057,521,152 bytes storage, 524,288 bytes for index map m...@centaur:~/u% fmtindex venti.conf 2009/0822 22:27:10 err 2: arena has wrong magic number: expected ArenaMagic (0xf2a14ead) 2009/0822 22:27:10 err 2: arena header or trailer corrupted 2009/0822 22:27:10 err 2: arenas171: arena header or trailer corrupted fmtindex: can't initialize config venti.conf: arenas: arenas171: arena header or trailer corrupted m...@centaur:~/u% Thanks, -- vs
Re: [9fans] venti/copy -m
Thanks for the answer, In p9's venti/copy, in scoretreecmp, there are two casts, from Avl* to ScoreTree*; this depends on scoretree's avl being the first member of the structure; I thought kencc was allowed to reorder structure members, is that the case? Thanks, -- vs
[9fans] venti/copy question....
Hi, I have a few score trees in Venti that p9p's venti/copy wouldn't copy, but p9's would. venti/copy aborted with: copy: reading block (type 16): read asked for got da39a3ee5e6b4b0d3255bfef95601890afd80709 P9's venti/copy has a test, in its walk() function, as to whether a score is all-zeros. P9P's venti/copy doesn't. Adding this test allows my copy to proceed. Is there any reason p9's venti/copy has this test? It seems like a dubious one, since the all-zero score shouldn't ever be in Venti. But it's also necessary to copy these trees... Thanks, -- vs
[9fans] venti/copy -m
Hi, Plan 9's venti/copy has an undocumented -m option. What does it do? Thanks, -- vs
