Re: [9fans] 9grid again

[arisawa]

Hello 9fans.

I have started a new 9grid server with some services.
Please look http://p9.nyx.link/9grid2/9grid.html for details.
This is an experimental service.
If you noticed something bad, please inform me.

Kenji Arisawa

> 2015/11/12 16:30、arisawa  のメール：
> 
> hello,
> 
> I am considering a new project that is related to big data.
> assume we have data on several distributed server.
> current common approach is:
> downloading these data from servers to local side and process them.
> however data is big but program is small. 
> if the data is 1GB order, this approach is practically infeasible.
> 9grid enables opposite approach:
> login to servers and process the data there and gather the results.
> we have all the tools that are necessary to do this as much as 10 years ago.
> what is lacking is target data.
> any idea?
> 
> Kenji Arisawa
> 
> 

Re: [9fans] 9grid?

[erik quanstrom]

On Wed Oct 24 03:32:07 EDT 2012, al...@pbrane.org wrote:
> John Floren  once said:
> > Write a basic http server for Plan 9 (in C) and run Apache Benchmark
> > against it. Somewhere around 100 concurrent connections, I tend to get
> > failure. There's code in /sys/src/9/ip that has a hard limit on the #
> > of concurrent connections IIRC.
> 
> What happens when it fails? /sys/src/9/ip/ip.h:/Nchans isn't that low.
> Do you get the "garbage collecting Convs" message from the kernel?

tcp uses scalednconv() which is basically Nchans*4.

- erik

Re: [9fans] 9grid?

[Anthony Martin]

John Floren  once said:
> Write a basic http server for Plan 9 (in C) and run Apache Benchmark
> against it. Somewhere around 100 concurrent connections, I tend to get
> failure. There's code in /sys/src/9/ip that has a hard limit on the #
> of concurrent connections IIRC.

What happens when it fails? /sys/src/9/ip/ip.h:/Nchans isn't that low.
Do you get the "garbage collecting Convs" message from the kernel?

If you have stack traces or profiles, please post them on golang-dev
or send them directly to me. I'll try to get any bugs fixed promptly.

This goes for any other bugs encountered using Go on Plan 9.

Cheers,
  Anthony

Re: [9fans] 9grid?

[Bakul Shah]

If I wanted to be insulted, I'd subscribe to a Reddit feed. ;)   

On Tue, 23 Oct 2012 18:42:48 PDT "Don A. Bailey" 
  wrote:
> If only Joseph Campbell were around to accurately define this lot. I think
> he'd get a bigger kick out of 9fans than he did the Grateful Dead. Well,
> maybe not. But close, I'm sure.
> 
> D
> 
> On Tue, Oct 23, 2012 at 6:34 PM, erik quanstrom wrote:
> 
> > On Tue Oct 23 21:30:08 EDT 2012, d...@capitolhillconsultants.com wrote:
> >
> > > Haha, it's good to know the caustic wit of 9fans hasn't changed in the
> > > twelve years I've participated in it. Screen names change, but trolls
> > will
> > > always persist.
> >
> > biologists call it an ecological niche.

Egological.

Re: [9fans] 9grid?

[Kurt H Maier]

On Tue, Oct 23, 2012 at 06:29:19PM -0700, Don A. Bailey wrote:
> Haha, it's good to know the caustic wit of 9fans hasn't changed in the
> twelve years I've participated in it. Screen names change, but trolls will
> always persist.

Unlike your desire to look at code?  Why have you suddenly begun asking
questions that can easily be answered by opening source files?  Are you
under coercion?  Are you sending us a signal?





blink twice for yes, they can't record everything

Re: [9fans] 9grid?

[Don A. Bailey]

If only Joseph Campbell were around to accurately define this lot. I think
he'd get a bigger kick out of 9fans than he did the Grateful Dead. Well,
maybe not. But close, I'm sure.

D

On Tue, Oct 23, 2012 at 6:34 PM, erik quanstrom wrote:

> On Tue Oct 23 21:30:08 EDT 2012, d...@capitolhillconsultants.com wrote:
>
> > Haha, it's good to know the caustic wit of 9fans hasn't changed in the
> > twelve years I've participated in it. Screen names change, but trolls
> will
> > always persist.
>
> biologists call it an ecological niche.
>
> - erik
>
>


-- 
Don A. Bailey
CEO/Founding Partner
Capitol Hill Consultants LLC
1-303-947-6557

Re: [9fans] 9grid?

[erik quanstrom]

On Tue Oct 23 21:30:08 EDT 2012, d...@capitolhillconsultants.com wrote:

> Haha, it's good to know the caustic wit of 9fans hasn't changed in the
> twelve years I've participated in it. Screen names change, but trolls will
> always persist.

biologists call it an ecological niche.

- erik

Re: [9fans] 9grid?

[erik quanstrom]

> Write a basic http server for Plan 9 (in C) and run Apache Benchmark
> against it. Somewhere around 100 concurrent connections, I tend to get
> failure. There's code in /sys/src/9/ip that has a hard limit on the #
> of concurrent connections IIRC.
> 
> I'd post the code for the server I wrote, but it was written as part
> of work so I can't. Still, it's not hard to put together a server
> which responds only to a GET.

the limit is 1024 for a terminal, and 4096 for a cpu server.
if you get a failure at a lower number, then it's not because
of concurrent connections, but something else.

i know the original nix tcp code had trouble with zero-window
probes.  i'm not sure if the same was ever true of the distribution.

- erik

Re: [9fans] 9grid?

[Don A. Bailey]

Haha, it's good to know the caustic wit of 9fans hasn't changed in the
twelve years I've participated in it. Screen names change, but trolls will
always persist.

<3
D

On Tue, Oct 23, 2012 at 6:26 PM, Kurt H Maier  wrote:

> On Tue, Oct 23, 2012 at 06:11:54PM -0700, Don A. Bailey wrote:
> > If I wanted to be insulted, I'd subscribe to a Reddit feed. ;)
> >
> > D
>
>
> Thanks for letting us know, D.
>
> The Right Reverend Kurt H. Maier, Esq.
> Daughter of the Fifth House of Betazed
> Holder of the Sacred Chalice of Rixx
> Heir to the Holy Rings of Betazed
> Ferenginar Acquisitions, LLC
> 1-631-960-7187
>
>


-- 
Don A. Bailey
CEO/Founding Partner
Capitol Hill Consultants LLC
1-303-947-6557

Re: [9fans] 9grid?

[Kurt H Maier]

On Tue, Oct 23, 2012 at 06:11:54PM -0700, Don A. Bailey wrote:
> If I wanted to be insulted, I'd subscribe to a Reddit feed. ;)
> 
> D


Thanks for letting us know, D.

The Right Reverend Kurt H. Maier, Esq.
Daughter of the Fifth House of Betazed
Holder of the Sacred Chalice of Rixx
Heir to the Holy Rings of Betazed
Ferenginar Acquisitions, LLC
1-631-960-7187

Re: [9fans] 9grid?

[Don A. Bailey]

Thankfully, we are very well versed at tuning the Plan 9 kernel. I'll take
your advice and develop the benchmark.

Is your code concealed per work on NxM? Just curious as NxM seems to solve
(in what little I know about it) some of the issues we are trying to solve
with Go+Plan 9.

D

On Tue, Oct 23, 2012 at 6:15 PM, John Floren  wrote:

> On Tue, Oct 23, 2012 at 5:41 PM, Matthew Veety  wrote:
> > On 10/23/2012 8:11 PM, Don A. Bailey wrote:
> >>
> >> Go embeds parallel/grid functionality now instead of just lightweight
> >> thread execution? Which packages would you point me at?
> >>
> >> Thanks,
> >> D
> >>
> >> On Tue, Oct 23, 2012 at 5:09 PM, ron minnich  >> > wrote:
> >>
> >> On Tue, Oct 23, 2012 at 1:22 PM, Don A. Bailey
> >>  >> > wrote:
> >>
> >>  > I'm interested in the code for managing grid nodes and delegating
> >> tasks.
> >>
> >> Real code? talk to charles.
> >>
> >> Or now that Go works, you could look at some of those packages.
> >>
> >> ron
> >>
> >>
> >>
> >>
> >> --
> >> Don A. Bailey
> >> CEO/Founding Partner
> >> Capitol Hill Consultants LLC
> >> 1-303-947-6557
> >>
> >>
> >
> > I would avoid using Go on Plan 9 right now for anything production
> because
> > it has issues when using many concurrent tcp connections. If you do want
> to
> > use Go, stick with reading and writing files, and let 9P do it's thing.
> >
> > --
> > Veety
> >
> >
>
> Write a basic http server for Plan 9 (in C) and run Apache Benchmark
> against it. Somewhere around 100 concurrent connections, I tend to get
> failure. There's code in /sys/src/9/ip that has a hard limit on the #
> of concurrent connections IIRC.
>
> I'd post the code for the server I wrote, but it was written as part
> of work so I can't. Still, it's not hard to put together a server
> which responds only to a GET.
>
> john
>
>


-- 
Don A. Bailey
CEO/Founding Partner
Capitol Hill Consultants LLC
1-303-947-6557

Re: [9fans] 9grid?

[Don A. Bailey]

Cool, thanks. I appreciate the response.

D

On Tue, Oct 23, 2012 at 5:51 PM, Matthew Veety  wrote:

> On 10/23/2012 8:44 PM, Don A. Bailey wrote:
>
>> Does Go have issues in general with TCP connections, or is this a Plan 9
>> specific issue?
>>
>> D
>>
>
> I haven't had any problems with Go on other platforms (FreeBSD and Mac OS
> X) except for CPU use sometimes and scheduling of goroutines. Both you can
> kinda work around.
>
> --
> Veety
>
>
>
>


-- 
Don A. Bailey
CEO/Founding Partner
Capitol Hill Consultants LLC
1-303-947-6557

Re: [9fans] 9grid?

[John Floren]

On Tue, Oct 23, 2012 at 5:41 PM, Matthew Veety  wrote:
> On 10/23/2012 8:11 PM, Don A. Bailey wrote:
>>
>> Go embeds parallel/grid functionality now instead of just lightweight
>> thread execution? Which packages would you point me at?
>>
>> Thanks,
>> D
>>
>> On Tue, Oct 23, 2012 at 5:09 PM, ron minnich > > wrote:
>>
>> On Tue, Oct 23, 2012 at 1:22 PM, Don A. Bailey
>> > > wrote:
>>
>>  > I'm interested in the code for managing grid nodes and delegating
>> tasks.
>>
>> Real code? talk to charles.
>>
>> Or now that Go works, you could look at some of those packages.
>>
>> ron
>>
>>
>>
>>
>> --
>> Don A. Bailey
>> CEO/Founding Partner
>> Capitol Hill Consultants LLC
>> 1-303-947-6557
>>
>>
>
> I would avoid using Go on Plan 9 right now for anything production because
> it has issues when using many concurrent tcp connections. If you do want to
> use Go, stick with reading and writing files, and let 9P do it's thing.
>
> --
> Veety
>
>

Write a basic http server for Plan 9 (in C) and run Apache Benchmark
against it. Somewhere around 100 concurrent connections, I tend to get
failure. There's code in /sys/src/9/ip that has a hard limit on the #
of concurrent connections IIRC.

I'd post the code for the server I wrote, but it was written as part
of work so I can't. Still, it's not hard to put together a server
which responds only to a GET.

john

Re: [9fans] 9grid?

[Don A. Bailey]

If I wanted to be insulted, I'd subscribe to a Reddit feed. ;)

D

On Tue, Oct 23, 2012 at 6:03 PM, Kurt H Maier  wrote:

> On Tue, Oct 23, 2012 at 05:44:42PM -0700, Don A. Bailey wrote:
> > Does Go have issues in general with TCP connections, or is this a Plan 9
> > specific issue?
> >
> > D
>
> Dear Don A. "D" Bailey, CEO/Cofounder:
>
> You should probably harass the Go people about this.
>
> Thank you,
> Kurt H Maier
> Grand Poobah/Arch-inquisitor At Large
> Totally a Business, Inc
> 1-212-479-7990
>
>


-- 
Don A. Bailey
CEO/Founding Partner
Capitol Hill Consultants LLC
1-303-947-6557

Re: [9fans] 9grid?

[Kurt H Maier]

On Tue, Oct 23, 2012 at 05:44:42PM -0700, Don A. Bailey wrote:
> Does Go have issues in general with TCP connections, or is this a Plan 9
> specific issue?
> 
> D

Dear Don A. "D" Bailey, CEO/Cofounder:

You should probably harass the Go people about this.

Thank you,
Kurt H Maier
Grand Poobah/Arch-inquisitor At Large
Totally a Business, Inc
1-212-479-7990

Re: [9fans] 9grid?

[Matthew Veety]

On 10/23/2012 8:44 PM, Don A. Bailey wrote:

Does Go have issues in general with TCP connections, or is this a Plan 9
specific issue?

D


I haven't had any problems with Go on other platforms (FreeBSD and Mac 
OS X) except for CPU use sometimes and scheduling of goroutines. Both 
you can kinda work around.


--
Veety

Re: [9fans] 9grid?

[Don A. Bailey]

Does Go have issues in general with TCP connections, or is this a Plan 9
specific issue?

D

On Tue, Oct 23, 2012 at 5:41 PM, Matthew Veety  wrote:

> On 10/23/2012 8:11 PM, Don A. Bailey wrote:
>
>> Go embeds parallel/grid functionality now instead of just lightweight
>> thread execution? Which packages would you point me at?
>>
>> Thanks,
>> D
>>
>> On Tue, Oct 23, 2012 at 5:09 PM, ron minnich > > wrote:
>>
>> On Tue, Oct 23, 2012 at 1:22 PM, Don A. Bailey
>> 
>> 
>> >
>> wrote:
>>
>>  > I'm interested in the code for managing grid nodes and delegating
>> tasks.
>>
>> Real code? talk to charles.
>>
>> Or now that Go works, you could look at some of those packages.
>>
>> ron
>>
>>
>>
>>
>> --
>> Don A. Bailey
>> CEO/Founding Partner
>> Capitol Hill Consultants LLC
>> 1-303-947-6557
>>
>>
>>
> I would avoid using Go on Plan 9 right now for anything production because
> it has issues when using many concurrent tcp connections. If you do want to
> use Go, stick with reading and writing files, and let 9P do it's thing.
>
> --
> Veety
>
>
>


-- 
Don A. Bailey
CEO/Founding Partner
Capitol Hill Consultants LLC
1-303-947-6557

Re: [9fans] 9grid?

[Matthew Veety]

On 10/23/2012 8:11 PM, Don A. Bailey wrote:

Go embeds parallel/grid functionality now instead of just lightweight
thread execution? Which packages would you point me at?

Thanks,
D

On Tue, Oct 23, 2012 at 5:09 PM, ron minnich mailto:rminn...@gmail.com>> wrote:

On Tue, Oct 23, 2012 at 1:22 PM, Don A. Bailey
mailto:d...@capitolhillconsultants.com>> wrote:

 > I'm interested in the code for managing grid nodes and delegating
tasks.

Real code? talk to charles.

Or now that Go works, you could look at some of those packages.

ron




--
Don A. Bailey
CEO/Founding Partner
Capitol Hill Consultants LLC
1-303-947-6557




I would avoid using Go on Plan 9 right now for anything production 
because it has issues when using many concurrent tcp connections. If you 
do want to use Go, stick with reading and writing files, and let 9P do 
it's thing.


--
Veety

Re: [9fans] 9grid?

[Don A. Bailey]

Go embeds parallel/grid functionality now instead of just lightweight
thread execution? Which packages would you point me at?

Thanks,
D

On Tue, Oct 23, 2012 at 5:09 PM, ron minnich  wrote:

> On Tue, Oct 23, 2012 at 1:22 PM, Don A. Bailey
>  wrote:
>
> > I'm interested in the code for managing grid nodes and delegating tasks.
>
> Real code? talk to charles.
>
> Or now that Go works, you could look at some of those packages.
>
> ron
>
>


-- 
Don A. Bailey
CEO/Founding Partner
Capitol Hill Consultants LLC
1-303-947-6557

Re: [9fans] 9grid?

[ron minnich]

On Tue, Oct 23, 2012 at 1:22 PM, Don A. Bailey
 wrote:

> I'm interested in the code for managing grid nodes and delegating tasks.

Real code? talk to charles.

Or now that Go works, you could look at some of those packages.

ron

Re: [9fans] 9grid?

[Don A. Bailey]

I only really considered one project to be substantial. That may be unfair.
It was the one that LANL/UCal were involved in.

I'm interested in the code for managing grid nodes and delegating tasks.

Best,
D

On Tue, Oct 23, 2012 at 7:46 AM, Anthony Sorace  wrote:

> That name has referred to at least two, possibly three,
> distinct and unrelated projects. I don't believe any of
> them are still ongoing. I also don't think any of them
> were aimed at becoming what i'd call a "production
> resource". What is it you're looking for?
>
>
>


-- 
Don A. Bailey
CEO/Founding Partner
Capitol Hill Consultants LLC
1-303-947-6557

Re: [9fans] 9grid?

[Anthony Sorace]

That name has referred to at least two, possibly three,
distinct and unrelated projects. I don't believe any of
them are still ongoing. I also don't think any of them
were aimed at becoming what i'd call a "production
resource". What is it you're looking for?

Re: [9fans] 9grid

[lupin636]

thanks a lot eric..
what do you mean with "match"?
>
> > Hi Eric, (sic)
> > I did what you told me, and i got the same as you wrote below.
> > if you still have any idea, it will be helpful.
> > thanks again,
>
> you need to verify the keys also match.
>
> - erik

Re: [9fans] 9grid

[erik quanstrom]

On Mon Nov 17 09:17:49 EST 2008, [EMAIL PROTECTED] wrote:
> Hi Eric, (sic)
> I did what you told me, and i got the same as you wrote below.
> if you still have any idea, it will be helpful.
> thanks again,

you need to verify the keys also match.

- erik

Re: [9fans] 9grid

[lupin636]

Hi Eric,
I did what you told me, and i got the same as you wrote below.
if you still have any idea, it will be helpful.
thanks again,

Armando.

> looking in /mnt/factotum/ctl on the console, you should see
> keys like these at a minimum
>
> terminal:
> key proto=p9sk1 dom=quanstro.net user=quanstro !password?
>
> cpu server and file server:
> key proto=p9sk1 user=bootes dom=quanstro.net !hex? !password?
>
> of course you should substitute your authentication domain
> (which may be the same text but is different from your dns domain)
> for quanstro.net
>
> hope that gets you a bit further.
>
> - erik

Re: [9fans] 9grid

[erik quanstrom]

> my /lib/ndb/auth is:
>hostid=bootes
>uid=!sys uid=!adm uid=*
> but i just remember that "armando"(user name) is also sys and adm,
> could it be the problem for "speaks for"?
> because, as i explained above i have "no speaks for" in /sys/log/auth
> when i try cpu(1) command from terminal, i also have /mnt/factotum/
> log:
> 14: no key matches proto=p9sk1 role=server dom?
> 14: failure no key matches proto=p9sk1 role=server dom?

looking in /mnt/factotum/ctl on the console, you should see
keys like these at a minimum

terminal:
key proto=p9sk1 dom=quanstro.net user=quanstro !password?

cpu server and file server:
key proto=p9sk1 user=bootes dom=quanstro.net !hex? !password?

of course you should substitute your authentication domain
(which may be the same text but is different from your dns domain)
for quanstro.net

hope that gets you a bit further.

- erik

Re: [9fans] 9grid

[lupin636]

Hi All,

my /lib/ndb/auth is:
   hostid=bootes
   uid=!sys uid=!adm uid=*
but i just remember that "armando"(user name) is also sys and adm,
could it be the problem for "speaks for"?
because, as i explained above i have "no speaks for" in /sys/log/auth
when i try cpu(1) command from terminal, i also have /mnt/factotum/
log:
14: no key matches proto=p9sk1 role=server dom?
14: failure no key matches proto=p9sk1 role=server dom?
suggestions please?
thanks in advance for all responses.

Armando

Re: [9fans] 9grid

[lupin636]

Hi Eric,

I did every thing you told me, but when i try again:
   fsname%  ndb/query -f /lib/ndb/auth hostid bootes
   fsname%
without no response.
do you still have any idea??...because i'm thinkig to take a
drill ;-)))
thank you very much for your time.

Armando


> easy fix
>         9fs sources && cp /n/sources/plan9/lib/ndb/auth /lib/ndb/auth
>
> - erik

Re: [9fans] 9grid

[erik quanstrom]

On Fri Nov 14 04:48:04 EST 2008, [EMAIL PROTECTED] wrote:
> On Nov 13, 8:45 am, [EMAIL PROTECTED] (erik quanstrom) wrote:
> ...
> > cpu -h node -c 'name=`{cat ''#c/sysname''}; echo do something with $name'
> 
> 
> I added 'role=cpu' to my cpu servers in /lib/ndb/local, now I can do
> this
> 
> for (node in `{ndb/query -a role cpu sys})
>   cpu -h $node -c 'name=`{cat ''#c/sysname''}; echo do something with
> $name'
> 
> 
> torsbohn - Nothing can stop Tor!

not sure what the deal with these emails is.
they're not from gmail and they contain
invalid quoted-printable, which is why things
look so bad.

i've got a patch for upas/fs that i'll put up shortly.
but you may just want to apply this yourself since
you're probablly not running nupas:

mbox.c:/^hex2int should change:

static int
hex2int(int x)
{
if(x >= '0' && x <= '9')
return x - '0';
if(x >= 'A' && x <= 'F')
return x - 'A' + 10;
if(x >= 'a' && x <= 'f')
return x - 'a' + 10;
return -1;
}

then :/^decquotedline should have the case '='
changed like so

case '=':
c = hex2int(*in++)<<4;
c |= hex2int(*in++);
if(c != -1)
*out++ = c;
else{
*out++ = '=';
in -= 2;
}
break;

- erik

Re: [9fans] 9grid

[erik quanstrom]

> Hi,
> I just did it:
>fsname%  ndb/query -f /lib/ndb/auth hostid bootes
>fsname%
> i got no response,only the fs prompt..
> but in /lib/ndb/auth.mio i have the same lines,so:
>hostid°otes
> uid
> dm uid
> ??

easy fix
9fs sources && cp /n/sources/plan9/lib/ndb/auth /lib/ndb/auth

- erik

Re: [9fans] 9grid

[lupin636]

Hi,
I just did it:
   fsname%  ndb/query -f /lib/ndb/auth hostid bootes
   fsname%
i got no response,only the fs prompt..
but in /lib/ndb/auth.mio i have the same lines,so:
   hostid=bootes
uid=!sys uid=!adm uid=*
??

thanks guys,

Armando

> > could it be that the equals sign (=) you typed in /lib/ndb/auth is not
> > the normal equals sign (ascii 3d) but the equal sign of another
> > encoding? that could be the reason why your /lib/ndb/auth can't be
> > pasted properly in an email and can't be parsed correctly by
> > tokenize().
>
> easy test.
>         ndb/query -f /lib/ndb/auth hostid bootes
> should ->
> hostid=bootes uid=!sys uid=!adm uid=*
>
> - erik

Re: [9fans] 9grid

[torsbohn]

On Nov 13, 8:45 am, [EMAIL PROTECTED] (erik quanstrom) wrote:
...
> cpu -h node -c 'name=`{cat ''#c/sysname''}; echo do something with $name'


I added 'role=cpu' to my cpu servers in /lib/ndb/local, now I can do
this

for (node in `{ndb/query -a role cpu sys})
  cpu -h $node -c 'name=`{cat ''#c/sysname''}; echo do something with
$name'


torsbohn - Nothing can stop Tor!

Re: [9fans] 9grid

[erik quanstrom]

> could it be that the equals sign (=) you typed in /lib/ndb/auth is not
> the normal equals sign (ascii 3d) but the equal sign of another
> encoding? that could be the reason why your /lib/ndb/auth can't be
> pasted properly in an email and can't be parsed correctly by
> tokenize().

easy test.
ndb/query -f /lib/ndb/auth hostid bootes
should ->
hostid=bootes uid=!sys uid=!adm uid=* 

- erik

Re: [9fans] 9grid

[andrey mirtchovski]

could it be that the equals sign (=) you typed in /lib/ndb/auth is not
the normal equals sign (ascii 3d) but the equal sign of another
encoding? that could be the reason why your /lib/ndb/auth can't be
pasted properly in an email and can't be parsed correctly by
tokenize().

Re: [9fans] 9grid

[lupin636]

Thanks again Eric..
It works, sorry but i'm newbie in shell ;-)
About the authentication problem, i checked /sys/log/auth, and i
noticed that there are some lines with "no speaks for",like this:

fs name nov 13 18:50:09 tr-fail [EMAIL PROTECTED] (NODE ip address) ->
[EMAIL PROTECTED] no speaks for
fs name nov 13 18:50:10 tr-ok [EMAIL PROTECTED] (NODE ip address) ->
[EMAIL PROTECTED]

this happens when i try:cpu -h NODE
could it be the problem?
thanks,

Armando

> > I did it and it works, but do you have any idea why i can do it from
> > file server as bootes but not from terminal as armando?
>
> there's probablly something wrong in your authentication setup.
>
> >      fs name% cpu -h NODE -c 'name=(equal sign)cat ''#c/sysname'';
> > echo'
>
> cpu -h node -c 'name=`{cat ''#c/sysname''}; echo do something with $name'
>
> - erik

Re: [9fans] 9grid

[lupin636]

Thanks a lot Eric,
I did it and it works, but do you have any idea why i can do it from
file server as bootes but not from terminal as armando?
Furthermore, i would like to put  that line into a variable, by doing
(maybe in a wrong way):
 fs name% cpu -h NODE -c 'name=(equal sign)cat ''#c/sysname'';
echo'
  /dev/sysname: '/dev/sysname'  permission denied
 fs name%
where is my mistake?
thanks,

Armando

> cpu -c 'cat ''#c/sysname''; echo'
>
> - erik

Re: [9fans] 9grid

[erik quanstrom]

> I did it and it works, but do you have any idea why i can do it from
> file server as bootes but not from terminal as armando?

there's probablly something wrong in your authentication setup.

>  fs name% cpu -h NODE -c 'name=(equal sign)cat ''#c/sysname'';
> echo'

cpu -h node -c 'name=`{cat ''#c/sysname''}; echo do something with $name' 

- erik

Re: [9fans] 9grid

[erik quanstrom]

> if i do this:
> cpu -h NODE -c cmd args
> How can i be really sure that the command was performed by that node?

cpu -c 'cat ''#c/sysname''; echo'

- erik

Re: [9fans] 9grid

[lupin636]

Hi all,

I was trying to use cpu(1) command from file server, because from
terminal as you see the post above i still have problems..
As cpu(1) command build the name space by running /usr/$user/lib/
profile with root of the invoking name space binding it to /mnt/term,
if i do this:
cpu -h NODE -c cmd args
How can i be really sure that the command was performed by that node?
is there some way on /mnt/term that let me know the node used by me?
for example to know the node name..
thanks for responses

Armando.

Re: [9fans] 9grid

[lupin636]

I have the same in lib/ndb/auth, i think google doesn't read the equal
sign  "=" from here, i don't know how you did it..
I don't understand very well what do you mean with "as their root
filesystem", but if you mean the bootargs line in the plan9.ini, they
have the same bootargs line, otherwise i don't really know, can you re-
explain to me?
thank you for you response,

Armando

> No, that's really really wrong.  You need to have this in your
> /lib/ndb/auth:
> hostid=bootes
>                uid=!sys uid=!adm uid=*
>
> Also, are your terminals and CPU servers using the fileserver as their
> root filesystem, the way they should be?
>
> John

Re: [9fans] 9grid

[john]

> Thanks a lot Ron, it was clearly a really nice response.
> You leave in no doubt about using between cpu and ssh.
> I would like to try to do it now, but it semms to me that i have
> authentication problems from terminal, because when i try to do cpu(1)
> command from terminal (log in as Armando) i got nothing, i.e.
> term% cpu -h NODE -c date
> term%
> otherwise by doing:
>  term% cpu -h NODE
>  term%
> i got the same, and /mnt/term is empty, instead i think that cpu's
> name space should be mounted on /mtn/term, isn't it?
> Furthermore i also checked lib/ndb/auth on the file server, and  this
> is what i have:
> hostid°otes
> uidys uid
> dm uid

No, that's really really wrong.  You need to have this in your
/lib/ndb/auth:
hostid=bootes
   uid=!sys uid=!adm uid=*

Also, are your terminals and CPU servers using the fileserver as their
root filesystem, the way they should be?



John

Re: [9fans] 9grid

[lupin636]

Thanks a lot Ron, it was clearly a really nice response.
You leave in no doubt about using between cpu and ssh.
I would like to try to do it now, but it semms to me that i have
authentication problems from terminal, because when i try to do cpu(1)
command from terminal (log in as Armando) i got nothing, i.e.
term% cpu -h NODE -c date
term%
otherwise by doing:
 term% cpu -h NODE
 term%
i got the same, and /mnt/term is empty, instead i think that cpu's
name space should be mounted on /mtn/term, isn't it?
Furthermore i also checked lib/ndb/auth on the file server, and  this
is what i have:
hostid=bootes
uid=!sys uid=!adm uid=*
I think that is correct, is it?
Thank you very much for your patience,

Armando.

> suppose you have a list of nodes
>
> cpu% NODES=(a b c d)
> cpu% echo $NODES
> a b c d
> cpu% for (i in $NODES) {
>         cpu -h $i -c some-command&
>         }
>
> Go ahead. Try it!
>  for (i in $NODES) {
>         cpu -h $i -c date&
>         }
>
> OK, now suppose you have what in the high end business is still called
> an 'input deck'. It's in a weird place. You get to it by saying
> some-command -i input-file
>
> for (i in $NODES) {
>         cpu -h $i -c some-command -i your-file&
>         }
>
> This will work whether there is a mount on those nodes for your home
> directory or not. Comes free with cpu.
>
> What if you for whatever reason want a ps to show all the proces on
> all the nodes you're running on.
>
> for (i in $NODES) {
>  import -a $i .com /proc /proc
>
> }
>
> Your /proc is now the unified /proc of all your nodes. (I used to do
> this all the time with my plan 9 minicluster)
>
> That way, if you want to kill all the some-commands running on ALL your nodes:
> slay some-command | rc
>
> The point being that you only need to run this command on the
> front-end, not on each node.
>
> You just can't even try to do this sort of thing with ssh.
>
> ron

Re: [9fans] 9grid

[ron minnich]

On Tue, Nov 11, 2008 at 4:36 PM, erik quanstrom <[EMAIL PROTECTED]> wrote:

> the trivial solution on your hardware would be to partition
> the pid space, wouldn't it.  just have 64bit pids?  let each
> machine start at a 1<<32 boundary?

Sure. But you have to change the pid type in the kernel and  and and and and

The point here is that with fairly trivial mods to a few programs you
can build a cluster management suite that unix or windows based
cluster tools can not really touch.

But you don't have gcc. That's an issue. Not kidding here. Don't have
a good fortran compiler either. This is where binary support is very
useful.

ron

Re: [9fans] 9grid

[erik quanstrom]

> It never seemed to.
> 
> But of course if you have procs with same pid, the collisions are obvious.
> 
> So, do the easy thing:
> 
> for all nodes, mount them at
> /proc/localhost
> /proc/hostname/whatever
> 
> Then modify ps (takes about 5 minutes) so it iterates over /proc/*
> where * is a set of host names.
> 
> now you can do fun stuff
> slay node8/mpirun | rc
> slay node*/mpirun | rc
> 
> There's a lot of good stuff in there if you want to use it ... I
> actually implemented all this a few years back when Vic did hist first
> xcpu code. It was really nice.

the trivial solution on your hardware would be to partition
the pid space, wouldn't it.  just have 64bit pids?  let each
machine start at a 1<<32 boundary?

four billion machines ought with four billion processes each
ought to be enough for anyone.

- erik

Re: [9fans] 9grid

[ron minnich]

On Tue, Nov 11, 2008 at 4:11 PM, erik quanstrom <[EMAIL PROTECTED]> wrote:
>> What if you for whatever reason want a ps to show all the proces on
>> all the nodes you're running on.
>>
>> for (i in $NODES) {
>>  import -a $i .com /proc /proc
>> }
>
> what's the .com for?
>

it's when I forgot to take part of the test :-)

>> Your /proc is now the unified /proc of all your nodes. (I used to do
>> this all the time with my plan 9 minicluster)
>
> does ps not mind if several processes have the same pid?
>

It never seemed to.

But of course if you have procs with same pid, the collisions are obvious.

So, do the easy thing:

for all nodes, mount them at
/proc/localhost
/proc/hostname/whatever

Then modify ps (takes about 5 minutes) so it iterates over /proc/*
where * is a set of host names.

now you can do fun stuff
slay node8/mpirun | rc
slay node*/mpirun | rc

There's a lot of good stuff in there if you want to use it ... I
actually implemented all this a few years back when Vic did hist first
xcpu code. It was really nice.

ron

Re: [9fans] 9grid

[erik quanstrom]

> What if you for whatever reason want a ps to show all the proces on
> all the nodes you're running on.
> 
> for (i in $NODES) {
>  import -a $i .com /proc /proc
> }

what's the .com for?

> Your /proc is now the unified /proc of all your nodes. (I used to do
> this all the time with my plan 9 minicluster)

does ps not mind if several processes have the same pid?

- erik

Re: [9fans] 9grid

[ron minnich]

On Tue, Nov 11, 2008 at 7:12 AM,  <[EMAIL PROTECTED]> wrote:

> Thanks john, i would like to send simple programs (jobs) to the nodes
> (diskless cpu server) of a 9grid from terminal, and get responses from
> them. How can i do it?
>

suppose you have a list of nodes

cpu% NODES=(a b c d)
cpu% echo $NODES
a b c d
cpu% for (i in $NODES) {
cpu -h $i -c some-command&
}

Go ahead. Try it!
 for (i in $NODES) {
cpu -h $i -c date&
}

OK, now suppose you have what in the high end business is still called
an 'input deck'. It's in a weird place. You get to it by saying
some-command -i input-file

for (i in $NODES) {
cpu -h $i -c some-command -i your-file&
}

This will work whether there is a mount on those nodes for your home
directory or not. Comes free with cpu.

What if you for whatever reason want a ps to show all the proces on
all the nodes you're running on.

for (i in $NODES) {
 import -a $i .com /proc /proc
}

Your /proc is now the unified /proc of all your nodes. (I used to do
this all the time with my plan 9 minicluster)

That way, if you want to kill all the some-commands running on ALL your nodes:
slay some-command | rc

The point being that you only need to run this command on the
front-end, not on each node.

You just can't even try to do this sort of thing with ssh.

ron

Re: [9fans] 9grid

[Uriel]

How cool! Tell me more

Your ideas intrigue me and I wish to subscribe to your newsletter.

uriel

On Tue, Nov 11, 2008 at 5:32 PM,  <[EMAIL PROTECTED]> wrote:
> 9grid is a distributed computing project, which features prominently
> the Plan 9 from Bell Labs operating system
>
> Armando
>
> On 11 Nov, 16:43, [EMAIL PROTECTED] (Uriel) wrote:
>> What is a '9grid'?
>>
>> uriel
>
>

Re: [9fans] 9grid

[lupin636]

9grid is a distributed computing project, which features prominently
the Plan 9 from Bell Labs operating system

Armando

On 11 Nov, 16:43, [EMAIL PROTECTED] (Uriel) wrote:
> What is a '9grid'?
>
> uriel

Re: [9fans] 9grid

[Uriel]

What is a '9grid'?

uriel

On Tue, Nov 11, 2008 at 4:12 PM,  <[EMAIL PROTECTED]> wrote:
> On 11 Nov, 15:46, [EMAIL PROTECTED] wrote:
>> > Ok, i just ran some local  commands from cpu server, and it is ok, i'm
>> > gonna use the cpu servers only like a compute nodes. From cpu server i
>> > wanted to see responses , so  i did and got this:
>> > cpus# ssh terminal name
>> > ssh: dialing terminal name: connection refused
>> > cpus# ssh file server name
>> > ssh: reading server version: unexpected EOF
>> > the second error, i got also when i tried either from terminal or file
>> > server.
>> > yes, bootes is the cpu server's hostowner,but on terminal i log as
>> > Armando.
>> > what do you mean with "configuration error"?where could it be?
>> > is ssh right to launch a task from terminal?
>>
>> > thanks again
>>
>> > Armando
> Thanks john, i would like to send simple programs (jobs) to the nodes
> (diskless cpu server) of a 9grid from terminal, and get responses from
> them. How can i do it?
>
>> You want to be sitting at a terminal, and start a command on a cpu
>> server, right?
>>
>> cpu -h  -c  
>>
>> That will execute the command on the cpu server and leave you at the
>> terminal prompt when you are done.
>>
>> I have no idea why you are trying to ssh from your cpu server to your
>> terminal or to the fileserver.  Forget ssh.  If you have a Plan 9
>> network, ssh is 100% wrong for you.
>>
>> John
>
>

Re: [9fans] 9grid

[lupin636]

On 11 Nov, 15:46, [EMAIL PROTECTED] wrote:
> > Ok, i just ran some local  commands from cpu server, and it is ok, i'm
> > gonna use the cpu servers only like a compute nodes. From cpu server i
> > wanted to see responses , so  i did and got this:
> >         cpus# ssh terminal name
> >         ssh: dialing terminal name: connection refused
> >         cpus# ssh file server name
> >         ssh: reading server version: unexpected EOF
> > the second error, i got also when i tried either from terminal or file
> > server.
> > yes, bootes is the cpu server's hostowner,but on terminal i log as
> > Armando.
> > what do you mean with "configuration error"?where could it be?
> > is ssh right to launch a task from terminal?
>
> > thanks again
>
> > Armando
Thanks john, i would like to send simple programs (jobs) to the nodes
(diskless cpu server) of a 9grid from terminal, and get responses from
them. How can i do it?

> You want to be sitting at a terminal, and start a command on a cpu
> server, right?
>
> cpu -h  -c  
>
> That will execute the command on the cpu server and leave you at the
> terminal prompt when you are done.
>
> I have no idea why you are trying to ssh from your cpu server to your
> terminal or to the fileserver.  Forget ssh.  If you have a Plan 9
> network, ssh is 100% wrong for you.
>
> John

Re: [9fans] 9grid

[john]

> Ok, i just ran some local  commands from cpu server, and it is ok, i'm
> gonna use the cpu servers only like a compute nodes. From cpu server i
> wanted to see responses , so  i did and got this:
> cpus# ssh terminal name
> ssh: dialing terminal name: connection refused
> cpus# ssh file server name
> ssh: reading server version: unexpected EOF
> the second error, i got also when i tried either from terminal or file
> server.
> yes, bootes is the cpu server's hostowner,but on terminal i log as
> Armando.
> what do you mean with "configuration error"?where could it be?
> is ssh right to launch a task from terminal?
> 
> thanks again
> 
> Armando
> 

You want to be sitting at a terminal, and start a command on a cpu
server, right?

cpu -h  -c  

That will execute the command on the cpu server and leave you at the
terminal prompt when you are done.

I have no idea why you are trying to ssh from your cpu server to your
terminal or to the fileserver.  Forget ssh.  If you have a Plan 9
network, ssh is 100% wrong for you.


John

Re: [9fans] 9grid

[lupin636]

Ok, i just ran some local  commands from cpu server, and it is ok, i'm
gonna use the cpu servers only like a compute nodes. From cpu server i
wanted to see responses , so  i did and got this:
cpus# ssh terminal name
ssh: dialing terminal name: connection refused
cpus# ssh file server name
ssh: reading server version: unexpected EOF
the second error, i got also when i tried either from terminal or file
server.
yes, bootes is the cpu server's hostowner,but on terminal i log as
Armando.
what do you mean with "configuration error"?where could it be?
is ssh right to launch a task from terminal?

thanks again

Armando

> i assume poor spelling and cut-n-paste failure. :-)
> the key thing here is if the only hostid in /lib/ndb/auth
> is bootes, then the cpu server's hostowner must be
> bootes.  is it?  can you run commands from the cpu
> server's console?
>
> you're doing the right thing, you've just got a
> configuration error.
>
> - erik

Re: [9fans] 9grid

[erik quanstrom]

> Hi Eric,
> I check lib/ndb/auth in the file server, this is what i have:
> hostid°otes
> uid
> dm uid

i assume poor spelling and cut-n-paste failure. :-)
the key thing here is if the only hostid in /lib/ndb/auth
is bootes, then the cpu server's hostowner must be
bootes.  is it?  can you run commands from the cpu
server's console?

you're doing the right thing, you've just got a
configuration error.

- erik

Re: [9fans] 9grid

[lupin636]

Hi Eric,
I check lib/ndb/auth in the file server, this is what i have:
hostid=bootes
uid=!sys uid=!adm uid=*
I think that is correct, does it?
I'd want to launch a task from terminal (logged as Me) to a node of
the 9grid, i was trying with cpu, ssh; because i realize that i don't
need to connect to a node (diskless cpu server) for launching a task
to it, i'd just want to launch some tasks over the nodes of the 9 grid
and get a response from them. maybe it seems easy but i can't.
Thanks a lot for every response.

Armando

> you've got some sort of problem starting a shell on the remote machine.
> it's likely authentication.  make sure the hostowner has speaksfor
> abilities on the auth server (/lib/ndb/auth).

Re: [9fans] 9grid

[lupin636]

Thanks Ron, of course cpu doesn't exists on linux...but as you said,
it seems that i have to connect to a cpu server, but i would like to
launch a task from terminal to cpu server, and after obtain a result
from that cpu server,what do i have to do??or what do i have to use??

thanks in advance to everybody
Armando


> On Mon, Nov 10, 2008 at 9:35 AM,  <[EMAIL PROTECTED]> wrote:
> > I have a doubt.because i was thinking about all i have to do, and
> > i don't know if using cpu command is the right thing to do. anyway,
> > the fact is, i have to launch a simple task from terminal (connected
> > by armando) to a node on the cluster (diskless cpu server), i thought
> > that cpu command was right but i'm not really sure anymore, because in
> > unix i used to use "rsh" and "rcmd".
> > any suggestions please??
>
> rsh and ssh suck in clusters. cpu is almost exactly what you want. You
> don't get cpu on linux because the linux guys have not reinvented them
> yet. Give them time.
>
> ron

Re: [9fans] 9grid

[erik quanstrom]

> I have a doubt.because i was thinking about all i have to do, and
> i don't know if using cpu command is the right thing to do. anyway,
> the fact is, i have to launch a simple task from terminal (connected
> by armando) to a node on the cluster (diskless cpu server), i thought
> that cpu command was right but i'm not really sure anymore, because in
> unix i used to use "rsh" and "rcmd".
> any suggestions please??

you've got some sort of problem starting a shell on the remote machine.
it's likely authentication.  make sure the hostowner has speaksfor
abilities on the auth server (/lib/ndb/auth).

- erik

Re: [9fans] 9grid

[ron minnich]

On Mon, Nov 10, 2008 at 9:35 AM,  <[EMAIL PROTECTED]> wrote:
> I have a doubt.because i was thinking about all i have to do, and
> i don't know if using cpu command is the right thing to do. anyway,
> the fact is, i have to launch a simple task from terminal (connected
> by armando) to a node on the cluster (diskless cpu server), i thought
> that cpu command was right but i'm not really sure anymore, because in
> unix i used to use "rsh" and "rcmd".
> any suggestions please??
>

rsh and ssh suck in clusters. cpu is almost exactly what you want. You
don't get cpu on linux because the linux guys have not reinvented them
yet. Give them time.

ron

Re: [9fans] 9grid

[lupin636]

I have a doubt.because i was thinking about all i have to do, and
i don't know if using cpu command is the right thing to do. anyway,
the fact is, i have to launch a simple task from terminal (connected
by armando) to a node on the cluster (diskless cpu server), i thought
that cpu command was right but i'm not really sure anymore, because in
unix i used to use "rsh" and "rcmd".
any suggestions please??

thanks in advance to all of you,

bye,
Armando.


On 10 Nov, 15:13, [EMAIL PROTECTED] wrote:
> Thanks for replaying...
> Sorry but i got confusion about your replay, i think i don't
> understand very well
> I want to make it clear first that the file/auth server are the same
> pc, and the cpuservers are the nodes of the cluster (5 nodes), which
> are diskless, and a terminal (my laptop).
> I'd like to use the cpu(1) command to connect to one node of the
> cluster (cpuserver) from the terminal, i tried by doing:
>
> term% cpu -h cpus
> term% ls /mnt/term
> term%
>
> but i think it isn't correct, because the prompt is still term%, and /
> mnt/term is empty, i did "ls /mnt/term" because i wanted to see if the
> namespace was mounted.
> is that correct?
> I hope i explain to you in the right way.
> thanks again..
>
> bye,
>
> Armando

Re: [9fans] 9grid

[lupin636]

Thanks for replaying...
Sorry but i got confusion about your replay, i think i don't
understand very well
I want to make it clear first that the file/auth server are the same
pc, and the cpuservers are the nodes of the cluster (5 nodes), which
are diskless, and a terminal (my laptop).
I'd like to use the cpu(1) command to connect to one node of the
cluster (cpuserver) from the terminal, i tried by doing:

term% cpu -h cpus
term% ls /mnt/term
term%

but i think it isn't correct, because the prompt is still term%, and /
mnt/term is empty, i did "ls /mnt/term" because i wanted to see if the
namespace was mounted.
is that correct?
I hope i explain to you in the right way.
thanks again..

bye,

Armando



> there's nothing in the system itself that changes one's prompt.
> this is done by the profile.  typically one can use the convention
> that $sysname is the contents of /dev/sysname.
> ; echo $sysname $cpu
> brasstown ladd
> ; cpu
> ; echo $sysname
>
> - erik

Re: [9fans] 9grid

[erik quanstrom]

> trying with cpu(1) command, i was doing:
> cpu -h fileservername
> and the prompt changed from term% to cpu%, and i supposed that was
> correct, but when i tried to connect to a cpuserver (all nodes are
> diskless)
> cpu -h cpuservername
> the prompt didn't change, is that correct? before doing this, i tried
> with

there's nothing in the system itself that changes one's prompt.
this is done by the profile.  typically one can use the convention
that $sysname is the contents of /dev/sysname.
; echo $sysname $cpu
brasstown ladd
; cpu
; echo $sysname

- erik