X11 on small systems? (was: Qi-Hardware Nanonote group purchase?)

2010-09-13 Thread Joshua Judson Rosen 
Benjamin Scott  writes:
>
> On Sun, Sep 5, 2010 at 10:12 PM, Joshua Judson Rosen
>  wrote:
> > > X reports a resident size of 40 MB, although how much of
> > > that (if any) might actually be video card RAM I dunno.
> >
> > I bet none of it is video-card RAM; a significant (not necessarily
> > majority, but significant) portion of the RAM `used by X', though,
> > is shared libraries that are also used by other processes--and those
> > are basically `gratis' since you'd be using them regardless.
> 
>   I'm approaching the limits of my understanding now, but:
> 
>   I note that several of the shared libraries you list are specific to
> the X server, and thus aren't shared by any other process.

Yes, however: several of the libraries that are exclusive to
the X server are actually things that would (or could) be eliminated
in different use-cases; the 4-MB Intel DRI module, for example,
is unnecessary even on my laptop (unless I want accelerated 3-D,
which I *almost* never do--even on my laptop); libfreetype could
presumably be eliminated if we just used bitmap fonts (which is
probably what one wants on a QVGA display, anyway).

And, of course: there are plenty more, in the full listing,
of all types--ones that are specific to the X server, ones that are
specific to other applications (some of them X clients), and ones that
are shared between all sorts of processes. I wasn't trying to prove you
wrong (at all, let a lone by by slight of hand), just pointing out that
there's deeper analysis necessary in order to actually figure out what
the top-level memory-usage figures really mean.

Though...:

>   I've never used memstat before, but the manual page states that it
> reports "virtual memory".  I was looking at the "RSS" (resident
> segment size) column of ps.

Oh, right--you did say "resident", but I managed to miss it

I think the whole bit about picking through with memstat is
basically irrelevant, in that case, because...:

> Virtual size can include things which aren't main RAM.

I'm going to go a step further, contradicting the prevailing wisdom
of the Internet, and say that VSZ can even include that aren't even
properly `allocated *virtual* memory'. What I mean is:

RSS != (VSZ - (amount of memory paged out to swap))

Skimming & grep'ing through Linux 2.6 source that I have on hand,
it looks like mmap() contributes to VSZ but never to RSS, which
I guess makes sense--and it would explain why, even though
I'm *zero bytes* into swap, I still have a huge mismatch
between VSZ and RSS for many of my processes. And that mmap
business applies to shared libraries, if I recall correctly

The only thing that comes to mind about how RSS might be misleading,
here, is maybe some `unused regions' that *could* be swapped out
(if you had /proc/sys/vm/swappiness cranked up, or had more disk I/O)
but aren't.

I guess, in general, you have to worry about copy-on-write
pages from forks, but that's probably not so relevant here.

The bit I mentioned earlier, about how a bunch of the memory
that appears to be used by the X server process is actually
allocated on behalf of client processes (some applications--
and toolkits--are *much* `worse' about this than others),
may be most relevant, depending on whether you were actually
running any X clients when you did your measurements.


> It would appear that memstat breaks out memory-mapped files, but how
> does it treat things like pages swapped to disk?

Oh, it doesn't--at all. If you care about anything other than
making sense out of VSZ figures, memstat's basically too dumb
to be of service. But I'd say that the VSZ figures are what're
most in need of a tool like memstat to explain them, so... :)

Though, it *would* be nice to be able to get an idea about
how much of that RSS figure was actually due to loading
and using particular libraries. I don't know how one could
go about doing that.

If you just want to know whether Xorg can run on Ben Nanonote,
it'd probably be most straight-forward (and maybe even more
cost-effective, if your day-job pays hourly ;)) to just...
buy one and try it out. :)

>   (I wouldn't expect RSS to include RAM mapped from the video card,
> but I didn't know that for sure, hence my qualification earlier.)

I could be mistaken, but my bet is based on the idea that Xorg
just mmaps /dev/mem (and /dev/dri/card0 and stuff like that, too,
I guess?) to get at graphics cards' resources.

If you go through a full listing fro mmemstat, you'll actually see
things like /dev/dri/card0 and /dev/mem as separate maps (in use
exclusively by the X server, hopefully).

-- 
"Don't be afraid to ask (λf.((λx.xx) (λr.f(rr."

___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: X11 on small systems? (was: Qi-Hardware Nanonote group purchase?)

2010-09-13 Thread Tom Buskey 
On Mon, Sep 13, 2010 at 4:47 PM, Joshua Judson Rosen
wrote:

> Benjamin Scott  writes:
> >
> > On Sun, Sep 5, 2010 at 10:12 PM, Joshua Judson Rosen
> >  wrote:
> > > > X reports a resident size of 40 MB, although how much of
> > > > that (if any) might actually be video card RAM I dunno.
> > >
> > > I bet none of it is video-card RAM; a significant (not necessarily
> > > majority, but significant) portion of the RAM `used by X', though,
> > > is shared libraries that are also used by other processes--and those
> > > are basically `gratis' since you'd be using them regardless.
> >
> >   I'm approaching the limits of my understanding now, but:
> >
> >   I note that several of the shared libraries you list are specific to
> > the X server, and thus aren't shared by any other process.
>
> Yes, however: several of the libraries that are exclusive to
> the X server are actually things that would (or could) be eliminated
> in different use-cases; the 4-MB Intel DRI module, for example,
> is unnecessary even on my laptop (unless I want accelerated 3-D,
> which I *almost* never do--even on my laptop); libfreetype could
> presumably be eliminated if we just used bitmap fonts (which is
> probably what one wants on a QVGA display, anyway).
>
> And, of course: there are plenty more, in the full listing,
> of all types--ones that are specific to the X server, ones that are
> specific to other applications (some of them X clients), and ones that
> are shared between all sorts of processes. I wasn't trying to prove you
> wrong (at all, let a lone by by slight of hand), just pointing out that
> there's deeper analysis necessary in order to actually figure out what
> the top-level memory-usage figures really mean.
>

http://tinycorelinux.com/ manages to get X trimmed down into a 10MB image
using 48MB of RAM.  With the 2.6 kernel.

The RAM requirements will be different on ARM.  I know I see differences
with BSD and Solaris on intel vs sparc.
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: X11 on small systems?

2010-09-13 Thread Joseph Smith 
On 09/13/2010 05:07 PM, Tom Buskey wrote:
>
>
> On Mon, Sep 13, 2010 at 4:47 PM, Joshua Judson Rosen
> mailto:roz...@geekspace.com>> wrote:
>
> Benjamin Scott mailto:dragonh...@gmail.com>>
> writes:
>  >
>  > On Sun, Sep 5, 2010 at 10:12 PM, Joshua Judson Rosen
>  > mailto:roz...@geekspace.com>> wrote:
>  > > > X reports a resident size of 40 MB, although how much of
>  > > > that (if any) might actually be video card RAM I dunno.
>  > >
>  > > I bet none of it is video-card RAM; a significant (not necessarily
>  > > majority, but significant) portion of the RAM `used by X', though,
>  > > is shared libraries that are also used by other processes--and
> those
>  > > are basically `gratis' since you'd be using them regardless.
>  >
>  >   I'm approaching the limits of my understanding now, but:
>  >
>  >   I note that several of the shared libraries you list are
> specific to
>  > the X server, and thus aren't shared by any other process.
>
> Yes, however: several of the libraries that are exclusive to
> the X server are actually things that would (or could) be eliminated
> in different use-cases; the 4-MB Intel DRI module, for example,
> is unnecessary even on my laptop (unless I want accelerated 3-D,
> which I *almost* never do--even on my laptop); libfreetype could
> presumably be eliminated if we just used bitmap fonts (which is
> probably what one wants on a QVGA display, anyway).
>
> And, of course: there are plenty more, in the full listing,
> of all types--ones that are specific to the X server, ones that are
> specific to other applications (some of them X clients), and ones that
> are shared between all sorts of processes. I wasn't trying to prove you
> wrong (at all, let a lone by by slight of hand), just pointing out that
> there's deeper analysis necessary in order to actually figure out what
> the top-level memory-usage figures really mean.
>
>
> http://tinycorelinux.com/ manages to get X trimmed down into a 10MB
> image using 48MB of RAM.  With the 2.6 kernel.
>
> The RAM requirements will be different on ARM.  I know I see differences
> with BSD and Solaris on intel vs sparc.
>
>
You may also want to check out matchbox which is less than 1mb and works 
well with busybox and uclibc.

-- 
Thanks,
Joseph Smith
Set-Top-Linux
www.settoplinux.org
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: X11 on small systems?

2010-09-13 Thread Joseph Smith 
On 09/13/2010 06:20 PM, Joseph Smith wrote:
> On 09/13/2010 05:07 PM, Tom Buskey wrote:
>>
>>
>> On Mon, Sep 13, 2010 at 4:47 PM, Joshua Judson Rosen
>> mailto:roz...@geekspace.com>>  wrote:
>>
>>  Benjamin Scottmailto:dragonh...@gmail.com>>
>>  writes:
>>   >
>>   >  On Sun, Sep 5, 2010 at 10:12 PM, Joshua Judson Rosen
>>   >  mailto:roz...@geekspace.com>>  wrote:
>>   >  >  >  X reports a resident size of 40 MB, although how much of
>>   >  >  >  that (if any) might actually be video card RAM I dunno.
>>   >  >
>>   >  >  I bet none of it is video-card RAM; a significant (not 
>> necessarily
>>   >  >  majority, but significant) portion of the RAM `used by X', 
>> though,
>>   >  >  is shared libraries that are also used by other processes--and
>>  those
>>   >  >  are basically `gratis' since you'd be using them regardless.
>>   >
>>   >I'm approaching the limits of my understanding now, but:
>>   >
>>   >I note that several of the shared libraries you list are
>>  specific to
>>   >  the X server, and thus aren't shared by any other process.
>>
>>  Yes, however: several of the libraries that are exclusive to
>>  the X server are actually things that would (or could) be eliminated
>>  in different use-cases; the 4-MB Intel DRI module, for example,
>>  is unnecessary even on my laptop (unless I want accelerated 3-D,
>>  which I *almost* never do--even on my laptop); libfreetype could
>>  presumably be eliminated if we just used bitmap fonts (which is
>>  probably what one wants on a QVGA display, anyway).
>>
>>  And, of course: there are plenty more, in the full listing,
>>  of all types--ones that are specific to the X server, ones that are
>>  specific to other applications (some of them X clients), and ones that
>>  are shared between all sorts of processes. I wasn't trying to prove you
>>  wrong (at all, let a lone by by slight of hand), just pointing out that
>>  there's deeper analysis necessary in order to actually figure out what
>>  the top-level memory-usage figures really mean.
>>
>>
>> http://tinycorelinux.com/ manages to get X trimmed down into a 10MB
>> image using 48MB of RAM.  With the 2.6 kernel.
>>
>> The RAM requirements will be different on ARM.  I know I see differences
>> with BSD and Solaris on intel vs sparc.
>>
>>
> You may also want to check out matchbox which is less than 1mb and works
> well with busybox and uclibc.
>

and of course it works well with tinyx.

-- 
Thanks,
Joseph Smith
Set-Top-Linux
www.settoplinux.org
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: X11 on small systems? (was: Qi-Hardware Nanonote group purchase?)

2010-09-13 Thread Benjamin Scott 
On Mon, Sep 13, 2010 at 4:47 PM, Joshua Judson Rosen
 wrote:
> Yes, however: several of the libraries that are exclusive to
> the X server are actually things that would (or could) be eliminated
> in different use-cases; the 4-MB Intel DRI module, for example,

  Good point.  X.org doesn't necessarily have to be as heavy as it is
on a desktop PC.   And has been mentioned, there are more lightweight
X servers out there.

>> Virtual size can include things which aren't main RAM.
>
> I'm going to go a step further, contradicting the prevailing wisdom
> of the Internet, and say that VSZ can even include that aren't even
> properly `allocated *virtual* memory'. What I mean is:
>
>    RSS != (VSZ - (amount of memory paged out to swap))

  Contrary to popular belief and several OS GUIs, "virtual memory"
does not mean "using disk as virtual RAM".  That is, "virtual memory"
is *not* synonymous with "swap space".  Virtual memory is the address
space seen by a process and provided by the MMU.  On i386, the virtual
memory space is always 4 GiB (per process).

  The MMU can map pages of virtual memory to the hardware address
space, or leave them unmapped.  Hardware address space includes both
main RAM and other hardware stuff (like ROMs, often video RAM, some
other kinds of hardware buffers (depending on design), etc.).

  Pages which aren't mapped cause a page fault if the process tries to
read or write that page.  At that point it's up to the kernel memory
manager to decide what to do.   If it's a page written out to swap,
the kernel can read it back in.  If it's a page from a mmap'ed file,
the kernel can do the I/O to put that block in RAM.

  I don't actually know what the "virtual size" figure represents.

  My expectation was that "virtual size" was the total size of all
objects the kernel memory manager had associated with a given
process's virtual memory space (not necessarily mapped).  That would
include all "regular" memory allocated, plus every mmap'ed file (and
thus every shared library).  For X, that might also include video card
RAM.  It might even include things which have no "real" representation
at all:

blackfire$ sudo memstat | grep nvidia0
  42560k: PID  3000 (/dev/nvidia0)
3278332k: /dev/nvidia0 3000
blackfire$

  That's over 3 GiB.   My video card has 256 MiB RAM, main RAM is 1
GiB.  So whatever that is, it isn't hardware.

  However:

blackfire$ ps u 3000
USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
root  3000  0.6  4.6  72248 48448 tty7 SLs+ Aug16 255:59 /usr/bin/X :0
blackfire$

  Note that ps is reporting X as having a virtual size of roughly 72 MiB.

  The ps(1) man page says, of VSZ, that "Device mappings are currently
excluded".  I guess that helps explain the difference, but I'm still
not really sure what VSZ actually represents.

  But in any event, "virtual memory" doesn't have any necessary
correlation to committed primary or secondary storage (i.e., RAM or
disk).

>> It would appear that memstat breaks out memory-mapped files, but how
>> does it treat things like pages swapped to disk?
>
> Oh, it doesn't--at all. If you care about anything other than
> making sense out of VSZ figures ...

  Pages which are not mapped (which includes swapped pages) still
exist in virtual memory.  That's what lets the OS swap them back in as
needed.

>>  (I wouldn't expect RSS to include RAM mapped from the video card,
>> but I didn't know that for sure, hence my qualification earlier.)
>
> I could be mistaken, but my bet is based on the idea that Xorg
> just mmaps /dev/mem (and /dev/dri/card0 and stuff like that, too,
> I guess?) to get at graphics cards' resources.

  That's similar to my understanding as well, but again, I don't know
for sure.  :)

-- Ben

___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Koha - anyone do an install? MySQL question...

2010-09-13 Thread Bruce Labitt 
 I am trying to install koha, an open source library 
management system on my spouse's computer.  She wants to 
evaluate it.


I'm following the instructions on 
http://wiki.koha-community.org/wiki/Koha_on_Ubuntu and I 
need to understand what step C is really doing.


*Step #7 Create MySQL Database and grant all privileges to 
the koha database to the specified user*


/[mysql]/ = Command that needs to be typed from within the 
mysql program


/A. [cmd]/mysqladmin -uroot -ppassword create kohadata

/B. [cmd]/mysql -uroot -ppassword

/C. [mysql]/grant all on kohadata.* to 'root'@'localhost' 
identified by 'password';


/D. [mysql]/flush privileges;

/E. [mysql]/quit;

In particular, what goes in 'root' and 'password'?  Am I 
giving root privilege to kohadata?  Why would I want to do 
that?  Right now, my spouse does not have admin privileges 
on the system.  Can someone explain this to me?


Thanks!
Bruce
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: X11 on small systems?

2010-09-13 Thread Joshua Judson Rosen 
Benjamin Scott  writes:
> On Mon, Sep 13, 2010 at 4:47 PM, Joshua Judson Rosen
>  wrote:
> >
> > Yes, however: several of the libraries that are exclusive to
> > the X server are actually things that would (or could) be eliminated
> > in different use-cases; the 4-MB Intel DRI module, for example,
> 
>   Good point.  X.org doesn't necessarily have to be as heavy as it is
> on a desktop PC.   And has been mentioned, there are more lightweight
> X servers out there.

... assuming that you even need an X server :)

> > > Virtual size can include things which aren't main RAM.
> >
> > I'm going to go a step further, contradicting the prevailing wisdom
> > of the Internet, and say that VSZ can even include that aren't even
> > properly `allocated *virtual* memory'. What I mean is:
> >
> >    RSS != (VSZ - (amount of memory paged out to swap))
> 
>   Contrary to popular belief and several OS GUIs, "virtual memory"
> does not mean "using disk as virtual RAM". That is, "virtual memory"
> is *not* synonymous with "swap space".


Note that top(1), of all things, says:

   o: VIRT  --  Virtual Image (kb)
  The  total  amount  of virtual memory used by the task.  It includes
  all code, data and  shared  libraries  plus  pages  that  have  been
  swapped out.

  VIRT = SWAP + RES.

   p: SWAP  --  Swapped size (kb)
  The swapped out portion of a task’s total virtual memory image.

   q: RES  --  Resident size (kb)
  The non-swapped physical memory a task has used.


Is it any wonder that people are confused?

> Virtual memory is the address space seen by a process and provided
> by the MMU.
[... excellent description of paging and what "virtual memory"
 actually means, elided...]
>   My expectation was that "virtual size" was the total size of all
> objects the kernel memory manager had associated with a given
> process's virtual memory space (not necessarily mapped).  That would
> include all "regular" memory allocated, plus every mmap'ed file (and
> thus every shared library).

Exactly--thank you for taking the time to paint the picture more vividly :)


> For X, that might also include video card RAM.  It might even
> include things which have no "real" representation at all:
> 
> blackfire$ sudo memstat | grep nvidia0
>   42560k: PID  3000 (/dev/nvidia0)
> 3278332k: /dev/nvidia0 3000
> blackfire$
> 
>   That's over 3 GiB.   My video card has 256 MiB RAM, main RAM is 1
> GiB.  So whatever that is, it isn't hardware.
[...]
>   But in any event, "virtual memory" doesn't have any necessary
> correlation to committed primary or secondary storage (i.e., RAM or
> disk).

Which reminds me: let's not forget /proc/sys/vm/overcommit* :)

-- 
"Don't be afraid to ask (λf.((λx.xx) (λr.f(rr."

___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: Koha - anyone do an install? MySQL question...

2010-09-13 Thread Mark Komarinski 
On 09/13/2010 09:06 PM, Bruce Labitt wrote:
> /[mysql]/ = Command that needs to be typed from within the mysql program
>
> /A. [cmd]/mysqladmin -uroot -ppassword create kohadata
>
> /B. [cmd]/mysql -uroot -ppassword
>
> /C. [mysql]/grant all on kohadata.* to 'root'@'localhost' identified by
> 'password';
>
> /D. [mysql]/flush privileges;
>
> /E. [mysql]/quit;
>
> In particular, what goes in 'root' and 'password'? Am I giving root
> privilege to kohadata? Why would I want to do that? Right now, my spouse
> does not have admin privileges on the system. Can someone explain this
> to me?

The command in C should be something like:

grant all on kohadata.* to 'koha'@'localhost' identified by 'password';

That will create a non-root user (k...@localhost) that has a password of 
password.  When you set up the user in koha, make sure the username and 
hostname is set properly as k...@localhost is different from 
k...@foo.bar.com (or whatever domain you use).

Since you're using a grant statement, I don't think you need to run the 
command in D.

-Mark
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: X11 on small systems? (was: Qi-Hardware Nanonote group purchase?)

2010-09-13 Thread Joshua Judson Rosen 
Joshua Judson Rosen  writes:
> > Joshua Judson Rosen  writes:
> > >
> > > I'm going to buy one of these to see how well it can replace my
> > > now-defunct, Rockbox-running iPod:
> > >
> > > http://en.qi-hardware.com/wiki/Hardware-Ben
> > >
> > > They're $99, but they ship from Hong Kong so shipping a single unit
> > > adds ~$30 to the price; shipping more units together increases
> > > the shipping-price but only ~logarithmically, so there's an opportunity
> > > to spread the shipping-cost out with a `group buy'; for example,
> > > it looks like shipping goes down to $11/unit if I order 5.
> > >
> > > Anyone interested?
> >
[long thread about how likely it is that Xorg can run on Ben Nanonote...]
>
> If you just want to know whether Xorg can run on Ben Nanonote,
> it'd probably be most straight-forward (and maybe even more
> cost-effective, if your day-job pays hourly ;)) to just...
> buy one and try it out. :)

On that note..., I'm looking to place my order sometime this week;
if anyone's actually interested in co-shipping in that timeframe,
let me know.

I hear that Freedom Included  is also
likely to start selling Nanonotes some time in the next couple/few months,
if anyone is interested `but not right now' (maybe you want to wait and
see how I make out...).

-- 
"Don't be afraid to ask (λf.((λx.xx) (λr.f(rr."

___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/