Re: a 15 GB file on tmpfs

2005-07-22 Thread Stefan Smietanowski
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Bernd Eckenfels wrote:
> On Fri, Jul 22, 2005 at 01:00:18PM +0200, Stefan Smietanowski wrote:
> 
>>>You cant have 16GB of Memory with 32bit CPUs.
>>
>>PAE
>>CONFIG_HIGMEM64G
>>Supports a 36bit address space, which Xeons do support.
> 
> 
> Yes right, I was just not aware recent hardware (still) supports that. I
> mean even mit 2MB modules most of them are specified only to 8GB. I would
> consider buying such a system quite foolish. All of the HP servers with 12GB
> and more seem to support EM64T. Do you know vendors who ship non-EM64T
> servers with more than 16GB?

http://www.intel.com/products/processor/xeon/index.htm

Shows that the lowest speed Xeon with EM64T is 2.83GHz.

http://commerce.euro.dell.com/dellstore/config/frameset.asp?c=607&n=3410&b=62417&m=gbp&cs=ukbsdt1&sbc=ukbsdpedge&v=d&cu=etstore&l=en&s=ukbsd&store=ukbsd

Shows a system taking 1-4 of those processors at 2.0, 2.2, 2.7 and
3.0GHz, making them 32bit only.

Memory you can configure is from 1GiB (4x256MiB) to 32GiB (16x2GiB).

If I'd buy a server I wouldn't buy one of those however, I'm just
answering the question.

Disclaimer: I do not have anything to do with either of the above
mentioned vendors. I just looked up the first vendor that came into
my head.

// Stefan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (MingW32)

iD8DBQFC4WDjBrn2kJu9P78RAn9RAJ9wRffW5VB0WbRgRBNjfN9+k3XMvgCgtWAO
mu8p8nj8iIpNIkuYiMkiTuI=
=9kQz
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-22 Thread Bernd Eckenfels
On Fri, Jul 22, 2005 at 01:00:18PM +0200, Stefan Smietanowski wrote:
> > You cant have 16GB of Memory with 32bit CPUs.
> PAE
> CONFIG_HIGMEM64G
> Supports a 36bit address space, which Xeons do support.

Yes right, I was just not aware recent hardware (still) supports that. I
mean even mit 2MB modules most of them are specified only to 8GB. I would
consider buying such a system quite foolish. All of the HP servers with 12GB
and more seem to support EM64T. Do you know vendors who ship non-EM64T
servers with more than 16GB?

Gruss
Bernd
-- 
  (OO) -- [EMAIL PROTECTED] --
 ( .. )[EMAIL PROTECTED],linux.de,debian.org}  http://www.eckes.org/
  o--o   1024D/E383CD7E  [EMAIL PROTECTED]  v:+497211603874  f:+49721151516129
(OO)  When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl!
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-22 Thread linux
> I have a 15 GB file which I want to place in memory via tmpfs. I want to do 
> this because I need to have this data accessible with a very low seek time.

It should work fine.  tmpfs has the same limits as any other file system,
2 TB or more, and more than that with CONFIG_LBD.

NOTE, however, that tmpfs does NOT guarantee the data will be in RAM!  It
uses the page cache just like any other file system, and pages out unused
data just like any other file system.  If you just want average-case fast,
it'll work fine.  If you want guaranteed fast, you'll have to work harder.

> I want to know if this is possible before spending 10,000 euros on a machine 
> that has 16 GB of memory. 

So create a 15 GB file on an existing machine.  Make it sparse, so you
don't need so much RAM, but test to verify that the kernel doesn't
wrap at 4 GB, and can keep the data at offsets 0, 4 GB, 8 GB, and 12 GB
separate.

Works for me (test code below).

> The machine we plan to buy is a HP Proliant Xeon machine and I want to run a 
> 32 bit linux kernel on it (the xeon we want doesn't have the 64-bit stuff 
> yet)

If you're working with > 4GB data sets, I would recommend you think VERY hard
before deciding not to get a 64-bit machine.  If you could just put all 15 GB
into your application's address space:
- The application would be much simpler and faster.
- The kernel wouldn't be slowed by HIGHMEM workarounds.  It's not that bad,
  but it's definitely noticeable.
- Your expensive new machine won't be obsolete quite as fast.

I'd also like to mention that AMD's large L2 TLB is enormously helpful when
working with large data sets.  It's not discussed much on the web sites that
benchmark with games, but it really helps crunch a lot of data.



#define _GNU_SOURCE
#define _FILE_OFFSET_BITS 64
#include 
#include 
#include 
#include 
#include 

#define STRIDE (1<<20)

int
main(int argc, char **argv)
{
int fd;
off_t off;

if (argc != 2) {
fprintf(stderr, "Wrong number of arguments: %u\n", argc);
return 1;
}
fd = open(argv[1], O_RDWR|O_CREAT|O_LARGEFILE, 0666);
if (fd < 0) {
perror(argv[1]);
return 1;
}

for (off = 0; off < 0x4LL; off += STRIDE) {
char buf[40];
off_t res;
ssize_t ss1, ss2;;

ss1 = sprintf(buf, "%llu", off);

res = lseek(fd, off, SEEK_SET);
if (res == (off_t)-1) {
perror("lseek");
return 1;
}
ss2 = write(fd, buf, ++ss1);
if (ss2 != ss1) {
perror("write");
return 1;
}
}

for (off = 0; off < 0x4LL; off += STRIDE) {
char buf[40], buf2[40];
off_t res;
ssize_t ss1, ss2;;

ss1 = sprintf(buf, "%lld", off);

res = lseek(fd, off, SEEK_SET);
if (res == (off_t)-1) {
perror("lseek");
return 1;
}

ss2 = read(fd, buf2, ++ss1);
if (ss2 != ss1 || memcmp(buf, buf2, ss1) != 0) {
fprintf(stderr, "Mismatch at %llu: %.*s vs. %s\n", off, 
(int)ss2, buf2, buf);
return 1;
}
}
printf("All tests succeeded.\n");
return 0;
}

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-22 Thread Stefan Smietanowski
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Bernd Eckenfels wrote:
> In article <[EMAIL PROTECTED]> you wrote:
> 
>>The machine we plan to buy is a HP Proliant Xeon machine and I want to run a 
>>32 bit linux kernel on it (the xeon we want doesn't have the 64-bit stuff 
>>yet)
> 
> 
> You cant have 16GB of Memory with 32bit CPUs.

PAE

CONFIG_HIGMEM64G

Supports a 36bit address space, which Xeons do support.

That doesn't mean a program can access it, you still
have the same old limitations per process.

// Stefan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (MingW32)

iD8DBQFC4NHCBrn2kJu9P78RApKpAKCEUzgKKRY8VG8XIwlCBrTzWpE7LwCdGUss
bSM49ZnzRgFxJ4h0WF5Ulio=
=zwUX
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-22 Thread Bernd Eckenfels
In article <[EMAIL PROTECTED]> you wrote:
> The machine we plan to buy is a HP Proliant Xeon machine and I want to run a 
> 32 bit linux kernel on it (the xeon we want doesn't have the 64-bit stuff 
> yet)

You cant have 16GB of Memory with 32bit CPUs.

Bernd
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


RE: a 15 GB file on tmpfs

2005-07-22 Thread Cabaniols, Sebastien
Anyway, if you buy a brand new server, you will find it difficult to get
a 32 bit architecture.

What I don't understand is why you want it to go into tmpfs, why don't
you let the pagecache do its job of putting the data in RAM where it is
needed ? 

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Antonio Vargas
Sent: mercredi 20 juillet 2005 15:32
To: Erik Mouw
Cc: Bastiaan Naber; linux-kernel@vger.kernel.org
Subject: Re: a 15 GB file on tmpfs

On 7/20/05, Erik Mouw <[EMAIL PROTECTED]> wrote:
> On Wed, Jul 20, 2005 at 02:16:36PM +0200, Bastiaan Naber wrote:
> > I have a 15 GB file which I want to place in memory via tmpfs. I 
> > want to do this because I need to have this data accessible with a
very low seek time.
> 
> That should be no problem on a 64 bit architecture.
> 
> > I want to know if this is possible before spending 10,000 euros on a

> > machine that has 16 GB of memory.
> 
> If you want to spend that amount of money on memory anyway, the extra 
> cost for an AMD64 machine isn't that large.
> 
> > The machine we plan to buy is a HP Proliant Xeon machine and I want 
> > to run a
> > 32 bit linux kernel on it (the xeon we want doesn't have the 64-bit 
> > stuff
> > yet)
> 
> AFAIK you can't use a 15 GB tmpfs on i386 because large memory support

> is basically a hack to support multiple 4GB memory spaces (some VM 
> guru correct me if I'm wrong). Just get an Athlon64 machine and run a 
> 64 bit kernel on it. If compatibility is a problem, you can still run 
> a 32 bit
> i386 userland on an x86_64 kernel.
> 
> 
> Erik
> 
> --
> +-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
> | Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
> -

Bastian, Erik is dead-on on that one: go 64bit and forget all worries
about your 15GB filesize. Just don't forget to look not only x86-64
(intel or amd) but also itanium, ppc64 and s390 machines, you never know
about surprises!

--
Greetz, Antonio Vargas aka winden of network

http://wind.codepixel.com/

Las cosas no son lo que parecen, excepto cuando parecen lo que si son.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel"
in the body of a message to [EMAIL PROTECTED] More majordomo
info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-21 Thread Andrew Burgess
On Wed, Jul 20, 2005 at 02:16:36PM +0200, Bastiaan Naber wrote:
> I have a 15 GB file which I want to place in memory via tmpfs. I want to do 
> this because I need to have this data accessible with a very low seek time.

You don't want tmpfs. You want either (1) ramfs and copy the data once at
boot time or (2) any filesystem and mmap and lock which will read the data
every time your app starts. 

Tmpfs is backed by swap so other system activity will potentially cause some of
the file to go to swap which kills your latency spec.

HTH

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-21 Thread Bernd Petrovitsch
On Thu, 2005-07-21 at 11:12 +0200, Stefan Smietanowski wrote:
[...]
> On a 64bit machine:
> $ gcc test.c -o test64 ; ./test64; file ./test64
> sizeof(void *): 8
> sizeof(size_t): 8
> test64: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for
> GNU/Linux 2.4.0, dynamically linked (uses shared libs), not stripped
> 
> On a 32bit machine (or in this case, 32bit userland on a 64bit machine):
> $ gcc -m32 test.c -o test32 ; ./test32; file ./test32
> sizeof(void *): 4
> sizeof(size_t): 4
> test32: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for
> GNU/Linux 2.2.5, dynamically linked (uses shared libs), not stripped
> 
> Meaning both the pointer and the size argument are only 32bit (4byte)
> on 32-bit arches and 64bit (8 byte) on 64bit arches.

Which means that pure addressing of the 15GB on 32bit archs needs
explicit handling (yes, you can manage the offset in a "long long" but
you cannot index anything with it not addressable with a 32bit offset).

Well, 64bit is apparently the way to go 

Bernd
-- 
Firmix Software GmbH   http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
  Embedded Linux Development and Services

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-21 Thread Stefan Smietanowski
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Bernd Petrovitsch wrote:
> On Wed, 2005-07-20 at 14:16 +0200, Bastiaan Naber wrote:
> [...]
> 
>>I have a 15 GB file which I want to place in memory via tmpfs. I want to do 
>>this because I need to have this data accessible with a very low seek time.
> 
> 
> Apart fromn the 32-vs-64bit thing: Isn't it enough (and simpler and more
> flexible) to mmap(2) that file and mlock(2) it afterwards?
> 
>   Bernd

On 32bit arches, a pointer is 32bit large.
On 64bit arches, a pointer is 64bit large.

You can't mmap() the whole file if it's larger than 32bit on a 32bit
arch.

 void *mmap(void *start, size_t length, int prot, int flags,
   int fd, off_t offset);

test.c:
#include 
int main(void)
{
  printf("sizeof(void *): %d\n", sizeof(void *));
  printf("sizeof(size_t): %d\n", sizeof(size_t));
}

On a 64bit machine:
$ gcc test.c -o test64 ; ./test64; file ./test64
sizeof(void *): 8
sizeof(size_t): 8
test64: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for
GNU/Linux 2.4.0, dynamically linked (uses shared libs), not stripped

On a 32bit machine (or in this case, 32bit userland on a 64bit machine):
$ gcc -m32 test.c -o test32 ; ./test32; file ./test32
sizeof(void *): 4
sizeof(size_t): 4
test32: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for
GNU/Linux 2.2.5, dynamically linked (uses shared libs), not stripped

Meaning both the pointer and the size argument are only 32bit (4byte)
on 32-bit arches and 64bit (8 byte) on 64bit arches.

// Stefan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (MingW32)

iD4DBQFC32caBrn2kJu9P78RAujmAJ9J3xYdbAwYdpGXuu8kiTwdcloaSQCY1TD1
SuJJ3Ylpsa+Cdo2uIQ/Prw==
=IFvd
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-21 Thread Bernd Petrovitsch
On Wed, 2005-07-20 at 14:16 +0200, Bastiaan Naber wrote:
[...]
> I have a 15 GB file which I want to place in memory via tmpfs. I want to do 
> this because I need to have this data accessible with a very low seek time.

Apart fromn the 32-vs-64bit thing: Isn't it enough (and simpler and more
flexible) to mmap(2) that file and mlock(2) it afterwards?

Bernd
-- 
Firmix Software GmbH   http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
  Embedded Linux Development and Services

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-20 Thread Denis Vlasenko
On Wednesday 20 July 2005 15:16, Bastiaan Naber wrote:
> Hi,
> 
> I am not sure if I can ask this here but I could not find any other place 
> where I could fine anyone with this knowledge.
> 
> I have a 15 GB file which I want to place in memory via tmpfs. I want to do 
> this because I need to have this data accessible with a very low seek time.
> 
> I want to know if this is possible before spending 10,000 euros on a machine 
> that has 16 GB of memory. 

Make large swap file and test it yourself. Not a problem.
--
vda

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-20 Thread Jim Nance
On Wed, Jul 20, 2005 at 05:23:52PM +0200, Antonio Vargas wrote:
> Most probably the cost of programming and debugging the hand-made
> paging on 32bit machines will cost more than the difference for a
> 64bit machine.

I'll second that.  There may not even be a price difference.
I've had quotes for otherwise comprable 32 bit Xeon servers
come in more expensive than 64 bit AMD machines.

One piece of advice I would offer is to make sure you buy
good ECC memory.  I've been working with large servers
for the last few years, and trying to be cheap with
memory has caused a lot of hard to track down problems.

- Jim

-- 
[EMAIL PROTECTED]
SDF Public Access UNIX System - http://sdf.lonestar.org
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-20 Thread Antonio Vargas
On 7/20/05, Erik Mouw <[EMAIL PROTECTED]> wrote:
> On Wed, Jul 20, 2005 at 01:35:07PM +, Miquel van Smoorenburg wrote:
> > In article <[EMAIL PROTECTED]>,
> > Erik Mouw  <[EMAIL PROTECTED]> wrote:
> > >On Wed, Jul 20, 2005 at 02:16:36PM +0200, Bastiaan Naber wrote:
> > >AFAIK you can't use a 15 GB tmpfs on i386 because large memory support
> > >is basically a hack to support multiple 4GB memory spaces (some VM guru
> > >correct me if I'm wrong).
> >
> > I'm no VM guru but I have a 32 bit machine here with 8 GB of
> > memory and 8 GB of swap:
> >
> > # mount -t tmpfs -o size=$((12*1024*1024*1024)) tmpfs /mnt
> > # df
> > Filesystem   1K-blocks  Used Available Use% Mounted on
> > /dev/sda1 19228276   1200132  17051396   7% /
> > tmpfs 12582912 0  12582912   0% /mnt
> >
> > There you go, a 12 GB tmpfs. I haven't tried to create a 12 GB
> > file on it, though, since this is a production machine and it
> > needs the memory ..
> 
> I stand corrected.
> 
> > So yes that appears to work just fine.
> 
> The question is if it's a good idea to use a 15GB tmpfs on a 32 bit
> i386 class machine. I guess a real 64 bit machine will be much faster
> in handling suchs amounts of data simply because you don't have to go
> through the hurdles needed to address such memory on i386.
> 
> 
> Erik
> 

On 32bit: you would have to use read() and write() or mmap() munmap()
mremap() to perform your own paging, since you can't fit 15GB on a 4GB
address space.

On 64bit: you would simply mmap() the whole file and you are done.

Most probably the cost of programming and debugging the hand-made
paging on 32bit machines will cost more than the difference for a
64bit machine.

-- 
Greetz, Antonio Vargas aka winden of network

http://wind.codepixel.com/

Las cosas no son lo que parecen, excepto cuando parecen lo que si son.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-20 Thread Jan Engelhardt

>I want to know if this is possible before spending 10,000 euros on a machine 
>that has 16 GB of memory. 

I can get a Dual Opteron 242 with 16G for less than 10K euro. :)



Jan Engelhardt
-- 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-20 Thread Erik Mouw
On Wed, Jul 20, 2005 at 01:35:07PM +, Miquel van Smoorenburg wrote:
> In article <[EMAIL PROTECTED]>,
> Erik Mouw  <[EMAIL PROTECTED]> wrote:
> >On Wed, Jul 20, 2005 at 02:16:36PM +0200, Bastiaan Naber wrote:
> >AFAIK you can't use a 15 GB tmpfs on i386 because large memory support
> >is basically a hack to support multiple 4GB memory spaces (some VM guru
> >correct me if I'm wrong).
> 
> I'm no VM guru but I have a 32 bit machine here with 8 GB of
> memory and 8 GB of swap:
> 
> # mount -t tmpfs -o size=$((12*1024*1024*1024)) tmpfs /mnt
> # df
> Filesystem   1K-blocks  Used Available Use% Mounted on
> /dev/sda1 19228276   1200132  17051396   7% /
> tmpfs 12582912 0  12582912   0% /mnt
> 
> There you go, a 12 GB tmpfs. I haven't tried to create a 12 GB
> file on it, though, since this is a production machine and it
> needs the memory ..

I stand corrected.

> So yes that appears to work just fine.

The question is if it's a good idea to use a 15GB tmpfs on a 32 bit
i386 class machine. I guess a real 64 bit machine will be much faster
in handling suchs amounts of data simply because you don't have to go
through the hurdles needed to address such memory on i386.


Erik

-- 
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-20 Thread Miquel van Smoorenburg
In article <[EMAIL PROTECTED]>,
Erik Mouw  <[EMAIL PROTECTED]> wrote:
>On Wed, Jul 20, 2005 at 02:16:36PM +0200, Bastiaan Naber wrote:
>> I have a 15 GB file which I want to place in memory via tmpfs. I want to do 
>> this because I need to have this data accessible with a very low seek time.
>
>That should be no problem on a 64 bit architecture.
>
>AFAIK you can't use a 15 GB tmpfs on i386 because large memory support
>is basically a hack to support multiple 4GB memory spaces (some VM guru
>correct me if I'm wrong).

I'm no VM guru but I have a 32 bit machine here with 8 GB of
memory and 8 GB of swap:

# mount -t tmpfs -o size=$((12*1024*1024*1024)) tmpfs /mnt
# df
Filesystem   1K-blocks  Used Available Use% Mounted on
/dev/sda1 19228276   1200132  17051396   7% /
tmpfs 12582912 0  12582912   0% /mnt

There you go, a 12 GB tmpfs. I haven't tried to create a 12 GB
file on it, though, since this is a production machine and it
needs the memory ..

So yes that appears to work just fine.

Mike.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-20 Thread Antonio Vargas
On 7/20/05, Erik Mouw <[EMAIL PROTECTED]> wrote:
> On Wed, Jul 20, 2005 at 02:16:36PM +0200, Bastiaan Naber wrote:
> > I have a 15 GB file which I want to place in memory via tmpfs. I want to do
> > this because I need to have this data accessible with a very low seek time.
> 
> That should be no problem on a 64 bit architecture.
> 
> > I want to know if this is possible before spending 10,000 euros on a machine
> > that has 16 GB of memory.
> 
> If you want to spend that amount of money on memory anyway, the extra
> cost for an AMD64 machine isn't that large.
> 
> > The machine we plan to buy is a HP Proliant Xeon machine and I want to run a
> > 32 bit linux kernel on it (the xeon we want doesn't have the 64-bit stuff
> > yet)
> 
> AFAIK you can't use a 15 GB tmpfs on i386 because large memory support
> is basically a hack to support multiple 4GB memory spaces (some VM guru
> correct me if I'm wrong). Just get an Athlon64 machine and run a 64 bit
> kernel on it. If compatibility is a problem, you can still run a 32 bit
> i386 userland on an x86_64 kernel.
> 
> 
> Erik
> 
> --
> +-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
> | Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
> -

Bastian, Erik is dead-on on that one: go 64bit and forget all worries
about your 15GB filesize. Just don't forget to look not only x86-64
(intel or amd) but also itanium, ppc64 and s390 machines, you never
know about surprises!

-- 
Greetz, Antonio Vargas aka winden of network

http://wind.codepixel.com/

Las cosas no son lo que parecen, excepto cuando parecen lo que si son.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: a 15 GB file on tmpfs

2005-07-20 Thread Erik Mouw
On Wed, Jul 20, 2005 at 02:16:36PM +0200, Bastiaan Naber wrote:
> I have a 15 GB file which I want to place in memory via tmpfs. I want to do 
> this because I need to have this data accessible with a very low seek time.

That should be no problem on a 64 bit architecture.

> I want to know if this is possible before spending 10,000 euros on a machine 
> that has 16 GB of memory. 

If you want to spend that amount of money on memory anyway, the extra
cost for an AMD64 machine isn't that large.

> The machine we plan to buy is a HP Proliant Xeon machine and I want to run a 
> 32 bit linux kernel on it (the xeon we want doesn't have the 64-bit stuff 
> yet)

AFAIK you can't use a 15 GB tmpfs on i386 because large memory support
is basically a hack to support multiple 4GB memory spaces (some VM guru
correct me if I'm wrong). Just get an Athlon64 machine and run a 64 bit
kernel on it. If compatibility is a problem, you can still run a 32 bit
i386 userland on an x86_64 kernel.


Erik

-- 
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


a 15 GB file on tmpfs

2005-07-20 Thread Bastiaan Naber
Hi,

I am not sure if I can ask this here but I could not find any other place 
where I could fine anyone with this knowledge.

I have a 15 GB file which I want to place in memory via tmpfs. I want to do 
this because I need to have this data accessible with a very low seek time.

I want to know if this is possible before spending 10,000 euros on a machine 
that has 16 GB of memory. 

The machine we plan to buy is a HP Proliant Xeon machine and I want to run a 
32 bit linux kernel on it (the xeon we want doesn't have the 64-bit stuff 
yet)

Thanks in advance,
Bastiaan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/