[OT] MLDBM Size Limit??

2003-01-21 Thread Jim Morrison [Mailinglists] 
Guys,

Sorry this is a little off topic...  Is there a size limit on DBM's? (Or
Linux files for that matter.. )

I've thrown some details of my box/code below..

Thing is I'm getting a "write error" and it seems to always happen when
the DBM gets to 2.0Gb .. (you may think I'm mad for trying to make 2G
DBM's - but they don't usually get more than a couple of 100k - it's
just a big site.. Honest ;-)

Anyway - if there's anyone with similar experience or anyone has some
advice for me it would be appreciated..

Thanks,
J.

Here are the details..


 :
 write error at /usr/local/lib/perl5/site_perl/5.8.0/MLDBM.pm line 161.

 :
 $s->{DB}->STORE($k, $v);


 use GDBM_File;
 use MLDBM qw(GDBM_File Storable);


 This is perl, v5.8.0 built for i686-linux


 Linux hope 2.4.7-10 #1 Thu Sep 6 16:46:36 EDT 2001 i686 unknown



Jim Morrison
_
Technology & Development Partner
Isotope LLP
9, 2 Laura Place
Bath, BA2 4BH
UK
+44  (0) 1225 444674
+44  (0) 7940 937822
www.mediaisotope.com

[OT] MLDBM size limit?? :'(

2003-01-25 Thread Jim Morrison 
Guys,

Sorry this is a little off topic...  Is there a size limit on DBM's? (Or
Linux files for that matter.. )

I've thrown some details of my box/code below..

Thing is I'm getting a "write error" and it seems to always happen when
the DBM gets to 2.0Gb .. (you may think I'm mad for trying to make 2G
DBM's - but they don't usually get more than a couple of 100k - it's
just a big site.. Honest ;-)

Anyway - if there's anyone with similar experience or anyone has some
advice for me it would be appreciated..

Thanks,
J.

Here are the details..


 :
 write error at /usr/local/lib/perl5/site_perl/5.8.0/MLDBM.pm line 161.

 :
 $s->{DB}->STORE($k, $v);


 use GDBM_File;
 use MLDBM qw(GDBM_File Storable);


 This is perl, v5.8.0 built for i686-linux


 Linux hope 2.4.7-10 #1 Thu Sep 6 16:46:36 EDT 2001 i686 unknown



Jim Morrison
_
Technology & Development Partner
Isotope LLP
9, 2 Laura Place
Bath, BA2 4BH
UK
+44  (0) 1225 444674
+44  (0) 7940 937822
www.mediaisotope.com

Re: [OT] MLDBM Size Limit??

2003-01-21 Thread Ged Haywood 
Hi there,

On Tue, 21 Jan 2003, Jim Morrison [Mailinglists] wrote:

> Sorry this is a little off topic...  Is there a size limit on DBM's? (Or
> Linux files for that matter.. )
[snip]
> Thing is I'm getting a "write error" and it seems to always happen when
> the DBM gets to 2.0Gb .. 
[snip]
>  Linux hope 2.4.7-10 #1 Thu Sep 6 16:46:36 EDT 2001 i686 unknown

I only have gdbm, which doesn't have at least some of the dbm restrictions,
so I don't know about dbm files.  There are some cautions about dbm files
in the mod_perl Guide.

For Linux files it depends on the filesystem you're using and how it was
initialized.  For example I use ext2 (most will), and in my copy of the
documentation (it's for 2.4.19) it gives a list of file and filesystem
sizes for different block sizes.  (I won't post the whole thing as it's
over 17kBytes:).

--
Filesystem block size: 1kB2kB4kB8kB

File size limit:  16GB  256GB 2048GB 2048GB
Filesystem size limit:  2047GB 8192GB16384GB32768GB

There is a 2.4 kernel limit of 2048GB for a single block device, so no
filesystem larger than that can be created at this time.  There is also
an upper limit on the block size imposed by the page size of the kernel,
so 8kB blocks are only allowed on Alpha systems (and other architectures
which support larger pages).
--

Check out linux-2.4.7/Documentation/filesystems/ext2.txt or whatever
you have on your system.  You might want to consider using gdbm, your
data could then be "as large as you want" according to the manpage...

73,
Ged.

Re: [OT] MLDBM Size Limit??

2003-01-21 Thread Robert Landrum 
On Tue, Jan 21, 2003 at 02:37:02PM +, Ged Haywood wrote:
> Hi there,
> 
> On Tue, 21 Jan 2003, Jim Morrison [Mailinglists] wrote:
> 
> > Sorry this is a little off topic...  Is there a size limit on DBM's? (Or
> > Linux files for that matter.. )
> [snip]
> > Thing is I'm getting a "write error" and it seems to always happen when
> > the DBM gets to 2.0Gb .. 
> [snip]
> >  Linux hope 2.4.7-10 #1 Thu Sep 6 16:46:36 EDT 2001 i686 unknown
> 

Actually, I finally had to upgrade to 2.4.17 (or .19) and custom compile the
kernel to get this to work.  I had a similar problem with mysql files
being 12GB.

I forget exactly the steps, but if you hunt around in the kernel configurator
I'm sure it'll show up...  :)

Rob

Re: [OT] MLDBM size limit?? :'(

2003-01-25 Thread Stas Bekman 
Jim Morrison wrote:

Guys,

Sorry this is a little off topic...  Is there a size limit on DBM's? (Or
Linux files for that matter.. )

I've thrown some details of my box/code below..

Thing is I'm getting a "write error" and it seems to always happen when
the DBM gets to 2.0Gb .. (you may think I'm mad for trying to make 2G
DBM's - but they don't usually get more than a couple of 100k - it's
just a big site.. Honest ;-)


You need to have Perl built with largefiles support to work with files larger 
than 2GB. Do:
./Configure -Duselargefiles [...] && make install
(check if you don't have it already via 'perl -V|grep uselargefiles')
and afterwards you need to rebuild mod_perl so it'll pick the new compile 
flags. You may need to have Apache compiled with largefiles support as well, 
which normally happens automatically if your fs/os supports it. Or it can be 
forced with:
% CFLAGS="D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" ./configure [...]

__
Stas BekmanJAm_pH --> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com