Re: 64 bit Perl memory test...
NIPP, SCOTT V (SBCSI) wrote: Very interesting... Running this script it dies at the 800MB attempt. However, watching this process in top, memory usage is actually double the amount that is being tested. In top, the last memory amount prior to it dying is 1400MB. Could be that the data is atctually using only the first 32 bits for each 64 allocated. That would mean that it would take twice as much memory to store the same amount of data. Gotta get to work. I'll try to expand on this later. Joseph -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] http://learn.perl.org/ http://learn.perl.org/first-response
Re: 64 bit Perl memory test...
R. Joseph Newton wrote:
NIPP, SCOTT V (SBCSI) wrote:
Very interesting... Running this script it dies at the 800MB
attempt. However, watching this process in top, memory usage is actually
double the amount that is being tested. In top, the last memory amount
prior to it dying is 1400MB.
Could be that the data is atctually using only the first 32 bits for each
64
allocated. That would mean that it would take twice as much memory to
store the
same amount of data. Gotta get to work. I'll try to expand on this
later.
another possible explanation is when your program is doing:
allocate 100mb memory to $i
undef $i
allocate 200mb memory to $i
undef $i
allocate 300mb memory to $i
undef $i
...
allocate 800mb memory to $i
undef $i
...
at the end of your program, you are not really only using 800mb of memory.
the 'undef $i' statment only tells Perl to free the memory $i is using if
there is no other reference to it. it does NOT return the memory back to
the os. which means all the memory allocated before the 800mb request are
still held by Perl but again this does not neccessary mean Perl is holding
100mb + 200mb + 300mb + ... + 800mb memory though. for example, when your
program is requesting 500mb of memory, Perl maybe able to find that by
reusing the undef-ed 100mb + 400mb previous allocated so it doesn't have to
ask the os for that. the behavior whether Perl will go ask the os for
memory or reuse previous allocated is unpredictable and depends on many
other factors at the time of the program is running. as a Perl programmer,
we have very little control over this behavior. if you must go with this
approach, i would trust top/ps whether than what the script output but
again, you should research a bit and see if there are tools designed
specially for this purpose. the toy-script approach can only get you this
far.
david
--
s,.*,,e,y,\n,,d,y,.s,10,,s
.ss.s.s...s.sss.s.ss
s.s.s...s...s..s
...s.ss..s.sss..ss.sss.s
s.s.s...ss.sss.s
..s..sss.s.ss.sss...
..ssss.sss.sss.s
,{4},|?{*=}_'y!'+0!$;
,ge,y,!#:$_(-*[./[EMAIL PROTECTED],b-t,
.y...,$~=q~=?,;^_#+?{~,,$~=~
y.!-*-/:[EMAIL PROTECTED] ().;s,;,
);,g,s,s,$~s,g,y,y,%,,g,eval
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/ http://learn.perl.org/first-response
Re: 64 bit Perl memory test...
It was Thursday, December 04, 2003 when Bakken, Luke took the soap box, saying:
: how they were monitoring and/or determining this. Basically,
: I would like
: help with a Perl test script to use memory until Perl reaches
: it's memory
: limit. This will help me to know when I have things right so
: I don't have
: to keep going back to the end users and tell them OK, try it now.
: Thanks in advance for any help.
:
: use strict;
: my $bigbuf;
: my $buf;
: open IN, '/dev/random' or die Can't open random device: $!;
: while( read(IN, $buf, 16384) 0 ) {
: $bigbuf .= $buf;
: }
The original poster was having trouble with data from Oracle, not the
filesystem. I'm not quite sure what you're trying to demonstrate
here.
Casey West
--
Shooting yourself in the foot with Oracle
You decide to shoot yuourself in the foot and go out to buy a gun -
except the gun won't work without deploying a shoulder holster
solution, and relational titanium alloy bullets, and body armour
infrastructure, and a laser sight assistant, and a retractable arm
stock application, and an enterprise team of ballistics experts and a
chiropodist.
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
[First Response Service] Re: 64 bit Perl memory test...
Hi. I'm in the employ of Casey West, a list admin, to assist you with your question. I've taken the liberty to search Google using the Subject line you provided in your email to the list. I hope one of the links below will be of service to you. Sadly Google hasn't given us a nice, legal API for searching newsgroups, so you may also want to try these searches. http://groups.google.com/groups?q=Perl+64+bit+Perl+memory+test%2E%2E%2E http://groups.google.com/groups?q=64+bit+Perl+memory+test%2E%2E%2E+group%3Acomp.lang.perl.* If you didn't get any useful results below, there's a chance your Subject line was not specific enough, or not detailed enough. For example, the following subject lines are not very good choices. Subject: Doubt Subject: HELP! Subject: Problem On the other hand, it's possible that your question is unique, or the search needs a human touch to get just the right results. Enjoy! Perl.org Beginners' Lists, 0 results. Searched: site:nntp.x.perl.org inurl:perl.beginners -inurl:show_headers 64 bit Perl memory test... Perl.org Lists, 9 results. Searched: site:nntp.x.perl.org -inurl:show_headers 64 bit Perl memory test... [1] nntp.perl.org - perl.golf (2093) http://nntp.x.perl.org/group/perl.golf/2093 ... the size of the basic perl structures vary quite a bit in 32- versus 64-bit perl). ... In contrast, my 49 solution uses 129M of memory to solve test 24 (the ... [2] nntp.perl.org - perl.dbi.announce (137) http://nntp.x.perl.org/group/perl.dbi.announce/137 ... (note that perl itself must also be built in 64 bit mode for ... Improved syb_err_handler handling (thanks to Matthew Persico) Fixed memory leak when opening ... [3] nntp.perl.org - perl.cpan.testers (32341) http://nntp.x.perl.org/group/perl.cpan.testers/32341 ... mips4 -TARG:platform=ip32'; perl -V:ccflags ... RLIM32_INFINITY 0x7fff #elif (_MIPS_SZLONG == 64) || (_MIPS_SIM == _ABIN32 ... Convert the given 32 bit limit spec ... [4] nntp.perl.org - perl.perl6.internals (14713) http://nntp.x.perl.org/group/perl.perl6.internals/14713 ... at]sidhe.org, perl6-internals[at]perl.org Subject ... packfile (on disk) or in the bytecode (in memory). ... 32-bit number of bytes (or maybe 64-bit) DATA: arbitrary ... [5] nntp.perl.org - perl.perl5.summary (64) http://nntp.x.perl.org/group/perl.perl5.summary/64 perl.perl5.summary (64). ... it may be a bit outdated though ... Weekly summaries are published on http://use.perl.org/ and on a mailing list, which subscription address ... [6] nntp.perl.org - perl.perl5.porters (84975) http://nntp.x.perl.org/group/perl.perl5.porters/84975 ... I can't seem to get Solaris to build as 64 bit. ... perl5/5.8.2/sun4-solaris-thread-multi-64/CORE privlib ... perl will # require -Ilib tags: TAGS perl emacs/e2ctags ... [7] nntp.perl.org - perl.perl5.changes (9220) http://nntp.x.perl.org/group/perl.perl5.changes/9220 ... item Calls to external programs - -=item Memory allocation - -=item ... Perl on OS/390 - -=item Build, Test, Install Perl ... over 4 - -=item 64-bit perl on Solaris. ... [8] nntp.perl.org - perl.cvs.parrot (5413) http://nntp.x.perl.org/group/perl.cvs.parrot/5413 ... c From: mrjoltcola[at]cvs.perl.org (Melvin ... char *yy_c_buf_p = (char *) 0; @@ -225,64 +293,100 @@ * instead ... YY_G(yy_c_buf_p); /* cast for 8-bit char's */ + *YY_G ... [9] nntp.perl.org - perl.perl5.porters (70689) http://nntp.x.perl.org/group/perl.perl5.porters/70689 ... We won't be including memory.h. I'll use memmove ... Checking how to print 64-bit integers... ... Checking the format strings to be used for Perl's internal types... ... search.cpan.org, 10 results. Searched: site:search.cpan.org 64 bit Perl memory test... [1] search.cpan.org: README.tru64 - Perl version 5 on Tru64 ( ... http://search.cpan.org/~jhi/perl-5.8.1/README.tru64 ... the toke.c gcc likes to have a lot of memory, 256 megabytes ... include and -Dloclibpth=/some/lib and before running make test setting your ... 64-bit Perl on Tru64. ... [2] Revision history for Perl extension BerkeleyDB. http://search.cpan.org/src/PMQS/BerkeleyDB-0.25/Changes ... 0.14 21st January 2001 * Silenced the warnings when build with a 64-bit Perl. ... or database failed, there was a small memory leak. ... A thread-enabled Perl it could ... [3] search.cpan.org: README.solaris - Perl version 5 on Solaris ... http://search.cpan.org/~rgarcia/perl-5.6.2/README.solaris ... To compile a 64-bit application on an UltraSparc with ... malloc has trouble allocating more than 2GB of memory. ... common error when trying to build perl
RE: 64 bit Perl memory test...
It was Thursday, December 04, 2003 when Bakken, Luke took the
soap box, saying:
: how they were monitoring and/or determining this. Basically,
: I would like
: help with a Perl test script to use memory until Perl reaches
: it's memory
: limit. This will help me to know when I have things right so
: I don't have
: to keep going back to the end users and tell them OK,
try it now.
:Thanks in advance for any help.
:
: use strict;
: my $bigbuf;
: my $buf;
: open IN, '/dev/random' or die Can't open random device: $!;
: while( read(IN, $buf, 16384) 0 ) {
: $bigbuf .= $buf;
: }
The original poster was having trouble with data from Oracle, not the
filesystem. I'm not quite sure what you're trying to demonstrate
here.
Casey West
It's bad enough that your spamming the list with useless posts, but here
you're demonstrating a complete inability to read what a person wants.
He said:
Basically, I would like help with a Perl test script to use memory
until Perl reaches it's memory limit.
That is exactly what this bit of code does - it fills $bigbuf with data
in 16K chunks until the OS kills it for using too much memory.
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
64 bit Perl memory test...
I have been asked to get a 64bit version of Perl compiled and working under HP-UX 11i. I have found some helpful hints on actually compiling this successfully. The reason for the 64bit version is memory utilization. My users have a script that processes an Oracle database of about 15 million records, and they informed me that the process requires about 5GB of memory. When attempting to run this in the standard (32bit) Perl, the process would fail at about 1GB of memory usage. I am not sure how they were monitoring and/or determining this. Basically, I would like help with a Perl test script to use memory until Perl reaches it's memory limit. This will help me to know when I have things right so I don't have to keep going back to the end users and tell them OK, try it now. Thanks in advance for any help. Scott Nipp Phone: (214) 858-1289 E-mail: [EMAIL PROTECTED] Web: http:\\ldsa.sbcld.sbc.com -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 64 bit Perl memory test...
It was Thursday, December 04, 2003 when Bakken, Luke took the soap box, saying:
: It was Thursday, December 04, 2003 when Bakken, Luke took the
: soap box, saying:
: : how they were monitoring and/or determining this. Basically,
: : I would like
: : help with a Perl test script to use memory until Perl reaches
: : it's memory
: : limit. This will help me to know when I have things right so
: : I don't have
: : to keep going back to the end users and tell them OK,
: try it now.
: : Thanks in advance for any help.
: :
: : use strict;
: : my $bigbuf;
: : my $buf;
: : open IN, '/dev/random' or die Can't open random device: $!;
: : while( read(IN, $buf, 16384) 0 ) {
: : $bigbuf .= $buf;
: : }
:
: The original poster was having trouble with data from Oracle, not the
: filesystem. I'm not quite sure what you're trying to demonstrate
: here.
:
: Basically, I would like help with a Perl test script to use memory
: until Perl reaches it's memory limit.
I missed that part, yes.
: That is exactly what this bit of code does - it fills $bigbuf with data
: in 16K chunks until the OS kills it for using too much memory.
Excellent.
Casey West
--
Good Idea: Playing the piccolo in a marching band.
Bad Idea: Playing the piano in a marching band.
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: 64 bit Perl memory test...
how they were monitoring and/or determining this. Basically,
I would like
help with a Perl test script to use memory until Perl reaches
it's memory
limit. This will help me to know when I have things right so
I don't have
to keep going back to the end users and tell them OK, try it now.
Thanks in advance for any help.
use strict;
my $bigbuf;
my $buf;
open IN, '/dev/random' or die Can't open random device: $!;
while( read(IN, $buf, 16384) 0 ) {
$bigbuf .= $buf;
}
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: 64 bit Perl memory test...
Unfortunately, there is no /dev/random in HP-UX.
Scott Nipp
Phone: (214) 858-1289
E-mail: [EMAIL PROTECTED]
Web: http:\\ldsa.sbcld.sbc.com
-Original Message-
From: Bakken, Luke [mailto:[EMAIL PROTECTED]
Sent: Thursday, December 04, 2003 9:36 AM
To: NIPP, SCOTT V (SBCSI); [EMAIL PROTECTED]
Subject: RE: 64 bit Perl memory test...
how they were monitoring and/or determining this. Basically,
I would like
help with a Perl test script to use memory until Perl reaches
it's memory
limit. This will help me to know when I have things right so
I don't have
to keep going back to the end users and tell them OK, try it now.
Thanks in advance for any help.
use strict;
my $bigbuf;
my $buf;
open IN, '/dev/random' or die Can't open random device: $!;
while( read(IN, $buf, 16384) 0 ) {
$bigbuf .= $buf;
}
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: 64 bit Perl memory test...
Scott V Nipp wrote:
Unfortunately, there is no /dev/random in HP-UX.
fortunately, you don't need /dev/random. any of the following should eat up
all of your machine's memory sooner or later:
[panda]# perl -e '$#a={}'
[panda]# perl -e '$a[{}]=1'
[panda]# perl -e '$#a+=0x while 1'
[panda]# perl -e '$i.=0x0x while 1'
david
--
s,.*,,e,y,\n,,d,y,.s,10,,s
.ss.s.s...s.sss.s.ss
s.s.s...s...s..s
...s.ss..s.sss..ss.sss.s
s.s.s...ss.sss.s
..s..sss.s.ss.sss...
..ssss.sss.sss.s
,{4},|?{*=}_'y!'+0!$;
,ge,y,!#:$_(-*[./[EMAIL PROTECTED],b-t,
.y...,$~=q~=?,;^_#+?{~,,$~=~
y.!-*-/:[EMAIL PROTECTED] ().;s,;,
);,g,s,s,$~s,g,y,y,%,,g,eval
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/ http://learn.perl.org/first-response
RE: 64 bit Perl memory test...
Will this tell me how much memory is used at the point of failure?
Scott Nipp
Phone: (214) 858-1289
E-mail: [EMAIL PROTECTED]
Web: http:\\ldsa.sbcld.sbc.com
-Original Message-
From: david [mailto:[EMAIL PROTECTED]
Sent: Thursday, December 04, 2003 1:11 PM
To: [EMAIL PROTECTED]
Subject: RE: 64 bit Perl memory test...
Scott V Nipp wrote:
Unfortunately, there is no /dev/random in HP-UX.
fortunately, you don't need /dev/random. any of the following should eat up
all of your machine's memory sooner or later:
[panda]# perl -e '$#a={}'
[panda]# perl -e '$a[{}]=1'
[panda]# perl -e '$#a+=0x while 1'
[panda]# perl -e '$i.=0x0x while 1'
david
--
s,.*,,e,y,\n,,d,y,.s,10,,s
.ss.s.s...s.sss.s.ss
s.s.s...s...s..s
...s.ss..s.sss..ss.sss.s
s.s.s...ss.sss.s
..s..sss.s.ss.sss...
..ssss.sss.sss.s
,{4},|?{*=}_'y!'+0!$;
,ge,y,!#:$_(-*[./[EMAIL PROTECTED],b-t,
.y...,$~=q~=?,;^_#+?{~,,$~=~
y.!-*-/:[EMAIL PROTECTED] ().;s,;,
);,g,s,s,$~s,g,y,y,%,,g,eval
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/ http://learn.perl.org/first-response
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/ http://learn.perl.org/first-response
RE: 64 bit Perl memory test...
Will this tell me how much memory is used at the point
of failure?
See code.
Unfortunately, there is no /dev/random in HP-UX.
But there is a /dev/zero
use strict;
$|++;
my $mem;
my $bigbuf;
my $buf;
open IN, '/dev/zero' or die Can't open random device: $!;
while( $mem += read(IN, $buf, 1048576) ) {
print $mem\n;
$bigbuf .= $buf;
}
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/ http://learn.perl.org/first-response
RE: 64 bit Perl memory test...
Scott V Nipp wrote:
Will this tell me how much memory is used at the point of failure?
no, it does not. you will have to do that manually. either watch the script
in top or ps or write something like:
#!/usr/bin/perl -w
use strict;
my $m = 1024 * 1000;
my $i = '';
for($m * 100, $m * 200, $m * 300, $m * 400,
$m * 500, $m * 600, $m * 700, $m * 800){
print Allocating $_ bytes memory...\n;
$i = 0 x $_;
undef $i;
}
__END__
the amount of memory that this script is capable of utilizing at the time of
testing does not equal to the amount of memory that are allowed by the OS
though. for example, if the script failed when asking for 800mb of memory,
does it mean the OS is unable to find 800mb of memory or does it mean the
process is not allowed to allocate this much memory? you might have better
luck finding a tool designed for testing memory allocation for your
machine.
david
--
s,.*,,e,y,\n,,d,y,.s,10,,s
.ss.s.s...s.sss.s.ss
s.s.s...s...s..s
...s.ss..s.sss..ss.sss.s
s.s.s...ss.sss.s
..s..sss.s.ss.sss...
..ssss.sss.sss.s
,{4},|?{*=}_'y!'+0!$;
,ge,y,!#:$_(-*[./[EMAIL PROTECTED],b-t,
.y...,$~=q~=?,;^_#+?{~,,$~=~
y.!-*-/:[EMAIL PROTECTED] ().;s,;,
);,g,s,s,$~s,g,y,y,%,,g,eval
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/ http://learn.perl.org/first-response
RE: 64 bit Perl memory test...
Nope... No /dev/zero either in 11i.
Scott Nipp
Phone: (214) 858-1289
E-mail: [EMAIL PROTECTED]
Web: http:\\ldsa.sbcld.sbc.com
-Original Message-
From: Bakken, Luke [mailto:[EMAIL PROTECTED]
Sent: Thursday, December 04, 2003 1:25 PM
To: NIPP, SCOTT V (SBCSI); [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: RE: 64 bit Perl memory test...
Will this tell me how much memory is used at the point
of failure?
See code.
Unfortunately, there is no /dev/random in HP-UX.
But there is a /dev/zero
use strict;
$|++;
my $mem;
my $bigbuf;
my $buf;
open IN, '/dev/zero' or die Can't open random device: $!;
while( $mem += read(IN, $buf, 1048576) ) {
print $mem\n;
$bigbuf .= $buf;
}
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/ http://learn.perl.org/first-response
RE: 64 bit Perl memory test...
Very interesting... Running this script it dies at the 800MB
attempt. However, watching this process in top, memory usage is actually
double the amount that is being tested. In top, the last memory amount
prior to it dying is 1400MB.
Scott Nipp
Phone: (214) 858-1289
E-mail: [EMAIL PROTECTED]
Web: http:\\ldsa.sbcld.sbc.com
-Original Message-
From: david [mailto:[EMAIL PROTECTED]
Sent: Thursday, December 04, 2003 2:17 PM
To: [EMAIL PROTECTED]
Subject: RE: 64 bit Perl memory test...
Scott V Nipp wrote:
Will this tell me how much memory is used at the point of failure?
no, it does not. you will have to do that manually. either watch the script
in top or ps or write something like:
#!/usr/bin/perl -w
use strict;
my $m = 1024 * 1000;
my $i = '';
for($m * 100, $m * 200, $m * 300, $m * 400,
$m * 500, $m * 600, $m * 700, $m * 800){
print Allocating $_ bytes memory...\n;
$i = 0 x $_;
undef $i;
}
__END__
the amount of memory that this script is capable of utilizing at the time of
testing does not equal to the amount of memory that are allowed by the OS
though. for example, if the script failed when asking for 800mb of memory,
does it mean the OS is unable to find 800mb of memory or does it mean the
process is not allowed to allocate this much memory? you might have better
luck finding a tool designed for testing memory allocation for your
machine.
david
--
s,.*,,e,y,\n,,d,y,.s,10,,s
.ss.s.s...s.sss.s.ss
s.s.s...s...s..s
...s.ss..s.sss..ss.sss.s
s.s.s...ss.sss.s
..s..sss.s.ss.sss...
..ssss.sss.sss.s
,{4},|?{*=}_'y!'+0!$;
,ge,y,!#:$_(-*[./[EMAIL PROTECTED],b-t,
.y...,$~=q~=?,;^_#+?{~,,$~=~
y.!-*-/:[EMAIL PROTECTED] ().;s,;,
);,g,s,s,$~s,g,y,y,%,,g,eval
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/ http://learn.perl.org/first-response
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
http://learn.perl.org/ http://learn.perl.org/first-response
