Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-05-13 Thread Martin Buchholz
OK, I committed the fix for this. I also added some gratuitous tests to the new test case. Martin On Thu, May 13, 2010 at 15:46, David Holmes wrote: > CR: 6952330 Fix for 6933217 broke contract of StringBuffer.ensureCapacity > > Thanks, > David > > David Holmes said the following on 05/13/10 11:

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-05-13 Thread David Holmes
CR: 6952330 Fix for 6933217 broke contract of StringBuffer.ensureCapacity Thanks, David David Holmes said the following on 05/13/10 11:59: Hi Martin, Bugtraq is offline so I can't file a CR right now. The was caught by Mauve tests. Kelly sent a link to the mailing list but I don't think he's

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-05-12 Thread David Holmes
Hi Martin, Bugtraq is offline so I can't file a CR right now. The was caught by Mauve tests. Kelly sent a link to the mailing list but I don't think he's a member so it's probably held up for approval. Martin Buchholz said the following on 05/13/10 11:38: Webrev at http://cr.openjdk.java.ne

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-05-12 Thread Martin Buchholz
Webrev at http://cr.openjdk.java.net/~martin/webrevs/openjdk7/StringBuffer-capacity/StringBuffer-capacity.patch Martin On Wed, May 12, 2010 at 18:23, Martin Buchholz wrote: > Alright, I'm convinced this is a bug.  Please file one in bugtraq > (or expunge from tl in the unlikely event that would

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-05-12 Thread Martin Buchholz
Alright, I'm convinced this is a bug. Please file one in bugtraq (or expunge from tl in the unlikely event that would be permitted). It appears we very carefully specified the capacity manipulation behavior, but didn't have any jtreg tests for it. Do y'all run the Mauve tests? (Probably a good id

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-05-12 Thread Kelly O'Hair
If it helps, I found an open test that should demonstrate it: http://www.sourceware.org/cgi-bin/cvsweb.cgi/mauve/gnu/testlet/java/lang/StringBuffer/StringBufferTest.java?rev=1.7&content-type=text/x-cvsweb-markup&cvsroot=mauve -kto On May 12, 2010, at 5:31 PM, David Holmes wrote: Chris, Marti

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-05-12 Thread David Holmes
Chris, Martin, The changes to AbstractStringBuilder violate the specification for ensureCapacity: it has to grow from N to 2N+2. The new code in: http://hg.openjdk.java.net/jdk7/tl-gate/jdk/rev/ec45423a4700 void expandCapacity(int minimumCapacity) { - int newCapacity = (value.length + 1) * 2

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-04-16 Thread Martin Buchholz
On Fri, Apr 16, 2010 at 03:17, David Holmes wrote: > Hi Martin, > > Martin Buchholz said the following on 04/16/10 16:09: >> >> I recently discovered another place to handle huge arrays better - in >> AbstractCollection. >> I've put those changes into >> http://cr.openjdk.java.net/~martin/webrevs/

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-04-16 Thread David Holmes
Hi Martin, Martin Buchholz said the following on 04/16/10 16:09: I recently discovered another place to handle huge arrays better - in AbstractCollection. I've put those changes into http://cr.openjdk.java.net/~martin/webrevs/openjdk7/ArrayResize2/ I don't understand what you are doing here:

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-04-16 Thread Chris Hegarty
Martin Buchholz wrote: Hi Chris, I recently discovered another place to handle huge arrays better - in AbstractCollection. I've put those changes into http://cr.openjdk.java.net/~martin/webrevs/openjdk7/ArrayResize2/ I propose to qfold these into the original changes for this bug http://cr.open

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-04-15 Thread Martin Buchholz
Hi Chris, I recently discovered another place to handle huge arrays better - in AbstractCollection. I've put those changes into http://cr.openjdk.java.net/~martin/webrevs/openjdk7/ArrayResize2/ I propose to qfold these into the original changes for this bug http://cr.openjdk.java.net/~martin/webre

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-10 Thread Martin Buchholz
On Wed, Mar 10, 2010 at 01:58, Goktug Gokdogan wrote: > Similarly, >   BitSet.ensureCapacity I don't think BitSet has this problem, because the bits are stored in longs, so the array can never overflow. But don't believe me - prove me wrong! >   AbstractStringBuilder.expandCapacity Yup. >   V

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-10 Thread Martin Buchholz
On Wed, Mar 10, 2010 at 09:36, Ulf Zibis wrote: > Am 10.03.2010 00:22, schrieb Martin Buchholz: >> >> On Tue, Mar 9, 2010 at 15:11, Ulf Zibis  wrote: >> >> >>> >>> Can you explain the mystery about "+ 2" ? >>> >> >> It's exactly the same as the old resizing behavior. > > In detail I meant, if you

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-10 Thread Ulf Zibis
Am 10.03.2010 00:22, schrieb Martin Buchholz: On Tue, Mar 9, 2010 at 15:11, Ulf Zibis wrote: Can you explain the mystery about "+ 2" ? It's exactly the same as the old resizing behavior. In detail I meant, if you have any idea, why the original designers could have chosen the "+1

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-10 Thread Goktug Gokdogan
Similarly, BitSet.ensureCapacity AbstractStringBuilder.expandCapacity Vector.ensureCapacityHelper methods need to have similar checks and/or throw proper exceptions. By the way, I did not understand why IdentityHashMap and HashMap have different MAXIMUM_CAPACITY and different logic to handle

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Martin Buchholz
On Tue, Mar 9, 2010 at 15:11, Ulf Zibis wrote: > Can you explain the mystery about "+ 2" ? It's exactly the same as the old resizing behavior. Martin

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Ulf Zibis
Am 09.03.2010 23:08, schrieb Martin Buchholz: On Tue, Mar 9, 2010 at 13:08, Ulf Zibis wrote: [1] current PriorityQueue snippet: ... int newCapacity = ((oldCapacity< 64)? ((oldCapacity + 1) * 2): ((oldCapacity / 2) * 3)); ...

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Martin Buchholz
On Tue, Mar 9, 2010 at 13:08, Ulf Zibis wrote: > > [1] current PriorityQueue snippet: > ... >     int newCapacity = ((oldCapacity < 64)? >    ((oldCapacity + 1) * 2): >    ((oldCapacity / 2) * 3)); > ... > [2] new PriorityQueue snippet: > ... >  

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Ulf Zibis
Am 09.03.2010 21:02, schrieb Osvaldo Doederlein: Should we really consider this a VM bug? I'm not sure that it's a good idea to allocate a single object which size exceeds 4Gb (for a byte[] - due to the object header and array size field) - even on a 64-bit VM. An array with 2^32 elements is im

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Ulf Zibis
Am 09.03.2010 20:18, schrieb Martin Buchholz: On Tue, Mar 9, 2010 at 03:59, Ulf Zibis wrote: In PriorityQueue: let's result newCapacity in 0x.FFFC =-4 then "if (newCapacity - MAX_ARRAY_SIZE> 0)" ---> false then Arrays.copyOf(queue, newCapacity) ---> ArrayIndexOutOfBoundsException

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Martin Buchholz
It surely is not a good idea to use a single backing array for huge arrays. As you point out, it's up to 32GB for just one object. But the core JDK doesn't offer a suitable alternative for users who need very large collections. It would have been more in the spirit of Java to have a collection c

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Martin Buchholz
ava spec, or will this break on an >> obscure machine that does not use, say, two's complement arithmetic? >> >> Regards, >> >> Kevin >> >> 2010/3/9 Dmytro Sheyko >>> >>> Is there any reason to use comparison like this

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Osvaldo Doederlein
Should we really consider this a VM bug? I'm not sure that it's a good idea to allocate a single object which size exceeds 4Gb (for a byte[] - due to the object header and array size field) - even on a 64-bit VM. An array with 2^32 elements is impossible, the maximum allowed by the size field is 2^

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Martin Buchholz
On Tue, Mar 9, 2010 at 03:59, Ulf Zibis wrote: > In PriorityQueue: > > let's result newCapacity in 0x.FFFC  =-4 > then "if (newCapacity - MAX_ARRAY_SIZE > 0)" ---> false > then Arrays.copyOf(queue, newCapacity) ---> ArrayIndexOutOfBoundsException How could newCapacity ever become -4? Since gr

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Ulf Zibis
Am 09.03.2010 12:02, schrieb Kevin L. Stern: I did a quick search and it appears that Java is indeed two's complement based. Nonetheless, please allow me to point out that, in general, this type of code worries me since I fully expect that at some point someone will come along and do exactly w

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Kevin L. Stern
>> Kevin >> >> 2010/3/9 Dmytro Sheyko >> >> Is there any reason to use comparison like this >>> >>> if (newCapacity - minCapacity < 0) >>> >>> if (newCapacity - MAX_ARRAY_SIZE > 0) { >>> >>> instead of &

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Christopher Hegarty -Sun Microsystems Ireland
Sorry Martin, I appear to have missed your original request to file this bug. I since filed the following: 6933217: Huge arrays handled poorly in core libraries The changes you are proposing seem reasonable to me. -Chris. Martin Buchholz wrote: [Chris or Alan, please review and file a bug]

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Kevin L. Stern
ZE > 0) { >> >> instead of >> >> if (newCapacity < minCapacity) >> >> if (newCapacity > MAX_ARRAY_SIZE) { >> >> Thanks, >> Dmytro >> >> > Date: Mon, 8 Mar 2010 18:10:37 -0800 >> > Subject: Re: Bugs in java.u

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Ulf Zibis
In PriorityQueue: let's result newCapacity in 0x.FFFC =-4 then "if (newCapacity - MAX_ARRAY_SIZE > 0)" ---> false then Arrays.copyOf(queue, newCapacity) ---> ArrayIndexOutOfBoundsException Am I wrong ? 2.) Why don't you prefer a system-wide constant for MAX_ARRAY_SIZE ??? -Ulf Am 09.03

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Kevin L. Stern
here any reason to use comparison like this > > if (newCapacity - minCapacity < 0) > > if (newCapacity - MAX_ARRAY_SIZE > 0) { > > instead of > > if (newCapacity < minCapacity) > > if (newCapacity > MAX_ARRAY_SIZE) { > > Thanks, > Dmytro > > >

RE: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-09 Thread Dmytro Sheyko
bject: Re: Bugs in java.util.ArrayList, java.util.Hashtable and > java.io.ByteArrayOutputStream > From: marti...@google.com > To: kevin.l.st...@gmail.com; christopher.hega...@sun.com; alan.bate...@sun.com > CC: core-libs-dev@openjdk.java.net > > [Chris or Alan, please review and file

Re: Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-08 Thread Martin Buchholz
2010/3/5 : > Hello, > > I'm using my own Collections if it's possible so I can add some thoughts: > > 1. I would decrease default array size to 4/6/8, for me it was few Mb more > of free memory ( i suggest testing on application that use at least 300Mb) > > I would test: > > initial size: 4 > long

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-08 Thread Martin Buchholz
On Fri, Mar 5, 2010 at 02:48, Kevin L. Stern wrote: > Hi Martin, > > Thank you for your reply.  If I may, PriorityQueue appears to employ the > simple strategy that I suggested above in its grow method: > >     int newCapacity = ((oldCapacity < 64)? >    ((oldCapacity +

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-08 Thread Martin Buchholz
[Chris or Alan, please review and file a bug] OK, guys, Here's a patch: http://cr.openjdk.java.net/~martin/webrevs/openjdk7/ArrayResize/ Martin On Fri, Mar 5, 2010 at 02:48, Kevin L. Stern wrote: > Hi Martin, > > Thank you for your reply.  If I may, PriorityQueue appears to employ the > simpl

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-05 Thread Kevin L. Stern
Hi Martin, Thank you for your reply. If I may, PriorityQueue appears to employ the simple strategy that I suggested above in its grow method: int newCapacity = ((oldCapacity < 64)? ((oldCapacity + 1) * 2): ((oldCapacity / 2) * 3));

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-05 Thread Kevin L. Stern
FYI, HashMap independently defines a MAXIMUM_CAPACITY variable; it might be a good idea to retrofit this and other such local definitions with any system wide variables that are defined. /** * The maximum capacity, used if a higher value is implicitly specified * by either of the con

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-05 Thread Ulf Zibis
Am 05.03.2010 10:04, schrieb Martin Buchholz: Hi Kevin, As you've noticed, creating objects within a factor of two of their natural limits is a good way to expose lurking bugs. I'm the one responsible for the algorithm in ArrayList. I'm a bit embarrassed, looking at that code today. We could se

Re: Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-05 Thread develop4lasu
Hello, I'm using my own Collections if it's possible so I can add some thoughts: 1. I would decrease default array size to 4/6/8, for me it was few Mb more of free memory ( i suggest testing on application that use at least 300Mb) I would test: initial size: 4 long newCapacity = ((long)oldC

Re: Bugs in java.util.ArrayList, java.util.Hashtable and java.io.ByteArrayOutputStream

2010-03-05 Thread Martin Buchholz
Hi Kevin, As you've noticed, creating objects within a factor of two of their natural limits is a good way to expose lurking bugs. I'm the one responsible for the algorithm in ArrayList. I'm a bit embarrassed, looking at that code today. We could set the array size to Integer.MAX_VALUE, but then