--- Begin Message ---
Hi,
Our customer's system was also crashed in the same case.
I check core image, and I suspect overflow of "pDst" in
"Java_sun_java2d_loops_MaskFill_MaskFill()"
In order to fix this problem, I made a patch for typecasting "ptrdiff_t" in
PtrCoord macro.
Please merge this patch if you don't fix this problem yet.
("test.c" is not a patch. It is minimal sample of this overflow problem.)
from hs_err log:
----------------------------
#
# An unexpected error has been detected by Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002aabcb644177, pid=27759, tid=1142659392
#
# Java VM: OpenJDK 64-Bit Server VM (1.6.0-b09 mixed mode linux-amd64)
# Problematic frame:
# C [libawt.so+0x63177] IntArgbSrcOverMaskFill+0x127
#
# If you would like to submit a bug report, please visit:
# http://icedtea.classpath.org/bugzilla
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
:
:
OS:Red Hat Enterprise Linux Server release 5.4 (Tikanga)
uname:Linux 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64
libc:glibc 2.5 NPTL 2.5
rlimit: STACK 10240k, CORE infinity, NPROC infinity, NOFILE 65536, AS infinity
load average:1.04 0.56 0.41
CPU:total 4 (1 cores per cpu, 1 threads per core) family 6 model 10 stepping 5,
cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3
Memory: 4k page, physical 5830108k(39684k free), swap 4192956k(4065544k free)
vm_info: OpenJDK 64-Bit Server VM (1.6.0-b09) for linux-amd64 JRE (1.6.0-b09),
built on Aug 5 2009 11:16:51 by "mockbuild" with gcc 4.1.2 20080704 (Red Hat
4.1.2-44)
time: Thu Jun 2 21:04:51 2011
elapsed time: 517630 seconds
----------------------------
from core image:
----------------------------
[root@RHEL5-4 T2011060009]# gdb java core.27759
:
:
(gdb) f 7
#7 0x00002aabcb61cd3d in Java_sun_java2d_loops_MaskFill_MaskFill
(env=0x2aabcc36f598,
self=<value optimized out>, sg2d=0x441b70c8, sData=<value optimized out>,
comp=<value optimized out>, x=50, y=26188, w=32, h=32, maskArray=0x441b7120,
maskoff=0, maskscan=32) at
../../../src/share/native/sun/java2d/loops/MaskFill.c:85
85 ../../../src/share/native/sun/java2d/loops/MaskFill.c: No such file or
directory.
in ../../../src/share/native/sun/java2d/loops/MaskFill.c
(gdb) p pDst
$1 = (void *) 0x2aaa8aaea6e0
(gdb) p rasInfo
$2 = {bounds = {x1 = 50, y1 = 26188, x2 = 82, y2 = 26220}, rasBase =
0x2aab0a4fc718,
pixelBitOffset = 0, pixelStride = 4, scanStride = 82240, lutSize = 0, lutBase
= 0x0,
invColorTable = 0x0, redErrTable = 0x0, grnErrTable = 0x0, bluErrTable = 0x0,
invGrayTable = 0x2aabb15d4d68, priv = {align = 0x3,
data = "\003\000\000\000\000\000\000\000\030ヌO\nォ*", '\0' <repeats 18
times>, "@\000\000\000\000\000\000\000X\213P爼*\000\000\001", '\0' <repeats 14
times>}}
----------------------------
"pDst" is calculated in "MaskFill.c" as following:
----------------------------
void *pDst = PtrCoord(rasInfo.rasBase,
rasInfo.bounds.x1, rasInfo.pixelStride,
rasInfo.bounds.y1, rasInfo.scanStride);
----------------------------
"PtrCoord" is defined in "GraphicsPrimitiveMgr.h":
----------------------------
#define PtrAddBytes(p, b) ((void *) (((intptr_t) (p)) + (b)))
#define PtrCoord(p, x, xinc, y, yinc) PtrAddBytes(p, (y)*(yinc) + (x)*(xinc))
----------------------------
In this case, "b" in PtrAddBytes macro is
(rasInfo.bounds.y1 * rasInfo.scanStride) + (rasInfo.bounds.x1 *
rasInfo.pixelStride)
= (26188 * 82240) + (50 * 4)
= 2153701320 ( > INT_MAX ( 2147483647 (0x7fffffff) ))
"b" sets to be -2141265976. So, "pDst" set to be as following:
pDst = rasInfo.bounds.rasBase - 2141265976
= 0x2aaa8aaea6e0
pDst should set to be 0x2aab8aaea6e0,
however, it set to be 0x2aaa8aaea6e0.
Best regards,
Yasumasa
#include <stdio.h>
#include <stdint.h>
#include <stddef.h>
#include <limits.h>
main(){
void *rasBase = (void *)0x2aab0a4fc718UL;
//void *rasBase = 0x2aab0a4fc718;
int x1 = 50;
int pixelStride = 4;
int y1 = 26188;
int scanStride = 82240;
int ofs = (y1*scanStride) + (x1*pixelStride);
ptrdiff_t new_ofs = ((ptrdiff_t)y1*(ptrdiff_t)scanStride) +
((ptrdiff_t)x1*(ptrdiff_t)pixelStride);
//ptrdiff_t new_ofs = (ptrdiff_t)(y1*scanStride) +
(ptrdiff_t)(x1*pixelStride);
void *ptr = (void *)(((intptr_t)rasBase) + (y1*scanStride) +
(x1*pixelStride));
void *ptr_2 = (void *)(((intptr_t)rasBase) + ofs);
void *ptr_3 = (void *)(((intptr_t)rasBase) + new_ofs);
printf("LONG_MAX = 0x%lx, sizeof(intptr_t) = %lu\n", LONG_MAX,
sizeof(intptr_t));
printf("INT_MAX = %d (0x%x)\n", INT_MAX, INT_MAX);
printf("ofs = %d\n", ofs);
printf("new_ofs = %lu\n", new_ofs);
printf("rasBase = %p, ofs = 0x%lx\n", rasBase, ofs);
printf("ptr = %p\n", ptr);
printf("ptr_2 = %p\n", ptr_2);
printf("ptr_3 = %p\n", ptr_3);
}
diff -r 1e04b38b3824 src/share/native/sun/java2d/loops/GraphicsPrimitiveMgr.h
--- a/src/share/native/sun/java2d/loops/GraphicsPrimitiveMgr.h Sat Jun 04
17:33:13 2011 -0700
+++ b/src/share/native/sun/java2d/loops/GraphicsPrimitiveMgr.h Mon Jun 06
19:45:52 2011 +0900
@@ -30,6 +30,8 @@
extern "C" {
#endif
+#include <stddef.h>
+
#include "java_awt_AlphaComposite.h"
#include "SurfaceData.h"
@@ -484,7 +486,8 @@
#define ArraySize(A) (sizeof(A) / sizeof(A[0]))
#define PtrAddBytes(p, b) ((void *) (((intptr_t) (p)) + (b)))
-#define PtrCoord(p, x, xinc, y, yinc) PtrAddBytes(p, (y)*(yinc) + (x)*(xinc))
+#define PtrCoord(p, x, xinc, y, yinc) \
+ PtrAddBytes(p, (ptrdiff_t)(y)*(ptrdiff_t)(yinc) +
(ptrdiff_t)(x)*(ptrdiff_t)(xinc))
/*
* The function to call with an array of NativePrimitive structures
--- End Message ---
