|
This deals with two possible "critical" issues in RubyFixnum.java,
and also suggests a possible improvement to related code which is working.
(As with my submission in JRUBY-6612, my apologies if I'm raising something
which you are already aware of, or if I'm looking at the wrong code,
or making another error which invalidates this bug report.
And I'd be happy to (try to!) answer any questions.)
I think there are some (critical?) bugs in
- private IRubyObject idivLong: RubyFixnum.java line 566;
- private IRubyObject divmodFixnum: RubyFixnum.java line 635;
The problems are very similar to those I raised in JRUBY-6612 for:
- public IRubyObject op_mul: RubyFixnum.java line 491.
Using jirb from JRUBY 1.6.6
v = 256
v *= v #=> 65536
v *= v * v * (v / -2) #=> -9223372036854775808 == Long.MIN_VALUE
v / -1 #=> -9223372036854775808
v.div(-1) #=> -9223372036854775808
v.divmod(-1) #=> [-9223372036854775808, 0]
- This next line is also a problem in JRuby 1.6.6, but following
JRUBY-6612 has, I think, been fixed in JRuby 1.7.0.preview1.
-1 * v #=> -9223372036854775808 # wrong
v * -1 #=> 9223372036854775808 # correct
Using jirb from JRUBY 1.7.0.preview1 gives the same results
except that "-1 * v" now gives the correct 9223372036854775808.
The problem seems to be that some long overflows aren't being detected.
I suggest patches in the attached file which:
(a) make what I think are appropriate checks for overflows;
(b) change the code to avoid using both "/" and "%", to speed things up.
I've put this as critical because when I raised a similar problem
for "public IRubyObject op_mul" (see JRUBY-6612) I called it "major"
and it was upgraded to "critical".
|
