Given the representation of 0.02
and 0.00
in the GSDecimal
format as you described, let's analyze how the GSDecimalCompare
function behaves with these inputs:
0.02 in
GSDecimal
:- Length: 1
- Exponent: -2
- cMantissa: [2]
- Length: 1
0.00 in
GSDecimal
:- Length: 0
- Exponent: 0
- cMantissa: []
- Length: 0
Analysis:
Exponent and Length Calculation: For
0.02
,s1
is 1 - 2 = -1. For0.00
,s2
is 0 + 0 = 0. TheGSDecimalCompare
function will then compares1
ands2
.Same Sign, Check Size: Since
s1
(-1) is less thans2
(0), the function will returnNSOrderedAscending
if0.02
is not negative, which seems to be the case here. This means the function considers0.02
to be smaller than0.00
, which is incorrect.Comparing Digits (cMantissa): Since
0.00
has a length of 0, the digit comparison loop inGSDecimalCompare
is skipped. This is a crucial point because it does not compare the actual digits if one of the numbers has a length of 0.
Conclusion:
The issue seems to be arising from how the GSDecimalCompare
function handles the case where one of the numbers (0.00
) has a length
of 0. It leads to skipping the digit comparison and making a decision based solely on the exponent and length calculations, which in this case, misleadingly indicates that 0.02
is smaller than 0.00
.
Suggested Fix:
A potential fix in the GSDecimalCompare
function could involve adding a check for cases where one operand has a length of 0. Specifically, if one operand has a length of 0 (indicating it's 0.00
), and the other operand has a non-zero length, the function should return NSOrderedDescending
if the non-zero length operand is positive, and NSOrderedAscending
if it's negative. This logic should be placed before the digit comparison loop.
NSDecimal_patch.m
Description: Binary data