BIN_SHL and BIN_SHR: Incorrect results when shift >= 64
-------------------------------------------------------
Key: CORE-5230
URL: http://tracker.firebirdsql.org/browse/CORE-5230
Project: Firebird Core
Issue Type: Bug
Components: Engine
Affects Versions: 3.0.0, 2.5.5
Environment: Win7, CentOS 7
Reporter: Paul Vinkenoog
Priority: Minor
If the second argument to BIT_SHL/BIT_SHR is 64 or greater, it is taken modulo
64 before the shift operation is performed. This leads to unexpected and
incorrect results.
Example with positive numbers:
select bin_shr(120, 1) from rdb$database -> 60
select bin_shr(120, 6) from rdb$database -> 1
select bin_shr(120, 7) from rdb$database -> 0
..
select bin_shr(120, 63) from rdb$database -> 0
select bin_shr(120, 64) from rdb$database -> 120 (wrong)
select bin_shr(120, 65) from rdb$database -> 60 (wrong)
Example with negative numbers:
select bin_shr(-120, 1) from rdb$database -> -60
select bin_shr(-120, 2) from rdb$database -> -30
select bin_shr(-120, 7) from rdb$database -> -1
select bin_shr(-120, 9) from rdb$database -> -1
..
select bin_shr(-120, 63) from rdb$database -> -1
select bin_shr(-120, 64) from rdb$database -> -120 (wrong)
select bin_shr(-120, 65) from rdb$database -> -60 (wrong)
Instead of MOD'ing the shift argument by 64, it should be maximized at 64
(because all the original bits have been shifted out by then) - at least as
long as the result type isn't bigger than int64.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
