| Issue |
243202
|
| Summary |
[flang-rt] Formatted input counts bytes, not characters: KIND=4 internal-file padding and SIZE= on UTF-8 files
|
| Labels |
flang:runtime
|
| Assignees |
|
| Reporter |
eugeneepshteyn
|
Formatted input keeps two counts in bytes where they should be in characters, so input with multi-byte characters goes wrong in two ways:
1. **CHARACTER(KIND=4) internal file, field longer than the record:** the field gets four padding blanks for each missing character. `(bz,z3)` on `4_'1F'` gives `Z'1F0000'`, while the same READ from a default-kind internal file gives `Z'1F0'`; `(bz,z6)` fails with an overflow (IOSTAT 1034).
2. **UTF-8 external file:** `SIZE=` counts bytes. Reading one two-byte character with `A1` sets `SIZE=2`, but F2023 12.6.2.16 p1 defines it as the count of characters transferred.
```fortran
program multibyte
implicit none
character(kind=4,len=2) :: s4
character(kind=4,len=1) :: w
character(kind=4,len=2) :: w2
character(2) :: s1
integer :: i, ios, u, n
! CHARACTER(KIND=4) internal file, short field padded with blanks under BZ
s4 = 4_'1F'
read (s4, '(bz,z3)', iostat=ios) i
print '(a,z0,a,i0)', 'kind4 (bz,z3): value=', i, ' iostat=', ios
read (s4, '(bz,z6)', iostat=ios) i
print '(a,i0)', 'kind4 (bz,z6): iostat=', ios
s1 = '1F'
read (s1, '(bz,z3)', iostat=ios) i
print '(a,z0,a,i0)', 'kind1 (bz,z3): value=', i, ' iostat=', ios
! UTF-8 external file: SIZE= for one two-byte character
open (newunit=u, file='multibyte.dat', status='replace', encoding='UTF-8')
write (u, '(a)') char(int(z'e9'), kind=4) // 4_'X'
rewind (u)
read (u, '(a1)', advance='no', size=n) w
print '(a,i0,a,i0)', 'utf8 (a1): SIZE=', n, ' char=', ichar(w)
rewind (u)
read (u, '(a2)', advance='no', size=n) w2
print '(a,i0,a,i0,1x,i0)', 'utf8 (a2): SIZE=', n, ' chars=', ichar(w2(1:1)), ichar(w2(2:2))
close (u, status='delete')
end program
```
flang (main at 20b943451383):
```
kind4 (bz,z3): value=1F0000 iostat=0
kind4 (bz,z6): iostat=1034
kind1 (bz,z3): value=1F0 iostat=0
utf8 (a1): SIZE=2 char=233
utf8 (a2): SIZE=3 chars=233 88
```
Expected: the KIND=4 results match default-kind input (`value=1F0` for `z3`), and `SIZE=` is 1 and 2. The characters themselves are read correctly; only the counts are wrong. (gfortran also reports bytes for `SIZE=` here.)
Causes:
- The field width that `CueUpInput` returns is in bytes by design: it is multiplied by the internal unit's character kind ([io-stmt.h#L238-L256](https://github.com/llvm/llvm-project/blob/24580866f5d731e4962829e7b0f298092ceacb0f/flang-rt/include/flang-rt/runtime/io-stmt.h#L238-L256)). The padding branch of `NextInField` decrements it by 1 for each blank it supplies, not by the character size ([io-stmt.cpp#L730-L733](https://github.com/llvm/llvm-project/blob/24580866f5d731e4962829e7b0f298092ceacb0f/flang-rt/lib/runtime/io-stmt.cpp#L730-L733)).
- `SIZE=` is incremented by byte counts: `GotChar(chunkBytes)` in `EditCharacterInput` ([edit-input.cpp#L1187-L1191](https://github.com/llvm/llvm-project/blob/24580866f5d731e4962829e7b0f298092ceacb0f/flang-rt/lib/runtime/edit-input.cpp#L1187-L1191)) and `GotChar(byteCount)` in `NextInField` ([io-stmt.cpp#L710-L712](https://github.com/llvm/llvm-project/blob/24580866f5d731e4962829e7b0f298092ceacb0f/flang-rt/lib/runtime/io-stmt.cpp#L710-L712), [#L724-L726](https://github.com/llvm/llvm-project/blob/24580866f5d731e4962829e7b0f298092ceacb0f/flang-rt/lib/runtime/io-stmt.cpp#L724-L726)).
Found while reviewing PR 238057, which does not change this.
(This issue was filed with the help of AI)
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs