> Any suggestions, guys? Can anyone please point us to the right place in the
> source code so we can have a closer look?
Physical layout of unpacked record in memory is simple:
- bitmak of NULL values, one bit per field
This bitmask contains one 32-bit intereg per every 32 fields in relation.
- data, field by field, aligned by value evaluated from data type and declared
data length.
This alignment rules could be found at MET_align() function. It returns 1
for CHAR
type, 2 for VARCHAR type and MIN(data_length, 8) for all other data types.
Firebird used almost the same format for external table's record as for its
internal
records with only distinctions: external record have no NULL's bitmask. But
offsets of fields
in record are evaluated by the common rules. Its just adjusted when external
table is
accessed.
For example
CREATE TABLE t1 (
ID BIGINT,
NAME VARCHAR(32),
FOO INT
)
The record layout is
offset length field_name
0 4 <nulls mask>
8 8 ID
16 34 NAME
52 4 FOO
Record length is 56 bytes.
Notes:
- offset of ID is 8 and not 4 as 8-bytes field must be aligned at 8-byte
position,
- length of VARCHAR field included additional 2 bytes for real data length
(32+2 = 34)
- offset of FOO is 52 and not 50 (34+16) as it must be aligned at 4-bytes
position
Above is so called format of record and it is used everywhere across the
engine.
Engine calculates and store this format in RDB$FORMATS and used it when table
is accesses.
But EXTERNAL tables have no NULL's bitmask therefore when data is mapped
from\to record buffer engine adjusted fields offsets in format by substracting
offset
of a 1st field in format (not size of NULL's mask!).
So, in fact offsets used to access external table with structure above
(i.e. on-disk
record representation) is:
offset length field_name
0 8 ID
8 34 NAME
46 4 FOO
and it could look strange from alignment point of view.
> Also, it appears that external table doesn't like a row being over 32k
> (including padding) in the external file. When we have rows over 32k,
> select from the external table just returns nothing. Is this a known
> limitation? I couldn't find this mentioned in any documentation.
>
> We are running Firebird 2.1.2 64bit Classic Server on Centos 6.
I'm not sure FB 2.1.x could correctly work with 32K+ records. But i can be
wrong, i don't remember exactly. Could you check it with FB 2.5 ?
Hope it helps,
Vlad
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
