So, FWIU, in vector selection, the output array would always have a non-null validity buffer, isn't it?
On Tue, Jun 29, 2021 at 11:54 AM Antoine Pitrou <anto...@python.org> wrote: > > > Le 29/06/2021 à 17:49, Niranda Perera a écrit : > > Hi all, > > > > I'm looking into this now, and I feel like there's a potential memory > > corruption at the very end of the out_data_ array. > > > > algo: > > bool advance = BitUtil::GetBit(filter_data_, filter_offset_ + > in_position); > > BitUtil::SetBitTo(out_is_valid_, out_offset_ + out_position_, advance); > > out_data_[out_position_] = values_data_[in_position]; > > out_position_ += advance; // may need static_cast<int> here > > ++in_position; > > say, > > in.data = [1, 2, 3, 4] // all valid > > filt.data = [1, 1, 0, 0] // all valid > > At in_position = 1, out_position becomes 2. And till the end of the loop > > (i.e. in_position <= 3), out_position[2] will be updated (but with > > out_is_valid_[2] set to 0). I belive at the end of the loop, following > > would be the output. > > out.data = [1, 2, 4] > > out.valid = [1, 1, 0] > > > > Wouldn't this be a problem? > > Indeed, you have to allocate N + 1 entries for the result array instead > of N. Probably not a big deal, IMHO. > > Regards > > Antoine. > -- Niranda Perera https://niranda.dev/ @n1r44 <https://twitter.com/N1R44>
