[
https://issues.apache.org/jira/browse/ARROW-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091600#comment-17091600
]
Mahmut Bulut commented on ARROW-5949:
-------------------------------------
Sorry, yes, that's exactly like that, it is ok and valid. Gave that example to
show that we can leave the indices as how -1 is masked on (unfortunately it
won't work with unsigned values, I think that's why the bit masking approach is
better). Thanks for the links they were fruitful.
I think I am more inclined to not build the primitive array, neither user
should collect the result from the iterator nor one by one look for the
Some(_), that said I tend to have slice given back from the array, which is
most probably enable users who are using SIMD later. Thou, it is also nice to
have a PrimitiveArray API given to users. Current stable SIMD instructions also
packed_simd are fill free so I need to use continuous scalars for dict encoded
operations, which are crucial for my use case (repacking the arrow array is an
overhead for me). So I have started to make a vectorized slice implementation
over current dictionary array, is it ok to include slice kind of approach to
Arrow? with chunked offsets, we can even use Rust arrays too. Wdyt?
> [Rust] Implement DictionaryArray
> --------------------------------
>
> Key: ARROW-5949
> URL: https://issues.apache.org/jira/browse/ARROW-5949
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Rust
> Reporter: David Atienza
> Assignee: David Atienza
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.17.0
>
> Time Spent: 18h
> Remaining Estimate: 0h
>
> I am pretty new to the codebase, but I have seen that DictionaryArray is not
> implemented in the Rust implementation.
> I went through the list of issues and I could not see any work on this. Is
> there any blocker?
>
> The specification is a bit
> [short|https://arrow.apache.org/docs/format/Layout.html#dictionary-encoding]
> or even
> [non-existant|https://arrow.apache.org/docs/format/Metadata.html#dictionary-encoding],
> so I am not sure how to implement it myself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
