+1 for solution 2
Regards
Kumar Vishal
On Thu, 30 Jul 2020 at 3:19 PM, Indhumathi wrote:
> Hi community,
>
> Currently, i am working on supporting SI with complex array type.
> In order to support it, we must decide, how we can store Array type
> in SI, to get better performance.
>
> Solution
Hi All,
+1 for solution 2. But don't store rowid as it makes the storage very big
and it gives a very slow performance. Let's go with the current model of SI
which stores till blocklet level. Don't make things complicated by storing
rowid.
Solution 1 makes the scan slower as it needs to construct
Hi David & Indhumathi,
Storing Array of String as just String column in SI by flattening [with row
level position reference] can result in slow performance in case of
* Multiple array_contains() or multiple array[0] = 'x'
* The join solution mentioned can result in multiple scan (once for every
com
+1 for solution2
Can we support more than one array_contains by using SI join (like SI on
primitive data type)?
-
Best Regards
David Cai
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Hi community,
Currently, i am working on supporting SI with complex array type.
In order to support it, we must decide, how we can store Array type
in SI, to get better performance.
Solution 1:
Store Array as complex(ARRAY) type in secondary index table.
Cons:
Pruning arrays of huge data on SI
