Hello,
I'm trying to filter the rows of a table by whether or not a value exists
in an array column of a table.
Simple example:
table.where("apple".in($"fruits"))
In this example, each row has a "fruits" Array column that could
have 1 or many fruit strings which may or may not be "apple".
Howeve
I believe this is happening because the type system does not recognize
that list of Strings as anything special but treats it as a black-box type.
@Timo: Would this work with the new type system?
Best,
Aljoscha
On 02.11.20 06:47, Rex Fenley wrote:
Hello,
I'm trying to filter the rows of a ta
For clarification, I'm using Pojo and operating on a column of this type
public java.util.List fruits
adding the following annotation does not help
@DataTypeHint("ARRAY")
On Mon, Nov 2, 2020 at 7:02 AM Aljoscha Krettek wrote:
> I believe this is happening because the type system does not recogn
Using a custom serializer to make sure I'm using a List does not
help.
[info] org.apache.flink.table.api.ValidationException: IN operator on
incompatible types: String and List.
On Tue, Nov 3, 2020 at 12:44 PM Rex Fenley wrote:
> For clarification, I'm using Pojo and operating on a column of
None of the following appear to work either. Flink 1.11.2, Scala 2.12.
table.filter("apple".in(List("apple")))
[info] org.apache.flink.table.api.ValidationException: IN operator on
incompatible types: String and ObjectArrayTypeInfo.
table.filter("apple".in(java.util.Arrays.asList("apple")))
[in
Hi Rex,
as far as I know, the IN operator only works on tables or a list of
literals where the latter one is just a shortcut for multiple OR
operations. I would just go with a UDF for this case. In SQL you could
do an UNNEST to convert the array into a table and then use the IN
operator. But
Thanks Timo,
Checking if an element is in an Array does seem like a very useful function
to have. Is there any plan to add it?
Thanks
On Thu, Nov 5, 2020 at 7:26 AM Timo Walther wrote:
> Hi Rex,
>
> as far as I know, the IN operator only works on tables or a list of
> literals where the latter
Hi Rex,
after verifying with Timo I created a new issue to address your proposal of
introducing a new operator [1]. Feel free to work on that one if you like.
Best,
Matthias
[1] https://issues.apache.org/jira/browse/FLINK-20148
On Thu, Nov 5, 2020 at 6:35 PM Rex Fenley wrote:
> Thanks Timo,
>
