As you can see from existing tests, Beam doesn't materialize the output
array directly. Instead you must use the PAssert API. I agree with Tyson's
suggestion to use `satisfies`, which lets you do arbitrary assertions on
the output data.
On Tue, Mar 2, 2021 at 3:57 AM Sonam Ramchand <
sonam.ramch...@venturedive.com> wrote:
> Is there any way I can access the output array resulting from the sql
> query? Then maybe I can sort and compare both *output array* and *expected
> output array *for the test to pass.
>
> On Tue, Mar 2, 2021 at 12:24 AM Kenneth Knowles <k...@apache.org> wrote:
>
>> Yea, the reason is that SQL relations are not ordered. So any ordering of
>> [1, 2, 3, 4] and [5, 6] and [7, 8, 9] is possible and correct.
>>
>> Kenn
>>
>> On Mon, Mar 1, 2021 at 11:01 AM Tyson Hamilton <tyso...@google.com>
>> wrote:
>>
>>> I didn't find anything like that after a brief look. What you could do
>>> instead is something like:
>>>
>>> PAssert.thatSingleton(stream).satisfies( row -> assertThat("array_field
>>> containsInAnyOrder", row.getArray("array_field"),
>>> containsInAnyOrder(Arrays.asList(...)));
>>>
>>> using junit/hamcrest matchers. I didn't verify this works myself but it
>>> should give you an idea for some next steps.
>>>
>>>
>>> On Mon, Mar 1, 2021 at 12:37 AM Sonam Ramchand <
>>> sonam.ramch...@venturedive.com> wrote:
>>>
>>>> Hi Devs,
>>>> I have implemented the ARRAY_CONCAT_AGG function for zetasql dialect. I
>>>> am trying to validate the test as:
>>>>
>>>> @Test
>>>> public void testArrayConcatAggZetasql() {
>>>> String sql =
>>>> "SELECT ARRAY_CONCAT_AGG(x) AS array_concat_agg FROM (SELECT [1, 2,
>>>> 3, 4] AS x UNION ALL SELECT [5, 6] UNION ALL SELECT [7, 8, 9])";
>>>>
>>>> ZetaSQLQueryPlanner zetaSQLQueryPlanner = new
>>>> ZetaSQLQueryPlanner(config);
>>>> BeamRelNode beamRelNode = zetaSQLQueryPlanner.convertToBeamRel(sql);
>>>> PCollection<Row> stream = BeamSqlRelUtils.toPCollection(pipeline,
>>>> beamRelNode);
>>>>
>>>> Schema schema = Schema.builder().addArrayField("array_field",
>>>> FieldType.INT64).build();
>>>> PAssert.that(stream)
>>>> .containsInAnyOrder(
>>>> Row.withSchema(schema).addArray(1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L,
>>>> 9L).build());
>>>>
>>>> pipeline.run().waitUntilFinish(Duration.standardMinutes(PIPELINE_EXECUTION_WAITTIME_MINUTES));
>>>> }
>>>>
>>>> Expected Output is: 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L.
>>>> But I am getting randomly different outputs:
>>>> 1. 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L
>>>> 2. 5L, 6L, 7L, 8L, 9L, 1L, 2L, 3L, 4L
>>>> 3. 7L, 8L, 9L, 5L, 6L, 1L, 2L, 3L, 4L
>>>>
>>>> As per my understanding, it is because of containsInAnyOrder function.
>>>> Is there anything Like:
>>>>
>>>> PAssert.that(stream)
>>>> .containsAnyOfThem(
>>>> Row.withSchema(schema).addArray(1L, 2L, 3L, 4L, 5L, 6L, 7L,
>>>> 8L, 9L).build(),
>>>> Row.withSchema(schema).addArray(5L, 6L, 7L, 8L, 9L, 1L,
>>>> 2L, 3L, 4L).build(),
>>>> Row.withSchema(schema).addArray(7L, 8L, 9L, 5L, 6L, 1L,
>>>> 2L, 3L, 4L).build());
>>>>
>>>> I would really appreciate if anyone can help me in knowing how to
>>>> handle such scenario in Beam.
>>>>
>>>> Thanks!
>>>> --
>>>> Regards,
>>>> *Sonam*
>>>> Software Engineer
>>>> Mobile: +92 3088337296 <+92%20308%208337296>
>>>>
>>>> <http://venturedive.com/>
>>>>
>>>
>
> --
>
> Regards,
> *Sonam*
> Software Engineer
> Mobile: +92 3088337296 <+92%20308%208337296>
>
> <http://venturedive.com/>
>
