Yeah, that sounds right. But the only thing that confuses me is:
PAssert.that(stream).satisfies(row ->
assertThat("array_agg_concat_field", actual ,
containsInAnyOrder(Arrays.asList(1L,2L,3L,4L,5L,6L))));
How come I can access *actual* here when output is not materialized.
On Tue, Mar 2, 2021 at 10:49 PM Kyle Weaver <kcwea...@google.com> wrote:
> As you can see from existing tests, Beam doesn't materialize the output
> array directly. Instead you must use the PAssert API. I agree with Tyson's
> suggestion to use `satisfies`, which lets you do arbitrary assertions on
> the output data.
>
> On Tue, Mar 2, 2021 at 3:57 AM Sonam Ramchand <
> sonam.ramch...@venturedive.com> wrote:
>
>> Is there any way I can access the output array resulting from the sql
>> query? Then maybe I can sort and compare both *output array* and *expected
>> output array *for the test to pass.
>>
>> On Tue, Mar 2, 2021 at 12:24 AM Kenneth Knowles <k...@apache.org> wrote:
>>
>>> Yea, the reason is that SQL relations are not ordered. So any ordering
>>> of [1, 2, 3, 4] and [5, 6] and [7, 8, 9] is possible and correct.
>>>
>>> Kenn
>>>
>>> On Mon, Mar 1, 2021 at 11:01 AM Tyson Hamilton <tyso...@google.com>
>>> wrote:
>>>
>>>> I didn't find anything like that after a brief look. What you could do
>>>> instead is something like:
>>>>
>>>> PAssert.thatSingleton(stream).satisfies( row -> assertThat("array_field
>>>> containsInAnyOrder", row.getArray("array_field"),
>>>> containsInAnyOrder(Arrays.asList(...)));
>>>>
>>>> using junit/hamcrest matchers. I didn't verify this works myself but it
>>>> should give you an idea for some next steps.
>>>>
>>>>
>>>> On Mon, Mar 1, 2021 at 12:37 AM Sonam Ramchand <
>>>> sonam.ramch...@venturedive.com> wrote:
>>>>
>>>>> Hi Devs,
>>>>> I have implemented the ARRAY_CONCAT_AGG function for zetasql dialect.
>>>>> I am trying to validate the test as:
>>>>>
>>>>> @Test
>>>>> public void testArrayConcatAggZetasql() {
>>>>> String sql =
>>>>> "SELECT ARRAY_CONCAT_AGG(x) AS array_concat_agg FROM (SELECT [1, 2,
>>>>> 3, 4] AS x UNION ALL SELECT [5, 6] UNION ALL SELECT [7, 8, 9])";
>>>>>
>>>>> ZetaSQLQueryPlanner zetaSQLQueryPlanner = new
>>>>> ZetaSQLQueryPlanner(config);
>>>>> BeamRelNode beamRelNode = zetaSQLQueryPlanner.convertToBeamRel(sql);
>>>>> PCollection<Row> stream = BeamSqlRelUtils.toPCollection(pipeline,
>>>>> beamRelNode);
>>>>>
>>>>> Schema schema = Schema.builder().addArrayField("array_field",
>>>>> FieldType.INT64).build();
>>>>> PAssert.that(stream)
>>>>> .containsInAnyOrder(
>>>>> Row.withSchema(schema).addArray(1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L,
>>>>> 9L).build());
>>>>>
>>>>> pipeline.run().waitUntilFinish(Duration.standardMinutes(PIPELINE_EXECUTION_WAITTIME_MINUTES));
>>>>> }
>>>>>
>>>>> Expected Output is: 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L.
>>>>> But I am getting randomly different outputs:
>>>>> 1. 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L
>>>>> 2. 5L, 6L, 7L, 8L, 9L, 1L, 2L, 3L, 4L
>>>>> 3. 7L, 8L, 9L, 5L, 6L, 1L, 2L, 3L, 4L
>>>>>
>>>>> As per my understanding, it is because of containsInAnyOrder function.
>>>>> Is there anything Like:
>>>>>
>>>>> PAssert.that(stream)
>>>>> .containsAnyOfThem(
>>>>> Row.withSchema(schema).addArray(1L, 2L, 3L, 4L, 5L, 6L, 7L,
>>>>> 8L, 9L).build(),
>>>>> Row.withSchema(schema).addArray(5L, 6L, 7L, 8L, 9L, 1L,
>>>>> 2L, 3L, 4L).build(),
>>>>> Row.withSchema(schema).addArray(7L, 8L, 9L, 5L, 6L, 1L,
>>>>> 2L, 3L, 4L).build());
>>>>>
>>>>> I would really appreciate if anyone can help me in knowing how to
>>>>> handle such scenario in Beam.
>>>>>
>>>>> Thanks!
>>>>> --
>>>>> Regards,
>>>>> *Sonam*
>>>>> Software Engineer
>>>>> Mobile: +92 3088337296 <+92%20308%208337296>
>>>>>
>>>>> <http://venturedive.com/>
>>>>>
>>>>
>>
>> --
>>
>> Regards,
>> *Sonam*
>> Software Engineer
>> Mobile: +92 3088337296 <+92%20308%208337296>
>>
>> <http://venturedive.com/>
>>
>
--
Regards,
*Sonam*
Software Engineer
Mobile: +92 3088337296
<http://venturedive.com/>
