Hi Jeff,
It's a simple pipeline that takes PCollection of TableRow which is selected
from Google Analytics export to BigQuery. So each TableRow follows this
scheme https://support.google.com/analytics/answer/3437719?hl=en
I have part of the code doing casting to TableRow like this:
Boolean isMobile = (Boolean) (((TableRow) row.get("device")).get("isMobile"));
or
List<Hit> hits = ((List<TableRow>)
row.get("hits")).stream().map(Hit::new).collect(Collectors.toList());
I don't have issues running this pipeline in production. I have this issue,
only when I tried to write end to end test.
Do you know if there are existing coders for TableRow that I can use? I've
tried TableRowJsonCoder, but seems like it converts all object inside
TableRow to LinkedHashMaps
On Wed, 8 Jul 2020 at 17:30, Jeff Klukas <jklu...@mozilla.com> wrote:
> Kirill - Can you tell us more about what Job.runJob is doing? I would not
> expect the Beam SDK itself to do any casting to TableRow, so is there a
> line in your code where you're explicitly casting to TableRow? There may be
> a point where you need to explicitly set the coder on a PCollection to
> deserialize back to TableRow objects.
>
> On Wed, Jul 8, 2020 at 10:11 AM Kirill Zhdanovich <kzhdanov...@gmail.com>
> wrote:
>
>> Here is a code example:
>>
>> List<TableRow> ss = Arrays.asList(session1, session2);
>> PCollection<TableRow> sessions = p.apply(Create.of(ss));
>> PCollection<MetricsWithDimension> res = Job.runJob(sessions, "20200614",
>> false, new ProductCatalog());
>> p.run();
>>
>>
>> On Wed, 8 Jul 2020 at 17:07, Kirill Zhdanovich <kzhdanov...@gmail.com>
>> wrote:
>>
>>> Hi,
>>> I want to test pipeline and the input for it is PCollection of
>>> TableRows. I've created a test, and when I run it I get an error:
>>>
>>> java.lang.ClassCastException: class java.util.LinkedHashMap cannot be
>>> cast to class com.google.api.services.bigquery.model.TableRow
>>>
>>> Is it a known issue? Thank you in advance
>>>
>>> --
>>> Best Regards,
>>> Kirill
>>>
>>
>>
>> --
>> Best Regards,
>> Kirill
>>
>
--
Best Regards,
Kirill
