sure thing!
This is the error from method *testSQLInsertRowsToPubsubFlat*
Given message schema: 'Fields:
Field{name=name, description=, type=STRING, options={{}}}
Field{name=height, description=, type=INT32, options={{}}}
Field{name=knowsJavascript, description=, type=BOOLEAN, options={{}}}
Options:{{}}'
does not match schema inferred from protobuf class.
Protobuf class:
'org.apache.beam.sdk.extensions.protobuf.PayloadMessages$NameHeightKnowsJSMessage'
Inferred schema: 'Fields:
Field{name=name, description=, type=STRING NOT NULL,
options={{beam:option:proto:meta:number=Option{type=INT32 NOT NULL,
value=1}}}}
Field{name=height, description=, type=INT32 NOT NULL,
options={{beam:option:proto:meta:number=Option{type=INT32 NOT NULL,
value=2}}}}
Field{name=knowsJs, description=, type=BOOLEAN NOT NULL,
options={{beam:option:proto:meta:number=Option{type=INT32 NOT NULL,
value=3}}}}
Options:{{beam:option:proto:meta:type_name=Option{type=STRING NOT NULL,
value=NameHeightKnowsJSMessage}}}'
And this is the message from
*testSQLInsertRowsToPubsubWithTimestampAttributeFlat*.
Given message schema: 'Fields:
Field{name=name, description=, type=STRING, options={{}}}
Field{name=height, description=, type=INT32, options={{}}}
Field{name=knowsJavascript, description=, type=BOOLEAN, options={{}}}
Options:{{}}'
does not match schema inferred from protobuf class.
Protobuf class:
'org.apache.beam.sdk.extensions.protobuf.PayloadMessages$NameHeightKnowsJSMessage'
Inferred schema: 'Fields:
Field{name=name, description=, type=STRING NOT NULL,
options={{beam:option:proto:meta:number=Option{type=INT32 NOT NULL,
value=1}}}}
Field{name=height, description=, type=INT32 NOT NULL,
options={{beam:option:proto:meta:number=Option{type=INT32 NOT NULL,
value=2}}}}
Field{name=knowsJs, description=, type=BOOLEAN NOT NULL,
options={{beam:option:proto:meta:number=Option{type=INT32 NOT NULL,
value=3}}}}
Options:{{beam:option:proto:meta:type_name=Option{type=STRING NOT NULL,
value=NameHeightKnowsJSMessage}}}'
Maybe the proto variable *knowsJs* should be called *knowsJavascript*?
On Tue, Apr 20, 2021 at 12:26 PM Daniel Collins <dpcoll...@google.com>
wrote:
> The error includes "NameHeightMessage". Can you provide an example error
> from one of the other methods?
>
> On Tue, Apr 20, 2021 at 1:56 PM Fernando Morales Martinez <
> fernando.mora...@wizeline.com> wrote:
>
>> Thanks for the heads up, Daniel!
>>
>> I missed changing that one, but even after making the change, I'm getting
>> the same error.
>>
>> The other two methods, *testSQLInsertRowsToPubsubFlat* and
>> *testSQLInsertRowsToPubsubWithTimestampAttributeFlat*, were already
>> using NameHeightKnowsJSMessage class but are still throwing the same error.
>>
>> Any idea what else might be going on?
>>
>> On Tue, Apr 20, 2021 at 11:13 AM Daniel Collins <dpcoll...@google.com>
>> wrote:
>>
>>> Thanks for working on this! It looks to me like the schemas don't match:
>>> you appear to be using NameHeightMessage defined as:
>>>
>>> ```
>>> message NameHeightMessage {
>>> string name = 1;
>>> int32 height = 2;
>>> }
>>> ```
>>>
>>> And expecting it to work with a table schema that has a "BOOL
>>> knowsJavascript" field. Did you mean to use the "NameHeightKnowsJSMessage"
>>> class?
>>>
>>> -Daniel
>>>
>>> On Tue, Apr 20, 2021 at 1:02 PM Fernando Morales Martinez <
>>> fernando.mora...@wizeline.com> wrote:
>>>
>>>> Sorry for the spam, forgot to add the pertinent link to the code change.
>>>>
>>>>
>>>> https://github.com/fernando-wizeline/beam/commit/abc17db41b6aabf3f337c7742526e5ae9655f40b
>>>>
>>>> Thanks!
>>>>
>>>> On Tue, Apr 20, 2021 at 10:17 AM Fernando Morales Martinez <
>>>> fernando.mora...@wizeline.com> wrote:
>>>>
>>>>> Hi team,
>>>>>
>>>>> I'm working on adding tests to PubsubTableProviderIT class to test the
>>>>> proto support added to Pubsub.
>>>>>
>>>>> The issue below occurs when running
>>>>> *testSQLReadAndWriteWithSameFlatTableDefinition*,
>>>>> *testSQLInsertRowsToPubsubFlat* and
>>>>> *testSQLInsertRowsToPubsubWithTimestampAttributeFlat*
>>>>> of PubsubTableProviderIT.
>>>>>
>>>>> Right now I'm facing an issue when executing method
>>>>> inferAndVerifySchema of class ProtoPayloadSerializerProvider. The expected
>>>>> schema is set as
>>>>>
>>>>> 'Fields:
>>>>> Field{name=name, description=, type=STRING, options={{}}}
>>>>> Field{name=height, description=, type=INT32, options={{}}}
>>>>> Field{name=knowsJavascript, description=, type=BOOLEAN, options={{}}}
>>>>> Options:{{}}'
>>>>>
>>>>> and the schema obtained from the protoclass is:
>>>>>
>>>>> 'Fields:
>>>>> Field{name=name, description=, type=STRING NOT NULL,
>>>>> options={{beam:option:proto:meta:number=Option{type=INT32 NOT NULL,
>>>>> value=1}}}}
>>>>> Field{name=height, description=, type=INT32 NOT NULL,
>>>>> options={{beam:option:proto:meta:number=Option{type=INT32 NOT NULL,
>>>>> value=2}}}}
>>>>> Options:{{beam:option:proto:meta:type_name=Option{type=STRING NOT
>>>>> NULL, value=NameHeightMessage}}}'
>>>>> java.lang.IllegalArgumentException: Given message schema: 'Fields:
>>>>> Field{name=name, description=, type=STRING, options={{}}}
>>>>> Field{name=height, description=, type=INT32, options={{}}}
>>>>> Field{name=knowsJavascript, description=, type=BOOLEAN, options={{}}}
>>>>> Options:{{}}'
>>>>>
>>>>> I'm guessing, by the name of the tests, the idea is to compare the
>>>>> payload (protoclass in this case) against the flat schema, but then the
>>>>> validation in inferAndVerifySchema fails.
>>>>>
>>>>> Should I be looking for a workaround for that and if so, can you shed
>>>>> some light on how to proceed?
>>>>>
>>>>> Thanks for the help!
>>>>> - Fernando Morales
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *This email and its contents (including any attachments) are being sent
>>>> toyou on the condition of confidentiality and may be protected by
>>>> legalprivilege. Access to this email by anyone other than the intended
>>>> recipientis unauthorized. If you are not the intended recipient, please
>>>> immediatelynotify the sender by replying to this message and delete the
>>>> materialimmediately from your system. Any further use, dissemination,
>>>> distributionor reproduction of this email is strictly prohibited. Further,
>>>> norepresentation is made with respect to any content contained in this
>>>> email.*
>>>
>>>
>>
>>
>>
>>
>>
>>
>>
>> *This email and its contents (including any attachments) are being sent
>> toyou on the condition of confidentiality and may be protected by
>> legalprivilege. Access to this email by anyone other than the intended
>> recipientis unauthorized. If you are not the intended recipient, please
>> immediatelynotify the sender by replying to this message and delete the
>> materialimmediately from your system. Any further use, dissemination,
>> distributionor reproduction of this email is strictly prohibited. Further,
>> norepresentation is made with respect to any content contained in this
>> email.*
>
>
--
*This email and its contents (including any attachments) are being sent to
you on the condition of confidentiality and may be protected by legal
privilege. Access to this email by anyone other than the intended recipient
is unauthorized. If you are not the intended recipient, please immediately
notify the sender by replying to this message and delete the material
immediately from your system. Any further use, dissemination, distribution
or reproduction of this email is strictly prohibited. Further, no
representation is made with respect to any content contained in this email.*
