Re: Thin Client examples for documentation

[Denis Magda]

> We need to make sure the spec is correct, so someone else (not me) has to
> follow the spec, write code and see if it works.
> It helps finding issues like missing 27 code, etc.

Exactly, Pavel! We’re on the same page here. Thanks for the help and 
responsiveness.

—
Denis

> On Jan 12, 2018, at 3:08 AM, Pavel Tupitsyn  wrote:
> 
> Hi Prachi,
> 
> I've forked your repo, my changes are there:
> https://github.com/ptupitsyn/ignite-examples/blob/fix/src/main/java/ignite/myexamples/thinclient/ThinClientExample2.java
>  
> 
> 
> * getOrCreateCache fixed
> * 27: documentation updated, I've forgot about wrapped complex objects
> * Common readBinaryObject method added, you can extend it further. Works
> with doSQLQuery
> * getQueryCursorPage: fixed by using readBinaryObject method
> * putBinaryType: Fixed that for you. Request length was wrong, and some
> extra data in the schema.
> * doQueryScan: filter object is a Java object
> implementing IgniteBiPredicate, serialized as BinaryObject.
> 
> I could write all the code for you, but that does not make sense.
> We need to make sure the spec is correct, so someone else (not me) has to
> follow the spec, write code and see if it works.
> It helps finding issues like missing 27 code, etc.
> 
> Thanks,
> Pavel
> 
> 
> 
> On Fri, Jan 12, 2018 at 4:57 AM, Prachi Garg  > wrote:
> 
>> The question was incomplete for one of the points :-)
>> 
>> 
>>   - doQueryScan() - I could not create a proper request.  If I want to
>>  scan for records where salary>1000, then what is the filter object
>>  in the request? Where is the query in the request?
>> 
>> 
>> On Thu, Jan 11, 2018 at 5:47 PM, Prachi Garg  wrote:
>> 
>>> Hi Pavel,
>>> 
>>> Thanks for your input. I have already used the debugger for several other
>>> operations :-) However, these 4 are giving me a hard time. Now as you
>>> suggested -
>>> 
>>>   1. I have created separate methods for request and response header,
>>>   and writeString. The code is much cleaner now.
>>>   2. I have tried to debug on the server side, but they all have
>>>   different issues -
>>> 
>>> 
>>>   - doSQLQuery() - Request is fine, but in response I get 27 as the
>>>  type code for value which is not mentioned in the wiki doc.
>>>  - getQueryCursorPage() - Request is fine, but response is some
>>>  lengthy numbers.
>>>  - putBinaryType() - I could not create a proper request. From the
>>>  wiki docs, it is unclear what "Binary schemas, set of (schemaId + 
>>> fieldIds)
>>>  pairs"
>>>  - doQueryScan() - I could not create a proper request. What is
>>>  filter object in the request?
>>> 
>>> Please go through the above 4 methods and fix the issues. I need your
>>> help to finish this doc before the release.
>>> 
>>> Also, until I get OP_QUERY_SCAN and OP_PUT_BINARY_TYPE working, I cannot
>>> write examples for OP_QUERY_SCAN_CURSOR_GET_PAGE and OP_GET_BINARY_TYPE.
>>> 
>>> Thanks,
>>> -P
>>> 
>>> On Thu, Jan 11, 2018 at 7:36 AM, Pavel Tupitsyn 
>>> wrote:
>>> 
 Hi Prachi,
 
 I've fixed cache creation method for you, see attachment. I did not fix
 anything else.
 Sorry, but this kind of code with hardcoded message lengths, operation
 codes, etc is very hard to work with.
 Hardcoded values may be useful for trivial operations so that users get
 an idea of the protocol.
 But for complex stuff like SQL this gets out of hand quickly.
 
 My recommendations:
 - Create a common SendRequest method which will deal with message
 lengths, op codes and request ids automatically
 - Create writeString method to deal with UTF stuff in one place
 - When something does not work, use debugger on the server side
 (see ClientMessageParser class), it is easy to step through and see which
 value went wrong
 
 Thanks,
 Pavel
 
 On Wed, Jan 10, 2018 at 11:58 PM, Denis Magda  wrote:
 
> Pavel, as a side note,
> 
> The methods/operations Prachi is struggling with look pretty standard
> to me.
> 
> Do you have tests for them in the code base? I mean *not* the tests you
> shared before where we use existing internal binary marshaller APIs but
> where we code every operation from scratch (what Prachi is doing for
> documentation code snippets).
> 
> Such tests would help to complete the doc quicker and would ensure that
> the protocol works as expected on the user side where people are not going
> to sit on the internal binary marshaller apis.
> 
> —
> Denis
> 
>> On Jan 10, 2018, at 12:29 PM, Prachi Garg  wrote:
>> 
>> Pavel,
>> 
>> I am having trouble creating examples for some of the thin protocol
>> operations. I have uploaded my project on github -
>> 
>> https://github.com/pgarg/ignite-exampl

Re: Thin Client examples for documentation

[Pavel Tupitsyn]

Hi Prachi,

I've forked your repo, my changes are there:
https://github.com/ptupitsyn/ignite-examples/blob/fix/src/main/java/ignite/myexamples/thinclient/ThinClientExample2.java

* getOrCreateCache fixed
* 27: documentation updated, I've forgot about wrapped complex objects
* Common readBinaryObject method added, you can extend it further. Works
with doSQLQuery
* getQueryCursorPage: fixed by using readBinaryObject method
* putBinaryType: Fixed that for you. Request length was wrong, and some
extra data in the schema.
* doQueryScan: filter object is a Java object
implementing IgniteBiPredicate, serialized as BinaryObject.

I could write all the code for you, but that does not make sense.
We need to make sure the spec is correct, so someone else (not me) has to
follow the spec, write code and see if it works.
It helps finding issues like missing 27 code, etc.

Thanks,
Pavel



On Fri, Jan 12, 2018 at 4:57 AM, Prachi Garg  wrote:

> The question was incomplete for one of the points :-)
>
>
>- doQueryScan() - I could not create a proper request.  If I want to
>   scan for records where salary>1000, then what is the filter object
>   in the request? Where is the query in the request?
>
>
> On Thu, Jan 11, 2018 at 5:47 PM, Prachi Garg  wrote:
>
>> Hi Pavel,
>>
>> Thanks for your input. I have already used the debugger for several other
>> operations :-) However, these 4 are giving me a hard time. Now as you
>> suggested -
>>
>>1. I have created separate methods for request and response header,
>>and writeString. The code is much cleaner now.
>>2. I have tried to debug on the server side, but they all have
>>different issues -
>>
>>
>>- doSQLQuery() - Request is fine, but in response I get 27 as the
>>   type code for value which is not mentioned in the wiki doc.
>>   - getQueryCursorPage() - Request is fine, but response is some
>>   lengthy numbers.
>>   - putBinaryType() - I could not create a proper request. From the
>>   wiki docs, it is unclear what "Binary schemas, set of (schemaId + 
>> fieldIds)
>>   pairs"
>>   - doQueryScan() - I could not create a proper request. What is
>>   filter object in the request?
>>
>> Please go through the above 4 methods and fix the issues. I need your
>> help to finish this doc before the release.
>>
>> Also, until I get OP_QUERY_SCAN and OP_PUT_BINARY_TYPE working, I cannot
>> write examples for OP_QUERY_SCAN_CURSOR_GET_PAGE and OP_GET_BINARY_TYPE.
>>
>> Thanks,
>> -P
>>
>> On Thu, Jan 11, 2018 at 7:36 AM, Pavel Tupitsyn 
>> wrote:
>>
>>> Hi Prachi,
>>>
>>> I've fixed cache creation method for you, see attachment. I did not fix
>>> anything else.
>>> Sorry, but this kind of code with hardcoded message lengths, operation
>>> codes, etc is very hard to work with.
>>> Hardcoded values may be useful for trivial operations so that users get
>>> an idea of the protocol.
>>> But for complex stuff like SQL this gets out of hand quickly.
>>>
>>> My recommendations:
>>> - Create a common SendRequest method which will deal with message
>>> lengths, op codes and request ids automatically
>>> - Create writeString method to deal with UTF stuff in one place
>>> - When something does not work, use debugger on the server side
>>> (see ClientMessageParser class), it is easy to step through and see which
>>> value went wrong
>>>
>>> Thanks,
>>> Pavel
>>>
>>> On Wed, Jan 10, 2018 at 11:58 PM, Denis Magda  wrote:
>>>
 Pavel, as a side note,

 The methods/operations Prachi is struggling with look pretty standard
 to me.

 Do you have tests for them in the code base? I mean *not* the tests you
 shared before where we use existing internal binary marshaller APIs but
 where we code every operation from scratch (what Prachi is doing for
 documentation code snippets).

 Such tests would help to complete the doc quicker and would ensure that
 the protocol works as expected on the user side where people are not going
 to sit on the internal binary marshaller apis.

 —
 Denis

 > On Jan 10, 2018, at 12:29 PM, Prachi Garg  wrote:
 >
 > Pavel,
 >
 > I am having trouble creating examples for some of the thin protocol
 > operations. I have uploaded my project on github -
 >
 > https://github.com/pgarg/ignite-examples/blob/master/src/mai
 n/java/ignite/myexamples/thinclient/ThinClientExample2.java
 >
 > Please look into the following methods and provide a fix for them:
 >
 >   - doSQLQuery()
 >   - getQueryCursorPage()
 >   - putBinaryType()
 >   - doQueryScan()
 >   - createCacheWithConfiguration()
 >
 > Thanks,
 > -Prachi


>>>
>>
>

Re: Thin Client examples for documentation

[Prachi Garg]

Pavel,

I don't see any attachment. Can you please resend.

-P

On Thu, Jan 11, 2018 at 7:36 AM, Pavel Tupitsyn 
wrote:

> Hi Prachi,
>
> I've fixed cache creation method for you, see attachment. I did not fix
> anything else.
> Sorry, but this kind of code with hardcoded message lengths, operation
> codes, etc is very hard to work with.
> Hardcoded values may be useful for trivial operations so that users get an
> idea of the protocol.
> But for complex stuff like SQL this gets out of hand quickly.
>
> My recommendations:
> - Create a common SendRequest method which will deal with message lengths,
> op codes and request ids automatically
> - Create writeString method to deal with UTF stuff in one place
> - When something does not work, use debugger on the server side
> (see ClientMessageParser class), it is easy to step through and see which
> value went wrong
>
> Thanks,
> Pavel
>
> On Wed, Jan 10, 2018 at 11:58 PM, Denis Magda  wrote:
>
>> Pavel, as a side note,
>>
>> The methods/operations Prachi is struggling with look pretty standard to
>> me.
>>
>> Do you have tests for them in the code base? I mean *not* the tests you
>> shared before where we use existing internal binary marshaller APIs but
>> where we code every operation from scratch (what Prachi is doing for
>> documentation code snippets).
>>
>> Such tests would help to complete the doc quicker and would ensure that
>> the protocol works as expected on the user side where people are not going
>> to sit on the internal binary marshaller apis.
>>
>> —
>> Denis
>>
>> > On Jan 10, 2018, at 12:29 PM, Prachi Garg  wrote:
>> >
>> > Pavel,
>> >
>> > I am having trouble creating examples for some of the thin protocol
>> > operations. I have uploaded my project on github -
>> >
>> > https://github.com/pgarg/ignite-examples/blob/master/src/
>> main/java/ignite/myexamples/thinclient/ThinClientExample2.java
>> >
>> > Please look into the following methods and provide a fix for them:
>> >
>> >   - doSQLQuery()
>> >   - getQueryCursorPage()
>> >   - putBinaryType()
>> >   - doQueryScan()
>> >   - createCacheWithConfiguration()
>> >
>> > Thanks,
>> > -Prachi
>>
>>
>

Re: Thin Client examples for documentation

[Denis Magda]

Please share a link. These tests still use Ignite predefined types as SqlQuery, 
SqlFields query and other existing Ignite APIs:
https://github.com/apache/ignite/tree/master/modules/platforms/dotnet/Apache.Ignite.Core.Tests/Client/Cache

I’m looking for test that will 100% correspond to what Prachi is it trying to 
do.

—
Denis

> On Jan 11, 2018, at 7:40 AM, Pavel Tupitsyn  wrote:
> 
> Denis,
> 
> Thin client protocol is fully covered by .NET Thin Client tests. Each and
> every operation is tested.
> 
> On Thu, Jan 11, 2018 at 6:36 PM, Pavel Tupitsyn 
> wrote:
> 
>> Hi Prachi,
>> 
>> I've fixed cache creation method for you, see attachment. I did not fix
>> anything else.
>> Sorry, but this kind of code with hardcoded message lengths, operation
>> codes, etc is very hard to work with.
>> Hardcoded values may be useful for trivial operations so that users get an
>> idea of the protocol.
>> But for complex stuff like SQL this gets out of hand quickly.
>> 
>> My recommendations:
>> - Create a common SendRequest method which will deal with message lengths,
>> op codes and request ids automatically
>> - Create writeString method to deal with UTF stuff in one place
>> - When something does not work, use debugger on the server side
>> (see ClientMessageParser class), it is easy to step through and see which
>> value went wrong
>> 
>> Thanks,
>> Pavel
>> 
>> On Wed, Jan 10, 2018 at 11:58 PM, Denis Magda  wrote:
>> 
>>> Pavel, as a side note,
>>> 
>>> The methods/operations Prachi is struggling with look pretty standard to
>>> me.
>>> 
>>> Do you have tests for them in the code base? I mean *not* the tests you
>>> shared before where we use existing internal binary marshaller APIs but
>>> where we code every operation from scratch (what Prachi is doing for
>>> documentation code snippets).
>>> 
>>> Such tests would help to complete the doc quicker and would ensure that
>>> the protocol works as expected on the user side where people are not going
>>> to sit on the internal binary marshaller apis.
>>> 
>>> —
>>> Denis
>>> 
 On Jan 10, 2018, at 12:29 PM, Prachi Garg  wrote:
 
 Pavel,
 
 I am having trouble creating examples for some of the thin protocol
 operations. I have uploaded my project on github -
 
 https://github.com/pgarg/ignite-examples/blob/master/src/
>>> main/java/ignite/myexamples/thinclient/ThinClientExample2.java
 
 Please look into the following methods and provide a fix for them:
 
  - doSQLQuery()
  - getQueryCursorPage()
  - putBinaryType()
  - doQueryScan()
  - createCacheWithConfiguration()
 
 Thanks,
 -Prachi
>>> 
>>> 
>> 

Re: Thin Client examples for documentation

[Pavel Tupitsyn]

Denis,

Thin client protocol is fully covered by .NET Thin Client tests. Each and
every operation is tested.

On Thu, Jan 11, 2018 at 6:36 PM, Pavel Tupitsyn 
wrote:

> Hi Prachi,
>
> I've fixed cache creation method for you, see attachment. I did not fix
> anything else.
> Sorry, but this kind of code with hardcoded message lengths, operation
> codes, etc is very hard to work with.
> Hardcoded values may be useful for trivial operations so that users get an
> idea of the protocol.
> But for complex stuff like SQL this gets out of hand quickly.
>
> My recommendations:
> - Create a common SendRequest method which will deal with message lengths,
> op codes and request ids automatically
> - Create writeString method to deal with UTF stuff in one place
> - When something does not work, use debugger on the server side
> (see ClientMessageParser class), it is easy to step through and see which
> value went wrong
>
> Thanks,
> Pavel
>
> On Wed, Jan 10, 2018 at 11:58 PM, Denis Magda  wrote:
>
>> Pavel, as a side note,
>>
>> The methods/operations Prachi is struggling with look pretty standard to
>> me.
>>
>> Do you have tests for them in the code base? I mean *not* the tests you
>> shared before where we use existing internal binary marshaller APIs but
>> where we code every operation from scratch (what Prachi is doing for
>> documentation code snippets).
>>
>> Such tests would help to complete the doc quicker and would ensure that
>> the protocol works as expected on the user side where people are not going
>> to sit on the internal binary marshaller apis.
>>
>> —
>> Denis
>>
>> > On Jan 10, 2018, at 12:29 PM, Prachi Garg  wrote:
>> >
>> > Pavel,
>> >
>> > I am having trouble creating examples for some of the thin protocol
>> > operations. I have uploaded my project on github -
>> >
>> > https://github.com/pgarg/ignite-examples/blob/master/src/
>> main/java/ignite/myexamples/thinclient/ThinClientExample2.java
>> >
>> > Please look into the following methods and provide a fix for them:
>> >
>> >   - doSQLQuery()
>> >   - getQueryCursorPage()
>> >   - putBinaryType()
>> >   - doQueryScan()
>> >   - createCacheWithConfiguration()
>> >
>> > Thanks,
>> > -Prachi
>>
>>
>

Re: Thin Client examples for documentation

[Pavel Tupitsyn]

Hi Prachi,

I've fixed cache creation method for you, see attachment. I did not fix
anything else.
Sorry, but this kind of code with hardcoded message lengths, operation
codes, etc is very hard to work with.
Hardcoded values may be useful for trivial operations so that users get an
idea of the protocol.
But for complex stuff like SQL this gets out of hand quickly.

My recommendations:
- Create a common SendRequest method which will deal with message lengths,
op codes and request ids automatically
- Create writeString method to deal with UTF stuff in one place
- When something does not work, use debugger on the server side
(see ClientMessageParser class), it is easy to step through and see which
value went wrong

Thanks,
Pavel

On Wed, Jan 10, 2018 at 11:58 PM, Denis Magda  wrote:

> Pavel, as a side note,
>
> The methods/operations Prachi is struggling with look pretty standard to
> me.
>
> Do you have tests for them in the code base? I mean *not* the tests you
> shared before where we use existing internal binary marshaller APIs but
> where we code every operation from scratch (what Prachi is doing for
> documentation code snippets).
>
> Such tests would help to complete the doc quicker and would ensure that
> the protocol works as expected on the user side where people are not going
> to sit on the internal binary marshaller apis.
>
> —
> Denis
>
> > On Jan 10, 2018, at 12:29 PM, Prachi Garg  wrote:
> >
> > Pavel,
> >
> > I am having trouble creating examples for some of the thin protocol
> > operations. I have uploaded my project on github -
> >
> > https://github.com/pgarg/ignite-examples/blob/master/
> src/main/java/ignite/myexamples/thinclient/ThinClientExample2.java
> >
> > Please look into the following methods and provide a fix for them:
> >
> >   - doSQLQuery()
> >   - getQueryCursorPage()
> >   - putBinaryType()
> >   - doQueryScan()
> >   - createCacheWithConfiguration()
> >
> > Thanks,
> > -Prachi
>
>

Re: Thin Client examples for documentation

[Denis Magda]

Pavel, as a side note,

The methods/operations Prachi is struggling with look pretty standard to me.

Do you have tests for them in the code base? I mean *not* the tests you shared 
before where we use existing internal binary marshaller APIs but where we code 
every operation from scratch (what Prachi is doing for documentation code 
snippets).

Such tests would help to complete the doc quicker and would ensure that the 
protocol works as expected on the user side where people are not going to sit 
on the internal binary marshaller apis.

—
Denis

> On Jan 10, 2018, at 12:29 PM, Prachi Garg  wrote:
> 
> Pavel,
> 
> I am having trouble creating examples for some of the thin protocol
> operations. I have uploaded my project on github -
> 
> https://github.com/pgarg/ignite-examples/blob/master/src/main/java/ignite/myexamples/thinclient/ThinClientExample2.java
> 
> Please look into the following methods and provide a fix for them:
> 
>   - doSQLQuery()
>   - getQueryCursorPage()
>   - putBinaryType()
>   - doQueryScan()
>   - createCacheWithConfiguration()
> 
> Thanks,
> -Prachi