Re: Signing off Ignite for export beyond the U.S.

[Alexey Kosenchuk]

Hi Denis,

Info about Python, PHP, Node.JS thin clients:

The clients itself do not contain any cryptographic code but use the 
features provided by the underlying platforms.


Python client uses Python's SSL lib [1] which is a wrapper over OpenSSL [2].
Python 2.7 and Python 3.4 require OpenSSL 1.0, Python 3.5 and higher 
require OpenSSL 1.1.


NodeJS client uses NodeJS's tls module [3] which is a wrapper over 
OpenSSL [2].

NodeJS 8.x requires OpenSSL 1.0, NodeJS 10.x and higher require OpenSSL 1.1.

PHP client uses PHP OpenSSL extension [4].
PHP 7.2 and higher require OpenSSL 1.0.

[1] https://docs.python.org/3/library/ssl.html
[2] https://www.openssl.org/
[3] https://nodejs.org/api/tls.html
[4] https://www.php.net/manual/en/book.openssl.php

Regards,
-Alexey


Igniters,

Regardless of the fact that Ignite is an open source software, ASF as an 
entity based in the U.S. has to comply with certain exporting 
regulations [1].


Dmitry Pavlov and I are working on adding Ignite to the table [2] of 
projects allowed for export and might need the assistance of some of you.


Here is a list of cryptographic functions used by Ignite (and provided 
by a 3rd party vendor):


 1. JDK SSL/TLS libraries if a user wishes to enable secured
connectivity between cluster nodes. Manufacturer - Oracle/OpenJDK
(https://apacheignite.readme.io/docs/ssltls)
 2. JDK AES/CBC/PKCS5Padding encryption from the Java libraries for
transparent data encryption of data on disk
(https://apacheignite.readme.io/docs/transparent-data-encryption)
 3. Libraries/vendors for .NET nodes security?*Pavel Tupitsyn*, could
you check?
 4. Libraries/vendors for C++ clients security (SSL, TLS, anything
else?). *Igor Sapego*, could you please check?
 5. Libraries/vendors for Python, PHP, Node.JS SSL/TLS? *Dear thin
client contributors*, please facilitate.
 6. Anything else missing from the list? We don't have any custom crypto
features, right?

All of these usages/integrations have to comply with the following 
checklist [3] before I, as a PMC Chair, submit a notice to Export 
Administration Regulations of the U.S.A.


[1] http://www.apache.org/licenses/exports/
[2] http://www.apache.org/licenses/exports/#matrix
[3] https://www.apache.org/dev/crypto.html#classify


-
Denis

[jira] [Created] (IGNITE-11540) Node.js thin client: best effort affinity

[Alexey Kosenchuk (JIRA)]

Alexey Kosenchuk created IGNITE-11540:
-

 Summary: Node.js thin client: best effort affinity
 Key: IGNITE-11540
 URL: https://issues.apache.org/jira/browse/IGNITE-11540
 Project: Ignite
  Issue Type: New Feature
  Components: thin client
Affects Versions: 2.7
Reporter: Alexey Kosenchuk
Assignee: Alexey Kosenchuk


Implement 
[IEP-23|https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients]
 for Node.js thin client.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Thin clients all in one

[Alexey Kosenchuk]

Hi Stepan,

1) About UUID...

UUID binary encoding must be defined in the Binary Protocol 
specification 
(https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-uuid-guid-)

So, it's the issue against the spec first.

Regarding the clients..

Python client uses Python UUID class 
(https://docs.python.org/2/library/uuid.html) which, if I understand 
correctly, can operate with different encodings. Dmitry M. can correct me.


NodeJs and PHP do not have standard UUID classes.
So, the clients return/accept UUID as just a 16-byte array with binary 
encoding inside, exactly as it is read / will be written via the binary 
protocol.

An application may convert this array to/from a required representation.

2) About Timestamp...

Python, NodeJs and PHP clients behave strictly according to the Binary 
Protocol specification 
(https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-timestamp) 
which defines that the second field is "a nanoseconds fraction of a last 
millisecond, which value could be in a range from 0 to 99."


Regards,
-Alexey

14.12.2018 17:41, Stepan Pilschikov пишет:

Hello again

Starting to check compatibility between thin clients in java/c++ and 
py/php/nodejs and met some problems


1) Found that UUID mixed in a strange way if we taken it from Java/C++ 
to PY/PHP/JS client and backwards
Have issue about it https://issues.apache.org/jira/browse/IGNITE-10691, 
please look at it


2) Is more like conversation about seamless experience between all thin 
clients
PY/PHP/JS/C++/Java have Timestamp data type which is contain 
tuple(millis, nanos)

Main concern about nanos

In PHP/JS/Py Nanos count like "nanoseconds of the last millisecond" but 
in Java and C++ its like we have millis and same millis but with nanos 
and its strange for PHP/PY/JS if we take Java how is Base thin client


For example in Java we have new Timestamp(11) and it 
(fastTime=111000, nanos=11100)
In C++ if we getting this cache we take (millis=11, nanos=11100) 
which is kind a right and same as in java
But in py/php/js its look like (millis=11, nanos=0), its kind a 
understandable what logic is, but sill different behavior and experience

What you thinking about it?
For now i can't understand why its done how its done, looks like it 
should be same but something going wrong


пт, 30 нояб. 2018 г. в 01:18, Alexey Kosenchuk 
mailto:alexey.kosenc...@nobitlost.com>>:


Hi Stepan,

pls check the Ignite cfg you use - see the comments in the jira.

Also, the examples executors (including AuthTlsExample) are included
into NodeJS test suite in TeamCity which run periodically and
successfully passed. Eg. the latest one:

https://ci.ignite.apache.org/viewLog.html?buildId=2426645=buildResultsDiv=IgniteTests24Java8_ThinClientNodeJs

Regards,
-Alexey

28.11.2018 17:08, Stepan Pilschikov пишет:
 > Hello again
 >
 > If NodeJS sources found that example AuthTlsExample.js throwing
exception
 > during execution
 > Output and grid configuration in
 > https://issues.apache.org/jira/browse/IGNITE-10447
 >
 > Can someone have a look at it?
 >
 > вс, 25 нояб. 2018 г. в 19:11, Stepan Pilschikov
mailto:pilshchikov@gmail.com>>:
 >
 >> My bad,
 >> You right
 >>
 >> вс, 25 нояб. 2018 г. в 05:37, Dmitry Melnichuk <
 >> dmitry.melnic...@nobitlost.com
<mailto:dmitry.melnic...@nobitlost.com>>:
 >>
 >>> Stepan,
 >>>
 >>> AFAIK Map type did always behave correctly on client side, as
it does
 >>> now. This is a corresponding piece of my test suite:
 >>>
 >>> ```
 >>> def test_put_get_map(client):
 >>>
 >>>       cache = client.get_or_create_cache('test_map_cache')
 >>>
 >>>       cache.put(
 >>>           'test_map',
 >>>           (
 >>>               MapObject.HASH_MAP,
 >>>               {
 >>>                   (123, IntObject): 'test_data',
 >>>                   456: ((1, [456, 'inner_test_string', 789]),
 >>> CollectionObject),
 >>>                   'test_key': 32.4,
 >>>               }
 >>>           ),
 >>>           value_hint=MapObject
 >>>       )
 >>>       value = cache.get('test_map')
 >>>       assert value == (MapObject.HASH_MAP, {
 >>>           123: 'test_data',
 >>>           456: (1, [456, 'inner_test_string', 789]),
 >>>           'test_key': 32.4,
 >>>       })
 >>>
 >>> ```
 >>>
 >>> Or is there another, more

Re: [RESULT] [VOTE] Apache Ignite 2.7.0 Release (RC2)

[Alexey Kosenchuk]

NodeJs client:
--
Instruction - 
https://github.com/nobitlost/ignite/blob/ignite--docs/modules/platforms/nodejs/README.md#publish-ignite-nodejs-client-on-npmjscom-instruction

Uploaded - https://www.npmjs.com/package/apache-ignite-client

PHP client:
---
Instruction - 
https://github.com/nobitlost/ignite/blob/ignite-7783-docs/modules/platforms/php/README.md#release-the-client-in-the-php-package-repository-instruction
Cannot be uploaded on Packagist as the client should be in a dedicated 
repository for that - 
https://issues.apache.org/jira/browse/IGNITE-7783?focusedCommentId=16595476=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16595476

Installation from the sources works.

Python client:
--
Dmitry M. will provide the details.

-Alexey

10.12.2018 14:15, Petr Ivanov пишет:

AFAIK, there is no such procedure yet and it definitely requires introduction.
I guess, thin client developers should share the process of updating official 
repositories.



On 10 Dec 2018, at 09:49, Nikolay Izhikov  wrote:

Hello, Roman.

Do we have release procedure for this pacakage?
Let's develop one.

Thin client developers, please,let me know, which artifacts should be uploaded 
to which package manager system.


В Пн, 10/12/2018 в 04:30 +, Roman Shtykh пишет:

Nikolay,
Do you know when npm (and other thin client) package will be uploaded?

   - $ npm i apache-ignite-client --save
   - npm ERR! code E404
   - npm ERR! 404 Not Found: apache-ignite-client@lates


-- Roman


On Thursday, December 6, 2018, 7:20:33 a.m. GMT+9, Prachi Garg 
 wrote:

Hello Igniters,

The documentation for Apache Ignite 2.7.0 is now public on readme.io
.

-Prachi

On Wed, Dec 5, 2018 at 9:35 AM Nikolay Izhikov  wrote:


Yes, Denis.

I will do my best to complete release steps today.
The only things that blocks me is bintray permissions.
They required to upload rpm and deb packages to corresponding repository.

I've created ticket to the INFRA [1]
Seems, they, resolve it eventually.

We discussed this issue with other Igniters and decided to do release
without rpm and deb. When I got permissions, I release them too.

[1] https://issues.apache.org/jira/browse/INFRA-17354

ср, 5 дек. 2018 г., 20:25 Denis Magda dma...@apache.org:


When are going to complete the post-release steps and announce the


release?

Once the binaries are available on the website we need to release the


docs.



I'm ready to prepare a blog post for blogs.apache.org.

--
Denis

On Mon, Dec 3, 2018 at 11:14 PM Nikolay Izhikov 
wrote:


Sorry, Alex.

I miss your +1.
Thank you, very much for checking RC artifacts.

вт, 4 дек. 2018 г., 7:10 Alexey Kuznetsov akuznet...@apache.org:


Nikolay,

Actually 4 "+1"  binding.

You did not count my "+1".

:)


On Tue, Dec 4, 2018 at 4:28 AM Nikolay Izhikov 
wrote:


Igniters,

Apache Ignite 2.7.0 release (RC2) has been accepted.

3 "+1" binding votes received:

- Pavel Tupitsyn
- Dmitriy Pavlov
- Nikolay Izhikov

Vote thread:





http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Apache-Ignite-2-7-0-RC2-td38788.html





--
Alexey Kuznetsov

Re: Thin clients all in one

[Alexey Kosenchuk]

Hi Stepan,

pls check the Ignite cfg you use - see the comments in the jira.

Also, the examples executors (including AuthTlsExample) are included 
into NodeJS test suite in TeamCity which run periodically and 
successfully passed. Eg. the latest one: 
https://ci.ignite.apache.org/viewLog.html?buildId=2426645=buildResultsDiv=IgniteTests24Java8_ThinClientNodeJs


Regards,
-Alexey

28.11.2018 17:08, Stepan Pilschikov пишет:

Hello again

If NodeJS sources found that example AuthTlsExample.js throwing exception
during execution
Output and grid configuration in
https://issues.apache.org/jira/browse/IGNITE-10447

Can someone have a look at it?

вс, 25 нояб. 2018 г. в 19:11, Stepan Pilschikov :


My bad,
You right

вс, 25 нояб. 2018 г. в 05:37, Dmitry Melnichuk <
dmitry.melnic...@nobitlost.com>:


Stepan,

AFAIK Map type did always behave correctly on client side, as it does
now. This is a corresponding piece of my test suite:

```
def test_put_get_map(client):

  cache = client.get_or_create_cache('test_map_cache')

  cache.put(
  'test_map',
  (
  MapObject.HASH_MAP,
  {
  (123, IntObject): 'test_data',
  456: ((1, [456, 'inner_test_string', 789]),
CollectionObject),
  'test_key': 32.4,
  }
  ),
  value_hint=MapObject
  )
  value = cache.get('test_map')
  assert value == (MapObject.HASH_MAP, {
  123: 'test_data',
  456: (1, [456, 'inner_test_string', 789]),
  'test_key': 32.4,
  })

```

Or is there another, more specific problem with maps?

Dmitry

On 11/25/18 3:56 AM, Stepan Pilschikov wrote:

Dmitry,

Great, checked, now all things woks well
Hope that Igor made review for this PR

But what about Maps? Looks like different ticket? or it can be done in

same

ticket scope?

пт, 23 нояб. 2018 г. в 23:58, Dmitry Melnichuk <
dmitry.melnic...@nobitlost.com>:


Stepan,

Sorry, I forgot to update from upstream prior to start working on this
issue, and thus brought a regression. My bad. Just merged with the
latest master. Please, check it out again.

Dmitry

On 11/24/18 1:37 AM, Stepan Pilschikov wrote:

Dmitry,

Iv checked and its actually work
But a specially in this branch i found another bug
Please look at my last comment:




https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285


пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
dmitry.melnic...@nobitlost.com>:


Stepan,

Thank you for your great job in evaluating Python thin client, as

well

as other thin clients.

There was indeed a bug in Python client regarding the handling of

type

hints in Collection type. I created a fix and did a PR under
IGNITE-10358 task, but the same PR is also fixes the problem in
IGNITE-10230 task.

As of handling the type mapping in gists you provided, I left

comments

on both tasks.

Dmitry

On 11/21/18 6:37 PM, Stepan Pilschikov wrote:

Dmitry, Alexey

Thank you for help, this answers help me a lot with understanding

how

clients are work

Not so long time ago i met problem which is have expected behavior,

but

its

may broke some workflows in future for some users

Its all about not specified data types in collections and map's
All description and examples in
https://issues.apache.org/jira/browse/IGNITE-10358

Dmitry, can you have a quick look at it and maybe in future somehow

fix

it?


пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
dmitry.melnic...@nobitlost.com>:


Stepan!

TL/DR: what you got with Python client in your gist is an intended
behavior.

Explanation: As per docs, Object array contains of type ID (which

is

defaults to -1) and an array of objects.








https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array


Your gist might be fixed accordingly:

```
from pyignite import Client
from pyignite.datatypes import *

OBJECT_ARRAY_TYPE_ID = -1
OBJECT_ARRAY_CONTENTS = [1, 2]

client = Client()
client.connect('127.0.0.1', 10800)
cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
cache.put(
 1,
 (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
 key_hint=IntObject,
 value_hint=ObjectArrayObject,
)

# Python  output: print(cache.get(1))
# (-1, [1, 2])
```

The situation is similar with Map and Collection, they have types.

Types

and type IDs are mostly useless in Python, but I left them for
interoperability reasons. If you think I should kick them out, just

let

me know.

The usage of these 3 data types is documented and tested. You can

refer

to the table in “Data types” section:








https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html


The tests are here:








https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124


On 10/26/18 11:57 PM, Stepan Pilschikov wrote:

Hi, everyone

Re: NodeJS thin client: full API

[Alexey Kosenchuk]

Done for NodeJs and PHP.
Tests added as well.
https://issues.apache.org/jira/browse/IGNITE-10022
https://github.com/apache/ignite/pull/5187

-Alexey

24.10.2018 17:41, Igor Sapego пишет:

Pavel,

Can we add a proper check and throw proper exception
when trying to deserialize enum value? NPE does not say
anything to a user.

Best Regards,
Igor


On Wed, Oct 24, 2018 at 5:34 PM Pavel Petroshenko 
wrote:


Hi Stepan,

Nodejs and PHP clients do not support enum type registration (and hence no
tests). So enum type must be registered from somewhere else in order to be
put or get from the Thin clients.

If you register the type say from Java, then put/get for Enum values should
work from the Thin clients. Can you please try this and let me know how it
works.

Thanks,
p.

On Tue, Oct 23, 2018 at 6:30 AM Stepan Pilshchikov <
pilshchikov@gmail.com> wrote:


Alexey,

I'm trying to get Enum from cache which placed by different thin client

and

find that i can't get it, only exception

And more when im trying to get Enum which is placed by nodejs client i

also

can't do it, have same exception

code and output -
https://gist.github.com/pilshchikov/ca9fda160e310f62dd4d1d6dbb2f904c

Can you please help me to investigate it, maybe something im doing wrong,
because its look like a bug



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Thin clients all in one

[Alexey Kosenchuk]

thanks, now it's clear.

For NodeJs and PHP clients:
  You cannot use OBJECT_ARRAY constant to specify a field type. This is 
true for all non-primitive types. That's why you get ""type" argument 
has incorrect value" exception.
  Non-primitive types can be specified using special objects. In this 
case - ObjectArrayType.

  All these are described in the API specs.
  So, you need to update your examples:
 PHP: setValueType(new ObjectArrayType())
 NodeJs: setValueType(new ObjectArrayType()) or
 setValueType(new 
ObjectArrayType(ObjectType.PRIMITIVE_TYPE.INTEGER)) (if you want an 
array of integers)


For Python client:
  Dmitry will answer.

Regards,
-Alexey

26.10.2018 17:22, Stepan Pilschikov пишет:

No

Now tried
code and output - 
https://gist.github.com/pilshchikov/4c02057e624baa54326a1e75b4ee3f46

And it looks like it works

пт, 26 окт. 2018 г. в 17:10, Alexey Kosenchuk 
mailto:alexey.kosenc...@nobitlost.com>>:


Hi Stepan,

Have you tried the same with Java and .net thin clients as well and
everything work fine?

Thanks,
-Alexey

26.10.2018 16:57, Stepan Pilschikov пишет:
 > Hi, everyone
 >
 > Create new thread to centralize cross compatibility and others
common
 > problems between thin clients
 >
 > Tying to use Object array to exchange different data between JS,
PHP and
 > Python thin clients
 >
 > JS and PHP simply can't put any type of arrays
 > Python can put data, but if you take it, data would be completely
 > different, maybe during this put process data is changed
 >
 > Code and output:
 > PHP -
https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
 > JS -
https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
 > Python -
 > https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
 >
 > I'm tried different data types (string, double, float, complex
objects,
 > just random objects, char, byte, Date), still
 >
 > How, from my perspective, it should works:
 > put array of any type and then get this array.
 > Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new
Date()] ->
 > get [[Date object], [Date object]] (like in java thin client)
 >
 > Looks like bug in all clients, what you think?
 >
 > I wanted to try Collections, but i think this type have same problem

Re: Thin clients all in one

[Alexey Kosenchuk]

Hi Stepan,

Have you tried the same with Java and .net thin clients as well and 
everything work fine?


Thanks,
-Alexey

26.10.2018 16:57, Stepan Pilschikov пишет:

Hi, everyone

Create new thread to centralize cross compatibility and others common 
problems between thin clients


Tying to use Object array to exchange different data between JS, PHP and 
Python thin clients


JS and PHP simply can't put any type of arrays
Python can put data, but if you take it, data would be completely 
different, maybe during this put process data is changed


Code and output:
PHP - https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
JS - https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
Python - 
https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f


I'm tried different data types (string, double, float, complex objects, 
just random objects, char, byte, Date), still


How, from my perspective, it should works:
put array of any type and then get this array.
Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()] -> 
get [[Date object], [Date object]] (like in java thin client)


Looks like bug in all clients, what you think?

I wanted to try Collections, but i think this type have same problem

Can't use array as a cache key from thin clients (ignite-9991)

[Alexey Kosenchuk]

The problem is reproduced with Java, NodeJs, PHP, Python thin clients.

This seems the server's issue.

Can an expert in the server have a look, please?

The details are in the jira [1]

Thanks,
-Alexey

[1] https://issues.apache.org/jira/browse/IGNITE-9991

[jira] [Created] (IGNITE-10022) JS, PHP thin clients: a more meaningful exception when ENUM type is not registered

[Alexey Kosenchuk (JIRA)]

Alexey Kosenchuk created IGNITE-10022:
-

 Summary: JS, PHP thin clients: a more meaningful exception when 
ENUM type is not registered
 Key: IGNITE-10022
 URL: https://issues.apache.org/jira/browse/IGNITE-10022
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Affects Versions: 2.7
Reporter: Alexey Kosenchuk
Assignee: ekaterina.vergizova


NodeJS and PHP thin clients should throw a more meaningful exception for 
put/get ENUM type data in case the ENUM type is not registered.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: PHP thin client

[Alexey Kosenchuk]

But now in PR we have no readme at all
Agree with examples on readme.io, but other information (how to install, how
to configurate, install from sources, basic description) may be in sources,
it help a lot
Or it also should be on readme.io?

This question because in other platforms have readme exist and they have
this basic information in it


You know, I'm confused with all this documentation stuff...
For nodejs client all docs were moved from the repo to readme.io but the 
readme.md keeps the installation instructions (duplicated with 
readme.io). Probably, that's ok.

Will add similar short readme.md to the PHP PR.

Thanks,
-Alexey

Re: PHP thin client

[Alexey Kosenchuk]

Hi Stepan,


1. Why exists second branches with all documentation? I mean they looks good
together, why need to separate?


Because finally the docs should not be in the repo (unfortunately) but 
on readme.io.

There is a task for that: https://issues.apache.org/jira/browse/IGNITE-9523


Also, one place when i found this branch it is this thread, not in jira, not
in a github


Added the link in the jira.


2. Examples in readme.md have better look in php/examples. I mean
php/examples have one set of examples and readme have second one, they not
match and they both looks good. Why doesn't add readme examples in
php/examples?


Examples in readme.md are more like code snippets (mostly not big) 
intended to help better understanding the usage of the client along the 
reading.
Examples in php/examples are more meaningful and fully cover the 
examples in readme.md.


Thanks,
-Alexey

Re: PHP thin client

[Alexey Kosenchuk]

11.09.2018 14:46, Igor Sapego пишет:

1) I meant auto-generated doc, this page: [1]. By the way,
 many methods do not have any description as well.


Updated - introduction sentences placed on the main page of the 
autogenerated API spec.
All public methods have description, except setters for cache 
configuration which do not need it.



2) You can, for example, print object as you generate and
 then get them from cache.


Done.


3) Yes, 10800 should be the default port.


Done.

All changes are in the updated PR.


4) You are right, I've just used server with SSL enabled.
 Everything works now.


Good.


5) Ok, can you just provide a list of directories/files, that
 should be copied to binary release?


ignite/modules/platforms/php/composer.json
ignite/modules/platforms/php/src
ignite/modules/platforms/php/examples

If cfg file to generate the API spec should be in the release
ignite/modules/platforms/php/api_docs/Doxyfile

If tests should be in the release
ignite/modules/platforms/php/tests

Regards,
-Alexey



[1] -
https://rawgit.com/nobitlost/ignite/ignite-7783-docs/modules/platforms/php/api_docs/html/index.html

Best Regards,
Igor


On Tue, Sep 11, 2018 at 1:31 AM Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:


Hi Igor,

thanks for the review.

Pls see below...

  >> 1. Main page for documentation is empty.

What is the main page for documentation?

As wrote,
the auto-generated API spec is here:

https://rawgit.com/nobitlost/ignite/ignite-7783-docs/modules/platforms/php/api_docs/html/index.html

All other docs are here:

https://github.com/nobitlost/ignite/blob/ignite-7783-docs/modules/platforms/php/README.md
Going to be placed on readme.io by Prachi (thanks!)
https://issues.apache.org/jira/browse/IGNITE-9523

  >> 2. More output for Auth example is needed. This is not a test, after
all, but example.

Will add a log when a connection happens with the details of the
connection. Nothing else to output in this trivial example.
In fact, the real profit for a user here is the source code with an
example of TLS/auth cfg for the client.

  >> 3. If I try run test with APACHE_IGNITE_CLIENT_ENDPOINTS=127.0.0.1,
they fail

Do you mean 10800 should be the default port when not specified by a user?
Will add.

  >> 4. If I try run test with
APACHE_IGNITE_CLIENT_ENDPOINTS=127.0.0.1:10800, they fail

Double checked exactly in the same environment you have. It works.

Please try again.
Do you use a server with all default settings? Did not you try with the
server after the Auth example?
Do other examples work with the same server?
If you still see the problem with the tests, please send the server's
log to us. And/or switch the client's debug on - call setDebug(true) -
and share the client's output.

  >> 5. When "maven package" command is executed on Ignite, no php
directory appears

Need a help from experts / release engineer.

Thanks,
-Alexey

10.09.2018 15:34, Igor Sapego пишет:

By the way, I used Ubuntu 18.04, PHP 7.2.7 and what seems
to be PhpUnit 7.3 (not sure here).

Best Regards,
Igor


On Mon, Sep 10, 2018 at 3:28 PM Igor Sapego  wrote:


Guys, I've reviewed the API (which looks good), run tests and examples

and

here are my commments:

1. Main page for documentation is empty.

2. More output for Auth example is needed. This is not a test, after

all,

but example.

3. If I try run test with APACHE_IGNITE_CLIENT_ENDPOINTS=127.0.0.1, they
fail with the following message:
Apache\Ignite\Exception\NoConnectionException: [127.0.0.1] Failed to

parse

address "127.0.0.1" in


/home/isapego/work/ignite/modules/platforms/php/src/Apache/Ignite/Internal/Connection/ClientFailoverSocket.php:107

Stack trace:
#0


/home/isapego/work/ignite/modules/platforms/php/src/Apache/Ignite/Internal/Connection/ClientFailoverSocket.php(54):



Apache\Ignite\Internal\Connection\ClientFailoverSocket->failoverConnect()

#1


/home/isapego/work/ignite/modules/platforms/php/src/Apache/Ignite/Client.php(61):



Apache\Ignite\Internal\Connection\ClientFailoverSocket->connect(Object(Apache\Ignite\ClientConfiguration))

#2


/home/isapego/work/ignite/modules/platforms/php/tests/TestingHelper.php(52):

Apache\Ignite\Client->connect(Object(Apache\Ignite\ClientConfiguration))
#3


/home/isapego/work/ignite/modules/platforms/php/tests/SqlQueryTest.php(49):

Apache\Ignite\Tests\TestingHelper::init()
#4


/home/isapego/work/ignite/modules/platforms/php/vendor/phpunit/phpunit/src/Framework/TestSuite.php(703):

Apache\Ignite\Tests\SqlQueryTestCase::setUpBeforeClass()
#5


/home/isapego/work/ignite/modules/platforms/php/vendor/phpunit/phpunit/src/Framework/TestSuite.php(750):

PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#6


/home/isapego/work/ignite/modules/platforms/php/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(587):

PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#7


/home/isap

Re: PHP thin client

[Alexey Kosenchuk]

/Internal/Connection/ClientSocket.php(107):
Apache\Ignite\Internal\Connection\ClientSocket->processRequest(Object(Apache\Ignite\Internal\Binary\Request))
#3
/home/isapego/work/ignite/modules/platforms/php/src/Apache/Ignite/Internal/Connection/ClientFailoverSocket.php(96):
Apache\Ignite\Internal\Connection\ClientSocket->connect()
#4
/home/isapego/work/ignite/modules/platforms/php/src/Apache/Ignite/Internal/Connection/ClientFailoverSocket.php(54):
Apache\Ignite\Internal\Connection\ClientFailoverSocket->failoverConnect()
#5
/home/isapego/work/ignite/modules/platforms/php/src/Apache/Ignite/Client.php(61):
Apache\Ignite\Internal\Connection\ClientFailoverSocket->connect(Object(Apache\Ignite\ClientConfiguration))
#6
/home/isapego/work/ignite/modules/platforms/php/tests/TestingHelper.php(52):
Apache\Ignite\Client->connect(Object(Apache\Ignite\ClientConfiguration))
#7
/home/isapego/work/ignite/modules/platforms/php/tests/SqlQueryTest.php(49):
Apache\Ignite\Tests\TestingHelper::init()
#8
/home/isapego/work/ignite/modules/platforms/php/vendor/phpunit/phpunit/src/Framework/TestSuite.php(703):
Apache\Ignite\Tests\SqlQueryTestCase::setUpBeforeClass()
#9
/home/isapego/work/ignite/modules/platforms/php/vendor/phpunit/phpunit/src/Framework/TestSuite.php(750):
PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#10
/home/isapego/work/ignite/modules/platforms/php/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(587):
PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#11
/home/isapego/work/ignite/modules/platforms/php/vendor/phpunit/phpunit/src/TextUI/Command.php(203):
PHPUnit\TextUI\TestRunner->doRun(Object(PHPUnit\Framework\TestSuite),
Array, true)
#12
/home/isapego/work/ignite/modules/platforms/php/vendor/phpunit/phpunit/src/TextUI/Command.php(159):
PHPUnit\TextUI\Command->run(Array, true)
#13
/home/isapego/work/ignite/modules/platforms/php/vendor/phpunit/phpunit/phpunit(53):
PHPUnit\TextUI\Command::main()
#14 {main}
 Am I doing something wrong?

5. When "maven package" command is executed on Ignite, no php directory
appears
 in target/release-package-fabric/platforms directory. You need to edit
 ignite/assembly/release-fabric-base.xml file and add appropriate
"copy" commands.
 Search "cpp" or "dotnet" for example.

Best Regards,
Igor


On Tue, Sep 4, 2018 at 3:21 PM Igor Sapego  wrote:


Great,

I'll take a look

Best Regards,
Igor


On Wed, Aug 29, 2018 at 10:37 PM Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:


Hi folks,

PHP thin client is ready for review.

Jira with the scope of work - [1].

Implementation, examples, tests:
PR - [2],
repository - [3].

API spec - [4].

Readme (how to for the client, instructions for the examples and tests,
etc.) - [5].

Regards,
-Alexey

[1] https://issues.apache.org/jira/browse/IGNITE-7783
[2] https://github.com/apache/ignite/pull/4649
[3]

https://github.com/nobitlost/ignite/tree/ignite-7783/modules/platforms/php
[4]

https://rawgit.com/nobitlost/ignite/ignite-7783-docs/modules/platforms/php/api_docs/html/index.html
[5]

https://github.com/nobitlost/ignite/blob/ignite-7783-docs/modules/platforms/php/README.md

[jira] [Created] (IGNITE-9465) Node.js client: improve complex object flags processing

[Alexey Kosenchuk (JIRA)]

Alexey Kosenchuk created IGNITE-9465:


 Summary: Node.js client: improve complex object flags processing
 Key: IGNITE-9465
 URL: https://issues.apache.org/jira/browse/IGNITE-9465
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Reporter: Alexey Kosenchuk
Assignee: ekaterina.vergizova
 Fix For: 2.7


1) fix the issue in the full schema support

2) do not throw exception when object with HAS_RAW_DATA flag is received

3) support OFFSET_*_BYTE flags/optimizations when writing data



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

PHP thin client

[Alexey Kosenchuk]

Hi folks,

PHP thin client is ready for review.

Jira with the scope of work - [1].

Implementation, examples, tests:
  PR - [2],
  repository - [3].

API spec - [4].

Readme (how to for the client, instructions for the examples and tests, 
etc.) - [5].


Regards,
-Alexey

[1] https://issues.apache.org/jira/browse/IGNITE-7783
[2] https://github.com/apache/ignite/pull/4649
[3] 
https://github.com/nobitlost/ignite/tree/ignite-7783/modules/platforms/php
[4] 
https://rawgit.com/nobitlost/ignite/ignite-7783-docs/modules/platforms/php/api_docs/html/index.html
[5] 
https://github.com/nobitlost/ignite/blob/ignite-7783-docs/modules/platforms/php/README.md

NodeJS client: npm publishing instruction

[Alexey Kosenchuk]

Denis,

below is instruction how to publish/update npm module with Ignite NodeJS 
client on npmjs.

Use it when you decide to publish the client.

Another short instruction is how to generate the API spec using jsdoc.

Pls place the instructions to appropriate locations.

Thanks,
-Alexey


How to publish Ignite NodeJS Client on npmjs.com:
-
1. Install NodeJS npm (https://nodejs.org/en/), if not installed yet.
2. Register an account at npmjs (https://www.npmjs.com/signup), if not 
registered yet.

3. Execute `npm login` command and provide your npmjs account credentials.
4. Clone or download Ignite repository 
https://github.com/apache/ignite.git to `local_ignite_path`

5. Go to `local_ignite_path/modules/platforms/nodejs`
6. Prepare/update 
`local_ignite_path/modules/platforms/nodejs/package.json` file. Pay 
attention to:

  - "name" - name of the npm module
  - "version" - version of the npm module, increment it if you update 
the module

  - "description" - description of the npm module
  - "repository" - type and link to the repository with the source code
  - "keywords" - keywords for the search of the module on npmjs
  - "license" - license type
  - other properties depend on the implementation/tests, do not touch them
Example of the package.json file is attached.
7. Prepare/update `local_ignite_path/modules/platforms/nodejs/README.md` 
file. It should exist and should not be empty. Eg. add a link to a place 
with the documentation.
8. Execute `npm publish` command from the 
`local_ignite_path/modules/platforms/nodejs` folder.
9. Check the module is published and well-described at 
https://www.npmjs.com/package/apache-ignite-client (assuming 
`apache-ignite-client` is the name of the module)


Common instruction about npm publishing:
https://docs.npmjs.com/getting-started/publishing-npm-packages

How to generate Ignite NodeJS Client API specification:
---
It should be done if a public API class/method has been changed.
1. Execute `npm install -g jsdoc` to install jsdoc 
(https://www.npmjs.com/package/jsdoc)
2. Clone or download Ignite repository 
https://github.com/apache/ignite.git to `local_ignite_path`

3. Go to `local_ignite_path/modules/platforms/nodejs/api_spec`
4. Only if a class has been removed from the public API, remove all 
files from `local_ignite_path/modules/platforms/nodejs/api_spec` except 
conf.json file.

5. Execute `jsdoc -c conf.json` command.

Note: `local_ignite_path/modules/platforms/nodejs/api_spec/conf.json` is 
a file with jsdoc configuration.
{
  "name": "apache-ignite-client",
  "version": "1.0.0",
  "description": "NodeJS Client for Apache Ignite",
  "main": "index.js",
  "repository": {
"type": "git",
"url": 
"https://github.com/apache/ignite/tree/master/modules/platforms/nodejs;
  },
  "keywords": [
"apache",
"ignite",
"client"
  ],
  "author": "",
  "license": "Apache-2.0",
  "engines": {
"node": ">=8.0.0"
  },
  "dependencies": {
"long": "latest",
"decimal.js": "latest"
  },
  "scripts": {
"test": "jasmine",
"test:examples": "node ./spec/ExamplesExecutor.js Examples",
"test:auth_example": "node ./spec/ExamplesExecutor.js AuthExample"
  },
  "devDependencies": {
"jasmine": "latest",
"jasmine-expect": "latest",
"jasmine-reporters": "latest"
  }
}

Re: Thin clients features wiki page

[Alexey Kosenchuk]

Not applicable for NodeJS client as there are no threads in NodeJS (only 
processes) [1]


Just in case - of course, it's possible to run multiple 
queries/operations concurrently from one thread/process.
Eg. see Promise.all in CachePutGetExample.js and AuthTlsExample.js 
examples [2]


Is Java thin client really not thread-safe?

Is thread-safety a mandatory req for all clients (where it is applicable)?

-Alexey

[1] https://nodejs.org/en/about/
[2] 
https://github.com/nobitlost/ignite/tree/master/modules/platforms/nodejs/examples


16.05.2018 12:53, Igor Sapego пишет:

Fixed for .NET.

What about NodeJS? Alexey?

Best Regards,
Igor

On Wed, May 16, 2018 at 12:48 PM, Pavel Tupitsyn <ptupit...@apache.org>
wrote:


Ignite.NET client is thread-safe as stated in XMLDoc [1]

[1]
https://github.com/apache/ignite/blob/master/modules/
platforms/dotnet/Apache.Ignite.Core/Client/IIgniteClient.cs

On Wed, May 16, 2018 at 12:34 PM, Guru Stron <gurustronpub...@gmail.com>
wrote:


  Hi Igor,

Nice one!

On 16 May 2018 at 11:26, Igor Sapego <isap...@apache.org> wrote:


Pavel,

This is about concurrent usage of the same connection (client)
by several threads.

By the ways guys, I wanted to ask you, if your clients support this
feature, as I was not sure. I mean, protocol seems to be designed
to support such use-case, but I was not sure about clients.

Best Regards,
Igor

On Tue, May 15, 2018 at 10:46 PM, Pavel Tupitsyn <ptupit...@apache.org



wrote:


Hi Igor,

Thanks for creating this!
One question: what is `Multithreading` feature?

Pavel

On Tue, May 15, 2018 at 9:45 PM, Denis Magda <dma...@gridgain.com>

wrote:



Igor,

That's a valid point. Will see how it goes. As for the new

documentation

engine, there are no more arguments. We will try to migrate within

2.6.

Just need to find more contributors who would help with this.

--
Denis

On Tue, May 15, 2018 at 2:11 AM, Igor Sapego <isap...@apache.org>

wrote:



Thanks for proposals, guys, I will do this.

Denis,
Readme.io is not very convenient for some big tables. They look

OK,

while they have 3-4 columns, but if we are going to add more

clients

(or should I say "when") it is going to be a mess. This may be a

one

more argument when we consider to change documentation engine.


Best Regards,
Igor

On Tue, May 15, 2018 at 2:37 AM, Denis Magda <

dma...@gridgain.com>

wrote:



Igor, thanks for putting together the page.

As for the immediate improvements, I'm backing up Alexey's

suggestions.


This page deserves to be moved to the readme.io. We'll do that

once

it's

sorted out how to group the thin clients' documentation.

--
Denis

On Mon, May 14, 2018 at 12:39 PM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:


Hi Igor,

"Get Binary Cache" (if it is about getting a complex object in

a

"binary" form, w/o deserialization) - is supported by NodeJS:
BinaryObject is returned for a complex object if an app has not
specified another JavaScript Object for deserialization.

Maybe worth to add type registration functionality:
- complex object type registration (supported by NodeJS)
- enum type registration (not supported by NodeJS)
- type name registration (not supported by NodeJS)

Also, maybe add a list of all type codes, supported for read

and

write.

But not sure, as all of them are supported by all three

clients.


Thanks,
-Alexey

14.05.2018 22:06, Igor Sapego пишет:

Hello Igniters,


I've created a new page in wiki, that summarizes a features
availability of thin clients. It should help us to plan

development

of thin clients while also helping out users to understand a

state

of clients.

Check it out and tell me what you think. And if there are some
mistakes, don't hesitate telling me.

[1] -
https://cwiki.apache.org/confluence/display/IGNITE/Thin+clie
nts+features

Best Regards,
Igor

Re: NodeJS thin client: full API

[Alexey Kosenchuk]

- Let's rename SqlDataProcessingExample.js to just SqlExample.js and
describe it as an example that shows primary APIs to use with Ignite as
with an SQL database.
- Let's rename SqlQueryExample.js to SqlQueryEntiriesExample.js.


Updated.

Re: NodeJS thin client: full API

[Alexey Kosenchuk]

Denis,

OK for the legacy docs.

Agree for the core docs (specs, common getting started, etc.).

But who/what does mandate that for totally all Ignite related docs?

Why do not follow an approach which many technologies/products follow? - 
maintain a centralized list of references to different client libs/SDKs 
(sometimes even alternative libs for the same language/platform).
It is possible to mention a "status" of every lib - is it "verified" 
(fully tested, fully documented, etc.) or just a prototype...
And it is not mandated that every lib must be developed/released under 
Apache...


The list could already have references to:
- java client
- .net client
- node.js client (as early access)
- Go client by Aleksandr S. (as prototype)
- Python client by Sergey K. (as prototype)

With this approach the docs naturally come together with a concrete 
client, but may be cross-referenced on readme.io for some clients as well...


-Alexey

12.05.2018 6:37, Denis Magda пишет:

Alexey,

Presently, Ignite hosts all the docs in readme.io without exception. It
means that once your contribution is accepted by the community the Node.JS
docs should be placed on readme.io.

You're right saying that we're planning to migrate from readme.io to
another documentation engine that would allow us storing doc sources in
Ignite repo. It might happen by 2.6 or might take longer.

Thus, we need to host the Node.JS docs on readme.io and edit them there
once your pull-request is merged (it means there wouldn't be docs' copy
added to Ignite repo for now). It's easy to move the docs to readme.io
which understands the standard markdown. I'll ask Prachi to assist here.

--
Denis

On Fri, May 11, 2018 at 1:45 PM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:


Denis,


As for the docs, are you ready to bring them to readme.io? Just let me

know

and I'll be happy to arrange an account for you and discuss the

structure.

I remember some discussion regarding moving the docs from readme.io to
GitHub pages in 2.6.
No?

In any case, in my opinion, a readme near the code is a right primary
place for the docs for thin clients.
Is there any script/automation to convert .md to readme.io?
Or maybe just place a link from the readme.io to the repo readme?
Manual support of the same docs in two places seems not an effective
solution.

The docs for NodeJS client is ready for review in the repo.
The links and the installation procedure will have to be updated when the
client is integrated into the apache repo and released on npmjs.

-Alexey

Re: NodeJS thin client: full API

[Alexey Kosenchuk]

Denis,

> As for the docs, are you ready to bring them to readme.io? Just let 
me know
> and I'll be happy to arrange an account for you and discuss the 
structure.


I remember some discussion regarding moving the docs from readme.io to 
GitHub pages in 2.6.

No?

In any case, in my opinion, a readme near the code is a right primary 
place for the docs for thin clients.

Is there any script/automation to convert .md to readme.io?
Or maybe just place a link from the readme.io to the repo readme?
Manual support of the same docs in two places seems not an effective 
solution.


The docs for NodeJS client is ready for review in the repo.
The links and the installation procedure will have to be updated when 
the client is integrated into the apache repo and released on npmjs.


-Alexey

Re: NodeJS thin client: full API

[Alexey Kosenchuk]

Hi Igor,

all types from the protocol v.2.4 are supported for both read and write.
See [1] for the mappings and explanation.

The idea is to minimize additional non-standard and/or 3rd-party 
JavaScript Objects for that.


Timestamp and Enum require additional classes - no better options were 
found. So, Timestamp and EnumItem classes have been introduced.


For Decimal - a 3rd-party Decimal type [2] seems the best choice and is 
utilized.


For all other Ignite types - only standard JavaScript types and Objects 
are used.

Eg. for UUID - it is simply an array of numbers (16-elements).

Regards,
-Alexey

[1] 
https://rawgit.com/nobitlost/ignite/master/modules/platforms/nodejs/api_spec/ObjectType.html

[2] https://github.com/MikeMcl/decimal.js


11.05.2018 19:38, Igor Sapego пишет:

Alexey,

I've checked out the code. Looks good to me. Great job!

What about data types support? I can see Timestamp.
Are you planning to implement other types, e.g. Decimal,
Guid?

Best Regards,
Igor

On Fri, May 11, 2018 at 11:43 AM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:


On Fri, May 11, 2018 at 9:14 AM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:


Not yet. Need a help with that.



I think we definitely need a load test before we merge to master. Can
anyone in the community assist Alexey?

Re: NodeJS thin client: full API

[Alexey Kosenchuk]

Not yet. Need a help with that.

-Alexey

11.05.2018 10:58, Dmitriy Setrakyan пишет:

This is great! Finally a native NodeJS client for Ignite.

Alexey, in addition to the functional tests, were you able to perform any
load tests?

D.

On Fri, May 11, 2018 at 12:07 AM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:


Folks,

The next version is ready -
in the pull request [1] or directly in the repo [2].

The version includes:
- full API
- full implementation
- examples
- tests (cover the full API but might need to be updated/extended)
- docs

The details are in the readme [3]

Regards,
-Alexey

[1] https://github.com/apache/ignite/pull/3978
[2] https://github.com/nobitlost/ignite/tree/master/modules/plat
forms/nodejs
[3] https://github.com/nobitlost/ignite/blob/master/modules/plat
forms/nodejs/README.md

NodeJS thin client: full API

[Alexey Kosenchuk]

Folks,

The next version is ready -
in the pull request [1] or directly in the repo [2].

The version includes:
- full API
- full implementation
- examples
- tests (cover the full API but might need to be updated/extended)
- docs

The details are in the readme [3]

Regards,
-Alexey

[1] https://github.com/apache/ignite/pull/3978
[2] https://github.com/nobitlost/ignite/tree/master/modules/platforms/nodejs
[3] 
https://github.com/nobitlost/ignite/blob/master/modules/platforms/nodejs/README.md

Re: Node.js client update: rev. 2

[Alexey Kosenchuk]

Hi Igor,

in the newer version (will be available soon) getValues() is changed to 
getValue() which gives sequential per-element access.


The API spec and the setup/run instructions are available with every 
version.


Thanks,
-Alexey

08.05.2018 15:14, Igor Sapego пишет:

Hi, Pavel,

I took a look, and everything looks OK to me (except for the
lack of documentation, but I guess, this is expected for the
early version).

I have a question regardless Cursor class though. Other thin
clients, as far as I know, implement Cursor as a simple
iterator, which gives sequential per-element access, while in
your implementation there is only getAll() and getValues()
methods. The last method returns a whole bunch of values
as an array of unknown size. Are there any reasons from
JS point of view for such architecture?


Best Regards,
Igor

On Fri, Apr 13, 2018 at 2:37 AM, Pavel Petroshenko 
wrote:


Hi Denis,

Thank you for looking at the proposal and raising these questions! We'll
take them to discuss offline and will address your recommendations in the
next revision.

I will let Vladimir comment on the QueryEntity here though to make sure we
are on the same page.

Thanks,

P.


On Thu, Apr 12, 2018 at 2:48 PM, Denis Magda  wrote:


Hello Pavel,

Thanks for the update. Haven't heard from you guys for a while but seems
you were too busy polishing the client. Looks great!

Please consider several questions/notes:

- Do we really want to migrate QueryEntity API [1] to Node.JS client?

I

heard we planned to deprecate it in the future. *Vladimir*, please

share

your thoughts.
- I wouldn't mix key-value and SQL examples together because they
represent different use cases of Ignite. For the sake of simplicity, I
would create a single SqlExample file that uses DDL to configure
tables/cache, inserts data with INSERTS and SELECTs it back (simple
queries
and queries with JOINs). We can base it on the database and queries

used

here [2]. Happy to brainstorm on this with you separately.
-


[1]
https://github.com/nobitlost/ignite/blob/master/modules/
clients/nodejs/examples/SqlQueryExample.js#L68
[2] https://apacheignite-sql.readme.io/docs/getting-started

--
Denis

On Wed, Apr 11, 2018 at 6:45 PM, Pavel Petroshenko <

pa...@petroshenko.com>

wrote:


Igniters,

Just to give you an update on the next iteration of the Ignite Node.js

thin

client implementation.

The second iteration is available for review/testing.

The changes are available in the pull request [1] or directly from our
repository [2].

The short README file [3] covers:

- the list of supported features
- simple instructions:
   * how to install the client
   * how to run the examples
   * how to run the tests

And we actually encourage you to give it a look or even a try.

The APIs are available both: in sources [4] and in a form of a

generated

specification, produced automatically from the jsdoc comments [5]

Please let us know if you have any questions.

Thanks!

P.

[1] https://github.com/apache/ignite/pull/3680
[2] https://github.com/nobitlost/ignite/tree/master/modules/

clients/nodejs

[3] https://github.com/nobitlost/ignite/blob/master/
modules/clients/nodejs/README.md
[4] https://github.com/nobitlost/ignite/blob/master/
modules/clients/nodejs/lib
[5] https://rawgit.com/nobitlost/ignite/master/modules/clients/

nodejs/api_

spec/index.html

Re: Apache Ignite 2.4+ Go language client

[Alexey Kosenchuk]

Hi Aleksandr,

> 1) Is SSL\TLS supported for binary client protocol?

It has no direct relation to the binary client protocol itself.
Just need to establish https connection.
But you need to setup Ignite to accept TLS (it's not trivial).
And generate certificates and keys.

Example of the Ignite cfg:
https://github.com/apache/ignite/blob/master/modules/platforms/cpp/odbc-test/config/queries-ssl.xml
Example of the SSL certificates and keys:
https://github.com/apache/ignite/tree/master/modules/platforms/cpp/odbc-test/config/ssl
Test for Java client:
https://github.com/apache/ignite/blob/master/modules/indexing/src/test/java/org/apache/ignite/client/SecurityTest.java#L57

> 2) Are authentication and authorization supported for binary client
> protocol?

Seems it's not documented yet.
But yes, it's supported since protocol version 1.1.0.

Handshake format:
[original handshake][username:String][password:String]

Example of implementation:
https://github.com/nobitlost/ignite/blob/master/modules/clients/nodejs/lib/internal/ClientSocket.js
method _handshakePayloadWriter()

In Ignite you need to switch authentication on by the flag 
IgniteConfiguration.authenticationEnabled


Example of the Ignite cfg:
https://github.com/apache/ignite/blob/master/modules/platforms/cpp/odbc-test/config/queries-auth.xml

Default user (username/password) : ignite/ignite

Regards,
-Alexey

30.04.2018 17:49, Aleksandr Sokolovskii пишет:

Hi Alexey,

Do you have info about the following:

1) Is SSL\TLS supported for binary client protocol?

2) Are authentication and authorization supported for binary client 
protocol?


Thanks,

Aleksandr

*From: *Aleksandr Sokolovskii <mailto:amso...@gmail.com>
*Sent: *25 апреля 2018 г. 16:37
*To: *dev@ignite.apache.org <mailto:dev@ignite.apache.org>
*Subject: *RE: Apache Ignite 2.4+ Go language client

Yes, it’s really challenge to write unit test for OP_QUERY_SQL. )))

Thanks,
Aleksandr

*From: *Alexey Kosenchuk <mailto:alexey.kosenc...@nobitlost.com>
*Sent: *25 апреля 2018 г. 16:33
*To: *dev@ignite.apache.org <mailto:dev@ignite.apache.org>
*Subject: *Re: Apache Ignite 2.4+ Go language client

No problem with OP_QUERY_SQL_FIELDS.

I mean OP_QUERY_SQL.

But I overlooked "OP_QUERY_SQL - Done without unit test. Need help to

develop unit test."

Sorry. I remove my question so far)))

Thanks,

-Alexey

25.04.2018 15:40, Aleksandr Sokolovskii пишет:

 > Hi Alexey,

 >

 >> Seems an important missed part is complex objects support.

 >> And it's the most difficult part.

 > Yes, you are absolutely right.

 > Complex object and binary types are not supported by my client now.

 > That’s why I’m here.

 > I plan to come back to this subject in the beginning of May.

 > I very much count on the community help.

 >

 >> Also, do you have a working example for SQL Query which first inserts

 >> cache entries and then selects them and successfully get the same values

 >> which were inserted ?

 > Yes, it works fine.

 > This one:

 > 
https://github.com/amsokol/ignite-go-client/blob/master/binary/v1/client-sql-and-scan-queries_test.go#L89


 > It’s OP_QUERY_SQL_FIELDS operation unit test.

 > It you are familiar with Go it’s easy to run in debug mode.

 >

 > If you explain me your problem I can try to help you.

 > There some mistakes in binary protocol documentation.

 > So I spent a lot of time in Apache Ignite server source code 
analyzing )))


 >

 > Thanks,

 > Aleksandr

 >

 > From: Alexey Kosenchuk

 > Sent: 25 апреля 2018 г. 12:05

 > To: dev@ignite.apache.org

 > Subject: Re: Apache Ignite 2.4+ Go language client

 >

 > Hi Aleksandr,

 >

 > Seems an important missed part is complex objects support.

 > And it's the most difficult part.

 > Actually, it is not possible to implement it correctly basing on the

 > current protocol spec only.

 > If you decide to support it, you should either wait for an updated spec

 > or ask for help.

 >

 > The issues against the current protocol spec are summarized in

 > IGNITE-8039 [1] and IGNITE-8212 [2]. One more jira with the issues

 > against the rest of the spec will be submitted soon.

 > If you see more issues, please feel free to added them as comments to

 > that jiras.

 >

 > Also, do you have a working example for SQL Query which first inserts

 > cache entries and then selects them and successfully get the same values

 > which were inserted ?

 >

 > Thanks,

 > -Alexey

 >

 > [1] https://issues.apache.org/jira/browse/IGNITE-8039

 > [2] https://issues.apache.org/jira/browse/IGNITE-8212

 >

 > 25.04.2018 0:11, Aleksandr Sokolovskii пишет:

 >> ...forgot OP_RESOURCE_CLOSE that is implemented also.

 >>

 >> Thanks,

 >> Aleksandr

 >>

 >> Thanks,

 >> Aleksandr

 >>

 &

[jira] [Created] (IGNITE-8411) Binary Client Protocol spec: other parts clarifications

[Alexey Kosenchuk (JIRA)]

Alexey Kosenchuk created IGNITE-8411:


 Summary: Binary Client Protocol spec: other parts clarifications
 Key: IGNITE-8411
 URL: https://issues.apache.org/jira/browse/IGNITE-8411
 Project: Ignite
  Issue Type: Bug
  Components: documentation, thin client
Affects Versions: 2.4
Reporter: Alexey Kosenchuk


Cache Configuration
 ---
 
[https://apacheignite.readme.io/docs/binary-client-protocol-cache-configuration-operations]
 - OP_CACHE_GET_CONFIGURATION and OP_CACHE_CREATE_WITH_CONFIGURATION - 
QueryEntity - Structure of QueryField:
 absent "default value - type Object" - it is the last field of the QueryField 
in reality.

 - OP_CACHE_GET_CONFIGURATION - Structure of Cache Configuration:
 Absent CacheAtomicityMode - is the first field in reality.
 Absent MaxConcurrentAsyncOperations - is between DefaultLockTimeout and 
MaxQueryIterators in reality.
 "Invalidate" field - does not exist in reality.

 - meaning and possible values of every configuration parameter must be 
clarified. If clarified in other docs, this spec must have link(s) to that docs.

 - suggest to combine somehow Cache Configuration descriptions in 
OP_CACHE_GET_CONFIGURATION and OP_CACHE_CREATE_WITH_CONFIGURATION - to avoid 
duplicated descriptions.

 

SQL and Scan Queries
 
 [https://apacheignite.readme.io/docs/binary-client-protocol-sql-operations]
 - "Flag. Pass 0 for default, or 1 to keep the value in binary form.":
 "the value in binary form" flag should be left end clarified in the operations 
to which it is applicable for.

 - OP_QUERY_SQL:
 most of the fields in the request must be clarified. If clarified in other 
docs, this spec must have link(s) to that docs.
 For example:
 ** "Name of a type or SQL table": name of what type?

 - OP_QUERY_SQL_FIELDS:
 most of the fields in the request must be clarified. If clarified in other 
docs, this spec must have link(s) to that docs.
 For example:
 ** is there any correlation between "Query cursor page size" and "Max rows"?
 ** "Statement type": why there are only three types? what about INSERT, etc.?

 - OP_QUERY_SQL_FIELDS_CURSOR_GET_PAGE Response does not contain Cursor id. But 
responses for all other query operations contain it. Is it intentional?

 - OP_QUERY_SCAN_CURSOR_GET_PAGE Response - Row count field: says type "long". 
But Row count field in responses for all other query operations is "int".

 - OP_QUERY_SCAN:
 format and rules of the Filter object must be clarified. If clarified in other 
docs, this spec must have link(s) to that docs.

 - OP_QUERY_SCAN:
 in general, it's not clear how this operation should be supported on platforms 
other than the mentioned in "Filter platform" field.

 

Binary Types
 
 
[https://apacheignite.readme.io/docs/binary-client-protocol-binary-type-operations]
 - somewhere should be explained when and why these operations need to be 
supported by a client.

 - Type id and Field id:
 should be clarified that before an Id calculation Type and Field names must be 
updated to low case.

 - OP_GET_BINARY_TYPE and OP_PUT_BINARY_TYPE - BinaryField - Type id:
 in reality it is not a type id (hash code) but a type code (1, 2,... 10,... 
103,...).

 - OP_GET_BINARY_TYPE and OP_PUT_BINARY_TYPE - "Affinity key field name":
 should be explained what is it. If explained in other docs, this spec must 
have link(s) to that docs.

 - OP_PUT_BINARY_TYPE - schema id:
 mandatory algorithm of schema Id calculation must be described somewhere. If 
described in other docs, this spec must have link(s) to that docs.

 - OP_REGISTER_BINARY_TYPE_NAME and OP_GET_BINARY_TYPE_NAME:
 should be explained when and why these operations need to be supported by a 
client.
 How this operation should be supported on platforms other than the mentioned 
in "Platform id" field.

 - OP_REGISTER_BINARY_TYPE_NAME:
 Type name - is it "full" or "short" name here?
 Type id - is it a hash from "full" or "short" name here?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Apache Ignite 2.4+ Go language client

[Alexey Kosenchuk]

No problem with OP_QUERY_SQL_FIELDS.
I mean OP_QUERY_SQL.
But I overlooked "OP_QUERY_SQL - Done without unit test. Need help to 
develop unit test."

Sorry. I remove my question so far)))

Thanks,
-Alexey

25.04.2018 15:40, Aleksandr Sokolovskii пишет:

Hi Alexey,


Seems an important missed part is complex objects support.
And it's the most difficult part.

Yes, you are absolutely right.
Complex object and binary types are not supported by my client now.
That’s why I’m here.
I plan to come back to this subject in the beginning of May.
I very much count on the community help.


Also, do you have a working example for SQL Query which first inserts
cache entries and then selects them and successfully get the same values
which were inserted ?

Yes, it works fine.
This one:
https://github.com/amsokol/ignite-go-client/blob/master/binary/v1/client-sql-and-scan-queries_test.go#L89
It’s OP_QUERY_SQL_FIELDS operation unit test.
It you are familiar with Go it’s easy to run in debug mode.

If you explain me your problem I can try to help you.
There some mistakes in binary protocol documentation.
So I spent a lot of time in Apache Ignite server source code analyzing )))

Thanks,
Aleksandr

From: Alexey Kosenchuk
Sent: 25 апреля 2018 г. 12:05
To: dev@ignite.apache.org
Subject: Re: Apache Ignite 2.4+ Go language client

Hi Aleksandr,

Seems an important missed part is complex objects support.
And it's the most difficult part.
Actually, it is not possible to implement it correctly basing on the
current protocol spec only.
If you decide to support it, you should either wait for an updated spec
or ask for help.

The issues against the current protocol spec are summarized in
IGNITE-8039 [1] and IGNITE-8212 [2]. One more jira with the issues
against the rest of the spec will be submitted soon.
If you see more issues, please feel free to added them as comments to
that jiras.

Also, do you have a working example for SQL Query which first inserts
cache entries and then selects them and successfully get the same values
which were inserted ?

Thanks,
-Alexey

[1] https://issues.apache.org/jira/browse/IGNITE-8039
[2] https://issues.apache.org/jira/browse/IGNITE-8212

25.04.2018 0:11, Aleksandr Sokolovskii пишет:

...forgot OP_RESOURCE_CLOSE that is implemented also.

Thanks,
Aleksandr

Thanks,
Aleksandr

On 25 April 2018 at 00:06, Aleksandr Sokolovskii <amso...@gmail.com> wrote:


Hello All,

Besides the SQL driver I implemented the following operations also:
OP_CACHE_GET_NAMES
OP_CACHE_CREATE_WITH_NAME
OP_CACHE_GET_OR_CREATE_WITH_NAME
OP_CACHE_CREATE_WITH_CONFIGURATION
OP_CACHE_GET_OR_CREATE_WITH_CONFIGURATION
OP_CACHE_GET_CONFIGURATION
OP_CACHE_DESTROY
OP_CACHE_GET
OP_CACHE_PUT
OP_CACHE_PUT_IF_ABSENT
OP_CACHE_GET_ALL
OP_CACHE_PUT_ALL
OP_CACHE_GET_AND_PUT
OP_CACHE_GET_AND_REPLACE
OP_CACHE_GET_AND_REMOVE
OP_CACHE_GET_AND_PUT_IF_ABSENT
OP_CACHE_REPLACE
OP_CACHE_REPLACE_IF_EQUALS
OP_CACHE_CONTAINS_KEY
OP_CACHE_CONTAINS_KEYS
OP_CACHE_CLEAR
OP_CACHE_CLEAR_KEY
OP_CACHE_CLEAR_KEYS
OP_CACHE_REMOVE_KEY
OP_CACHE_REMOVE_IF_EQUALS
OP_CACHE_REMOVE_KEYS
OP_CACHE_REMOVE_ALL
OP_CACHE_GET_SIZE
OP_QUERY_SQL
OP_QUERY_SQL_CURSOR_GET_PAGE
OP_QUERY_SQL_FIELDS
OP_QUERY_SQL_FIELDS_CURSOR_GET_PAGE

Look at the Client interface here:
https://github.com/amsokol/ignite-go-client/blob/master/
binary/v1/client.go

But not all Apache Ignite types are supported now.
See README in https://github.com/amsokol/ignite-go-client for details.

Thanks,
Aleksandr



Thanks,
Aleksandr

On 24 April 2018 at 13:16, Igor Sapego <isap...@apache.org> wrote:


Aleksandr,

Great job! Do you have any plans on adding new features to
your client?

Pavel,

There are  also CacheGet and CachePut [1] operations, as
far as I can see.

[1] -
https://github.com/amsokol/ignite-go-client/blob/master/bina
ry/v1/client.go#L120

Best Regards,
Igor

On Tue, Apr 24, 2018 at 10:14 AM, Dmitriy Setrakyan <
dsetrak...@apache.org>
wrote:


Any chance we can add key-value support as well?

On Tue, Apr 24, 2018, 2:48 PM Pavel Tupitsyn <ptupit...@apache.org>

wrote:



Hi Aleksandr,

This is awesome, thank you!

However, let's make it clear that this client supports SQL only,
and none of the other Thin Client protocol features.

Pavel

On Mon, Apr 23, 2018 at 10:41 PM, Aleksandr Sokolovskii <

amso...@gmail.com



wrote:


Hi Oleg,

Thanks for your answer.


Community is currently working on formal test specification.

Great. Waiting for this one.


As far as NodeJS please note that it is already being developed by

community at the moment [1].
Cool. I stop my initiatives.

Thanks,
Aleksandr

From: Vladimir Ozerov
Sent: 23 апреля 2018 г. 22:35
To: dev@ignite.apache.org
Subject: Re: Apache Ignite 2.4+ Go language client

Hi Alexander,

Awesome thing! Please note that before accepting the client we need

to

make

sure it is operational. Community is currently working on formal

test

specification. I hope it will be ready soon.

As far as NodeJS please no

Re: Apache Ignite 2.4+ Go language client

[Alexey Kosenchuk]

Hi Aleksandr,

Seems an important missed part is complex objects support.
And it's the most difficult part.
Actually, it is not possible to implement it correctly basing on the 
current protocol spec only.
If you decide to support it, you should either wait for an updated spec 
or ask for help.


The issues against the current protocol spec are summarized in 
IGNITE-8039 [1] and IGNITE-8212 [2]. One more jira with the issues 
against the rest of the spec will be submitted soon.
If you see more issues, please feel free to added them as comments to 
that jiras.


Also, do you have a working example for SQL Query which first inserts 
cache entries and then selects them and successfully get the same values 
which were inserted ?


Thanks,
-Alexey

[1] https://issues.apache.org/jira/browse/IGNITE-8039
[2] https://issues.apache.org/jira/browse/IGNITE-8212

25.04.2018 0:11, Aleksandr Sokolovskii пишет:

...forgot OP_RESOURCE_CLOSE that is implemented also.

Thanks,
Aleksandr

Thanks,
Aleksandr

On 25 April 2018 at 00:06, Aleksandr Sokolovskii  wrote:


Hello All,

Besides the SQL driver I implemented the following operations also:
OP_CACHE_GET_NAMES
OP_CACHE_CREATE_WITH_NAME
OP_CACHE_GET_OR_CREATE_WITH_NAME
OP_CACHE_CREATE_WITH_CONFIGURATION
OP_CACHE_GET_OR_CREATE_WITH_CONFIGURATION
OP_CACHE_GET_CONFIGURATION
OP_CACHE_DESTROY
OP_CACHE_GET
OP_CACHE_PUT
OP_CACHE_PUT_IF_ABSENT
OP_CACHE_GET_ALL
OP_CACHE_PUT_ALL
OP_CACHE_GET_AND_PUT
OP_CACHE_GET_AND_REPLACE
OP_CACHE_GET_AND_REMOVE
OP_CACHE_GET_AND_PUT_IF_ABSENT
OP_CACHE_REPLACE
OP_CACHE_REPLACE_IF_EQUALS
OP_CACHE_CONTAINS_KEY
OP_CACHE_CONTAINS_KEYS
OP_CACHE_CLEAR
OP_CACHE_CLEAR_KEY
OP_CACHE_CLEAR_KEYS
OP_CACHE_REMOVE_KEY
OP_CACHE_REMOVE_IF_EQUALS
OP_CACHE_REMOVE_KEYS
OP_CACHE_REMOVE_ALL
OP_CACHE_GET_SIZE
OP_QUERY_SQL
OP_QUERY_SQL_CURSOR_GET_PAGE
OP_QUERY_SQL_FIELDS
OP_QUERY_SQL_FIELDS_CURSOR_GET_PAGE

Look at the Client interface here:
https://github.com/amsokol/ignite-go-client/blob/master/
binary/v1/client.go

But not all Apache Ignite types are supported now.
See README in https://github.com/amsokol/ignite-go-client for details.

Thanks,
Aleksandr



Thanks,
Aleksandr

On 24 April 2018 at 13:16, Igor Sapego  wrote:


Aleksandr,

Great job! Do you have any plans on adding new features to
your client?

Pavel,

There are  also CacheGet and CachePut [1] operations, as
far as I can see.

[1] -
https://github.com/amsokol/ignite-go-client/blob/master/bina
ry/v1/client.go#L120

Best Regards,
Igor

On Tue, Apr 24, 2018 at 10:14 AM, Dmitriy Setrakyan <
dsetrak...@apache.org>
wrote:


Any chance we can add key-value support as well?

On Tue, Apr 24, 2018, 2:48 PM Pavel Tupitsyn 

wrote:



Hi Aleksandr,

This is awesome, thank you!

However, let's make it clear that this client supports SQL only,
and none of the other Thin Client protocol features.

Pavel

On Mon, Apr 23, 2018 at 10:41 PM, Aleksandr Sokolovskii <

amso...@gmail.com



wrote:


Hi Oleg,

Thanks for your answer.


Community is currently working on formal test specification.

Great. Waiting for this one.


As far as NodeJS please note that it is already being developed by

community at the moment [1].
Cool. I stop my initiatives.

Thanks,
Aleksandr

From: Vladimir Ozerov
Sent: 23 апреля 2018 г. 22:35
To: dev@ignite.apache.org
Subject: Re: Apache Ignite 2.4+ Go language client

Hi Alexander,

Awesome thing! Please note that before accepting the client we need

to

make

sure it is operational. Community is currently working on formal

test

specification. I hope it will be ready soon.

As far as NodeJS please note that it is already being developed by
community at the moment [1]. We hope to have it in Apache Ignite

2.6.


[1]
https://issues.apache.org/jira/browse/IGNITE-

пн, 23 апр. 2018 г. в 22:24, Aleksandr Sokolovskii <

amso...@gmail.com

:



Hi All,

I hope you are well.

I released Apache Ignite 2.4+ Go language client:
https://github.com/apache-ignite/go-client

I updated link here:
https://github.com/golang/go/wiki/SQLDrivers

Is it possible to add link to my repo to this page?:
https://apacheignite.readme.io/docs/binary-client-protocol
or this page:
https://apacheignite-net.readme.io/docs/thin-client
Golang is much more easy to understand than Java or С#.
It’s very easy to pull, build and run test for my library.
I believe it helps another guys to write more thin clients.

P.S.: I started developing Node.js client also.

Thanks,
Aleksandr

[jira] [Created] (IGNITE-8212) Binary Client Protocol spec: Key-Value Queries clarifications

[Alexey Kosenchuk (JIRA)]

Alexey Kosenchuk created IGNITE-8212:


 Summary: Binary Client Protocol spec: Key-Value Queries 
clarifications
 Key: IGNITE-8212
 URL: https://issues.apache.org/jira/browse/IGNITE-8212
 Project: Ignite
  Issue Type: Bug
  Components: documentation, thin client
Affects Versions: 2.4
Reporter: Alexey Kosenchuk


https://apacheignite.readme.io/v2.4/docs/binary-client-protocol-key-value-operations

- all operations - "Flag. Pass 0 for default, or 1 to keep the value in binary 
form":
-- remove words about binary form and keep "Pass 0 for default" for all 
operations where this flag has no meaning.
-- clarify about binary form in operations where it has a meaning.

- OP_CACHE_GET: clarify that null is returned if the provided key does not 
exist in the cache.

- OP_CACHE_GET_AND_PUT: clarify that new entry is created, if the provided key 
does not exist in the cache, and null is returned in this case.

- OP_CACHE_GET_AND_REPLACE:
-- "if and only if there is a value currently mapped for that key" - is 
confusing. Is it possible that an existing key has no associated value? Suggest 
to rephrase as "if and only if the key exists in the cache"
-- clarify that null is returned if the key does not exist in the cache.

- OP_CACHE_GET_AND_REMOVE: clarify that null is returned if the key does not 
exist in the cache.

- OP_CACHE_PUT_IF_ABSENT: clarify what is "Success Flag" - true if the new 
entry is created, false if the specified key already exists in the cache.

- OP_CACHE_GET_AND_PUT_IF_ABSENT: "Previously contained value regardless of 
whether put happened or not" - is confusing. Suggest to rewrite "the current 
value associated with the key if it already exists in the cache, null if the 
new entry is created".

- OP_CACHE_GET_SIZE: clarify Peek modes

- OP_CACHE_CLEAR_XXX and OP_CACHE_REMOVE_XXX:
-- what is the difference between the corresponding operations? Only in 
notifying / not notifying listeners and cache writers? Or there are other 
differences not clarified by the spec?
-- (the operations could be combined in one set - with a flag required/not 
required notifications)
-- there is OP_CACHE_REMOVE_IF_EQUALS. But there is no 
OP_CACHE_CLEAR_IF_EQUALS. Is it intentional?
-- OP_CACHE_REMOVE_KEY and OP_CACHE_REMOVE_IF_EQUALS have "Value indicating 
whether remove happened" in the response. But OP_CACHE_CLEAR_KEY does not have 
such a flag in the response. Is it intentional?
-- OP_CACHE_CLEAR_KEYS, OP_CACHE_REMOVE_KEYS: clarify that even if some of the 
specified keys do not exist other entries are removed anyway.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8039) Binary Client Protocol spec: data types/format clarifications

[Alexey Kosenchuk (JIRA)]

Alexey Kosenchuk created IGNITE-8039:


 Summary: Binary Client Protocol spec: data types/format 
clarifications
 Key: IGNITE-8039
 URL: https://issues.apache.org/jira/browse/IGNITE-8039
 Project: Ignite
  Issue Type: Bug
  Components: documentation, thin client
Affects Versions: 2.4
Reporter: Alexey Kosenchuk


Assuming the Binary Client Protocol spec should be detalized enough to allow a 
client development basing on the spec only, w/o looking at other client 
implementations and asking additional questions...

The following should be clarified / corrected in the Binary Client Protocol 
spec (v.2.4) 
(https://apacheignite.readme.io/v2.4/docs/binary-client-protocol#section-data-format):

Type Codes table:
-

- UUID (Guid) size: should be 16 bytes, not 8 (?) 

- what is Object array (type code 23) ? What is the difference between it and 
Objects Wrapped In​ Array (type code 27) ?

- what is Collection USER_SET ?

- what is Collection USER_COL ?

- what is Collection SINGLETON_LIST ?

- Collection: misprint: should be "... + length ..."

- what is Decimal ?

- what is Timestamp ?

- what is Time ?

Complex Objects:


- what does flag USER_TYPE mean ?

- Schema "field Id; Java-style hash code of field" -> should be "... of field 
name".

- "Repeat for as many times as the total number of schemas you have" -> should 
be "... total number of fields you have".

- is it mandated that the number of fields in the Schema must be equal to the 
number of fields in the Data Object ?

Objects Wrapped In​ Array


- may binary objects with different type codes be in the same array ?

- may complex objects with different type ids be in the same array ?

- "All cache operations return complex objects inside a wrapper (but not 
primitives)." -> does it mean that in general a complex object (103) must 
always be sent via the Binary Protocol in a wrapper (27)? 

- "Byte array size" -> "Payload size" or "Size of the whole array with header" ?

- Offset. What is "object graph" here ? The Binary Protocol nowhere describes 
any relations ("graph") between data objects in the protocol.

Terminology
---

Not critical but would be really convenient to define and use the same terms 
along the whole spec. For example:

- "binary object" is always the same as "data object" of any type (?). Can be 
"standard/predefined type object" or "complex object".

- "cluster" or "server" ?

- "cluster member" or "server nodes" ?




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8014) Node.js client: basic/minimal version

[Alexey Kosenchuk (JIRA)]

Alexey Kosenchuk created IGNITE-8014:


 Summary: Node.js client: basic/minimal version
 Key: IGNITE-8014
 URL: https://issues.apache.org/jira/browse/IGNITE-8014
 Project: Ignite
  Issue Type: Sub-task
  Components: thin client
Reporter: Alexey Kosenchuk
Assignee: Alexey Kosenchuk


Develop the first basic/minimal version - for initial review/feedback.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Thin client / Binary protocol: questions

[Alexey Kosenchuk]

Folks,

Designing node.js thin lib (IGNITE-) we have the following questions.
Could you please have a look and clarify.

Thanks,
-Alexey

- Supported standard types and their type codes.
  Are defined in the spec: 
https://apacheignite.readme.io/docs/binary-client-protocol#data-format
  But, as we see, java and .net libs (common parts which are used by 
thin clients as well) support additional types and type codes, not 
described in the spec. Eg. defined here: 
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/binary/GridBinaryMarshaller.java
  Just to double-check - we should support the types from the spec 
only. Right?


- Key-Value and SQL and Scan Queries.
  The most of operations has "Flag" field in Request: "Pass 0 for 
default, or 1 to keep the value in binary form."

  What is it for?
  Java and .net libs always pass 0. Why?

- OP_QUERY_SCAN: Filter platform.
  Why is it required?
  Why an app working remotely via the binary protocol should know the 
exact platform where the node is running?


- Binary Type operations (register/get type name, put/get type).
  What are they for?
  What is a use case?
  It seems possible to put data of any binary object type into a cache 
w/o registering type name and structure in advance.


- OP_REGISTER_BINARY_TYPE_NAME, OP_GET_BINARY_TYPE_NAME: Platform id.
  Is it a type of the node platform?
  Why a remote app should know it?
  Why non-identical values are used here and in OP_QUERY_SCAN: Filter 
platform?


- OP_REGISTER_BINARY_TYPE_NAME, OP_PUT_BINARY_TYPE
  Why the both - Type name and Java-style hash code of the type name - 
are in the request? (The same for Field name and Java-style hash code of 
the field name.)

  Seems redundant.

- Just interesting - why it was decided that hash code should be also 
calculated on a client side? Why it could not be returned by a server side?
  Eg. OP_CACHE_CREATE_WITH_NAME Response can return hash code for the 
provided Cache name.
  That would simplify thin client implementations, guarantee 
consistency/correctness of hash code calculations...


- Name/password authentication.
  When will it be available in the spec?
  Is there any draft we can look at?

- SSL/TLS communication.
  When will it be available in the spec?
  Is there any draft we can look at?

- As Denis said, we should implement a node failover algorithm 
(switching to another predefined node).

  Should the algorithm be the same in all thin libs?
  Is it described somewhere?
  We have not found it in java and .net libs. Will it be supported by 
them? When?


- Is there any recommendation/suggestion how to test a node failover 
case on a thin client side?


- We are going to use jsdoc comments for the API methods: 
http://usejsdoc.org/
  This tool, for example, may be used to generate the API spec from the 
comments: https://www.npmjs.com/package/jsdoc

  Any objections?

- We are going to write the lib readme/guide in the markdown format and 
place it at the repo (.md file).

  Is it OK? Or should we use dash.readme.io ?

- May we easily use 3rd-party components with the following licenses?
-- Apache 2.0
-- MIT
-- GNU LGPL v3
-- GNU GPL v2
  Should we pre-approve somehow / notify somebody about usage of 
concrete 3rd-party components?

[jira] [Created] (IGNITE-7783) Thin Client lib: PHP

[Alexey Kosenchuk (JIRA)]

Alexey Kosenchuk created IGNITE-7783:


 Summary: Thin Client lib: PHP
 Key: IGNITE-7783
 URL: https://issues.apache.org/jira/browse/IGNITE-7783
 Project: Ignite
  Issue Type: New Feature
  Components: thin client
Reporter: Alexey Kosenchuk
Assignee: Alexey Kosenchuk


Implement Thin (lightweight) Client lib in PHP programming language for Ignite 
Binary Client Protocol 
https://apacheignite.readme.io/docs/binary-client-protocol



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7782) Thin Client lib: Python

[Alexey Kosenchuk (JIRA)]

Alexey Kosenchuk created IGNITE-7782:


 Summary: Thin Client lib: Python
 Key: IGNITE-7782
 URL: https://issues.apache.org/jira/browse/IGNITE-7782
 Project: Ignite
  Issue Type: New Feature
  Components: thin client
Reporter: Alexey Kosenchuk
Assignee: Alexey Kosenchuk


Implement Thin (lightweight) Client lib in Python programming language for 
Ignite Binary Client Protocol 
https://apacheignite.readme.io/docs/binary-client-protocol



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ignite Thin clients for Node.js, Python, PHP

[Alexey Kosenchuk]

Denis, Alexey, Pavel, Sergey, thanks for info.

We will start from Node.js lib.
I've created Jira (https://issues.apache.org/jira/browse/IGNITE-) to 
track it's scope and status.


Sure, we will provide the main ideas / API for review first, etc. 
according to the process.


-Alexey

20.02.2018 1:09, Denis Magda пишет:

Hi Alexey and welcome to all of you! Look forward to seeing the clients in
Ignite codebase ;)

Granted contributors access to all of you in JIRA. Feel free to assign
existing tickets to yourself or create new ones.

However, before you're getting down to the implementation, please share the
design and architectural ideas around the clients. Plus, it will be
reasonable to start with one client first and then add the rest once you
find out how the community works and Ignite is architectured.

Pavel T., Alex K., as creators of .NET and Java thin clients, please share
pointers that might be useful for Alexey.

--
Denis


On Mon, Feb 19, 2018 at 1:23 PM, Alexey Kosenchuk <
alexey.kosenc...@nobitlost.com> wrote:


Hi All!

Let us join the party, please ;)

As we see, there is Binary Client Protocol to communicate with Ignite
cluster and a concept of Thin (lightweight) client.

If there are no objections or duplicated plans, we would like to develop
Thin Client libs for:
- Node.js
- Python
- PHP

Please add us as contributors and provide access to the Ignite Jira
components.

Usernames in the Apache Jira:
alexey.kosenchuk
ekaterina.vergizova
pavel.petroshenko

Thanks!
-Alexey

[jira] [Created] (IGNITE-7777) Thin Client lib: Node.js

[Alexey Kosenchuk (JIRA)]

Alexey Kosenchuk created IGNITE-:


 Summary: Thin Client lib: Node.js
 Key: IGNITE-
 URL: https://issues.apache.org/jira/browse/IGNITE-
 Project: Ignite
  Issue Type: New Feature
  Components: thin client
Reporter: Alexey Kosenchuk
Assignee: Alexey Kosenchuk


Implement Thin (lightweight) Client lib in Node.js programming language for 
Ignite Binary Client Protocol 
https://apacheignite.readme.io/docs/binary-client-protocol

Examples of other Thin Clients:
.net 
https://github.com/apache/ignite/tree/master/modules/platforms/dotnet/Apache.Ignite.Core/Impl/Client
java 
https://github.com/gridgain/apache-ignite/tree/ignite-7421/modules/thinclient

Scope of work
-

Functionality:
Support all operations of the Ignite Binary Client Protocol 2.4.
Support name/password authentication - TBD (not in the protocol yet).
Support optional SSL/TLS communication - TBD (not in the protocol yet).
Support optional failover/reconnect - TBD.

Minimal Node.js version - TBD.
Synch ops emulation - callbacks, or Promises, or asynch/await - TBD.

Examples:
Cover all basic features - Key-value API, SQL, Scan queries, Cluster 
configuration/management, Authentication, SSL/TLS.

Tests:
Jasmine tests for all API methods and all basic features.
Simple Jasmine tests to start examples.
How to emulate node failure to test failover/reconnect? - TBD.

Docs:
Auto-generated API spec from comments. JSdoc, or javadoc, or what? - TBD.
Readme for the lib.
Simple instruction to setup/run examples.
Simple instruction to setup/run Jasmine tests.

Docs format - .md in the repo, or dash.readme.io ? - TBD.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Ignite Thin clients for Node.js, Python, PHP

[Alexey Kosenchuk]

Hi All!

Let us join the party, please ;)

As we see, there is Binary Client Protocol to communicate with Ignite 
cluster and a concept of Thin (lightweight) client.


If there are no objections or duplicated plans, we would like to develop 
Thin Client libs for:

- Node.js
- Python
- PHP

Please add us as contributors and provide access to the Ignite Jira 
components.


Usernames in the Apache Jira:
alexey.kosenchuk
ekaterina.vergizova
pavel.petroshenko

Thanks!
-Alexey