Re: golang client for Ignite
Hi Aleksandr, That looks really interesting to me. Personally, I would like to see a dedicated Go module in Ignite. Do you support SQL API right now? If it’s so then you might want to switch to Ignite JDBC driver instead that should outperform the REST protocol. Otherwise, if the goal is to go beyond SQL boundaries adding key-value and transactions to the list then we should either use the REST or create a true native client. Read more about native clients development here: http://apache-ignite-developers.2346864.n4.nabble.com/Read-this-if-you-want-to-integrate-Ignite-with-other-platforms-Python-R-etc-td14006.html#a14028 — Denis > On Jun 27, 2017, at 2:29 PM, Aleksandr Sokolovskii wrote: > > Dear Ignite team, > > I see there is no native client for golang. > I tried to fix this gap a little bit and developed golang SQL driver for > ignite/gridgain: https://github.com/amsokol/go-ignite-client. > Enjoy ))). > It's in beta phase now. I’m focusing on test coverage now. > > Driver uses HTTP REST API which is overhead. > Could you please provide me specification of ignite native > platform-independent client-server protocol. > I easy add it as well. > > I think many people tell us thanks for golang native client and SQL driver ))) > > P.S.: If you are interesting in my project please let me know. I can easy > donate (and support) my code to your project. > > Thanks, > Aleksandr > >
RE: golang client for Ignite
Hi Deins, Thanks for answer. Link helps. It’s not good practice to call “c” functions from golang. That’s why I develop client by pure golang without Ignite “cpp” client API. I already tested golang “handshake” using ODBC endpoint server:10800. It works great. But I don’t see support of transactions via ODBC... Yes, I would like to develop true native client. How can I get protocol specification to develop client BinaryMarshaller with pure golang? Thanks, Aleksandr From: Denis Magda Sent: 28 июня 2017 г. 2:00 To: dev@ignite.apache.org Subject: Re: golang client for Ignite Hi Aleksandr, That looks really interesting to me. Personally, I would like to see a dedicated Go module in Ignite. Do you support SQL API right now? If it’s so then you might want to switch to Ignite JDBC driver instead that should outperform the REST protocol. Otherwise, if the goal is to go beyond SQL boundaries adding key-value and transactions to the list then we should either use the REST or create a true native client. Read more about native clients development here: http://apache-ignite-developers.2346864.n4.nabble.com/Read-this-if-you-want-to-integrate-Ignite-with-other-platforms-Python-R-etc-td14006.html#a14028 — Denis > On Jun 27, 2017, at 2:29 PM, Aleksandr Sokolovskii wrote: > > Dear Ignite team, > > I see there is no native client for golang. > I tried to fix this gap a little bit and developed golang SQL driver for > ignite/gridgain: https://github.com/amsokol/go-ignite-client. > Enjoy ))). > It's in beta phase now. I’m focusing on test coverage now. > > Driver uses HTTP REST API which is overhead. > Could you please provide me specification of ignite native > platform-independent client-server protocol. > I easy add it as well. > > I think many people tell us thanks for golang native client and SQL driver ))) > > P.S.: If you are interesting in my project please let me know. I can easy > donate (and support) my code to your project. > > Thanks, > Aleksandr > >
RE: golang client for Ignite
Denis, I take a look into ignite cpp core. Looks like it attaches to running jvm and calls java methods. Please tell me that I’m wrong! ))) It will not work for golang. Is there language-neutral protocol to communicate with Ignite server? Thanks, Aleksandr From: Aleksandr Sokolovskii Sent: 28 июня 2017 г. 22:26 To: dev@ignite.apache.org Subject: RE: golang client for Ignite Hi Deins, Thanks for answer. Link helps. It’s not good practice to call “c” functions from golang. That’s why I develop client by pure golang without Ignite “cpp” client API. I already tested golang “handshake” using ODBC endpoint server:10800. It works great. But I don’t see support of transactions via ODBC... Yes, I would like to develop true native client. How can I get protocol specification to develop client BinaryMarshaller with pure golang? Thanks, Aleksandr From: Denis Magda Sent: 28 июня 2017 г. 2:00 To: dev@ignite.apache.org Subject: Re: golang client for Ignite Hi Aleksandr, That looks really interesting to me. Personally, I would like to see a dedicated Go module in Ignite. Do you support SQL API right now? If it’s so then you might want to switch to Ignite JDBC driver instead that should outperform the REST protocol. Otherwise, if the goal is to go beyond SQL boundaries adding key-value and transactions to the list then we should either use the REST or create a true native client. Read more about native clients development here: http://apache-ignite-developers.2346864.n4.nabble.com/Read-this-if-you-want-to-integrate-Ignite-with-other-platforms-Python-R-etc-td14006.html#a14028 — Denis > On Jun 27, 2017, at 2:29 PM, Aleksandr Sokolovskii wrote: > > Dear Ignite team, > > I see there is no native client for golang. > I tried to fix this gap a little bit and developed golang SQL driver for > ignite/gridgain: https://github.com/amsokol/go-ignite-client. > Enjoy ))). > It's in beta phase now. I’m focusing on test coverage now. > > Driver uses HTTP REST API which is overhead. > Could you please provide me specification of ignite native > platform-independent client-server protocol. > I easy add it as well. > > I think many people tell us thanks for golang native client and SQL driver ))) > > P.S.: If you are interesting in my project please let me know. I can easy > donate (and support) my code to your project. > > Thanks, > Aleksandr > >
Re: golang client for Ignite
Aleksandr, > I take a look into ignite cpp core. > Looks like it attaches to running jvm and calls java methods. > Please tell me that I’m wrong! ))) That’s a correct observation. Both C++ and .NET clients spawn a JVM process underneath and redirect almost all the requests to it. That approach allowed us to support these languages easily. Otherwise, it would have taken ages to develop true C++ and .NET libs. > It will not work for golang. Why? > Is there language-neutral protocol to communicate with Ignite server? REST, ODBC and JDBC only. — Denis > On Jun 28, 2017, at 12:43 PM, Aleksandr Sokolovskii wrote: > > Denis, > > I take a look into ignite cpp core. > Looks like it attaches to running jvm and calls java methods. > Please tell me that I’m wrong! ))) > > It will not work for golang. > Is there language-neutral protocol to communicate with Ignite server? > > Thanks, > Aleksandr > > From: Aleksandr Sokolovskii > Sent: 28 июня 2017 г. 22:26 > To: dev@ignite.apache.org > Subject: RE: golang client for Ignite > > Hi Deins, > > Thanks for answer. > > Link helps. > > It’s not good practice to call “c” functions from golang. > That’s why I develop client by pure golang without Ignite “cpp” client API. > > I already tested golang “handshake” using ODBC endpoint server:10800. > It works great. > But I don’t see support of transactions via ODBC... > > Yes, I would like to develop true native client. > How can I get protocol specification to develop client BinaryMarshaller with > pure golang? > > Thanks, > Aleksandr > > From: Denis Magda > Sent: 28 июня 2017 г. 2:00 > To: dev@ignite.apache.org > Subject: Re: golang client for Ignite > > Hi Aleksandr, > > That looks really interesting to me. Personally, I would like to see a > dedicated Go module in Ignite. > > Do you support SQL API right now? If it’s so then you might want to switch to > Ignite JDBC driver instead that should outperform the REST protocol. > > Otherwise, if the goal is to go beyond SQL boundaries adding key-value and > transactions to the list then we should either use the REST or create a true > native client. Read more about native clients development here: > http://apache-ignite-developers.2346864.n4.nabble.com/Read-this-if-you-want-to-integrate-Ignite-with-other-platforms-Python-R-etc-td14006.html#a14028 > > — > Denis > >> On Jun 27, 2017, at 2:29 PM, Aleksandr Sokolovskii wrote: >> >> Dear Ignite team, >> >> I see there is no native client for golang. >> I tried to fix this gap a little bit and developed golang SQL driver for >> ignite/gridgain: https://github.com/amsokol/go-ignite-client. >> Enjoy ))). >> It's in beta phase now. I’m focusing on test coverage now. >> >> Driver uses HTTP REST API which is overhead. >> Could you please provide me specification of ignite native >> platform-independent client-server protocol. >> I easy add it as well. >> >> I think many people tell us thanks for golang native client and SQL driver >> ))) >> >> P.S.: If you are interesting in my project please let me know. I can easy >> donate (and support) my code to your project. >> >> Thanks, >> Aleksandr >> >> > > >
Re: golang client for Ignite
Denis, Perhaps adding the same level of support for Golang as we have for .NET or C++ would be asking too much. The reason we start a JVM in .NET and C++ is because we implemented the full Ignite API support, even including transactions and executing .NET closures on remote Java servers. Perhaps, from Golang client standpoint, it is enough to implement it directly over the REST protocol initially. D. On Wed, Jun 28, 2017 at 3:59 PM, Denis Magda wrote: > Aleksandr, > > > I take a look into ignite cpp core. > > Looks like it attaches to running jvm and calls java methods. > > Please tell me that I’m wrong! ))) > > That’s a correct observation. Both C++ and .NET clients spawn a JVM > process underneath and redirect almost all the requests to it. That > approach allowed us to support these languages easily. Otherwise, it would > have taken ages to develop true C++ and .NET libs. > > > It will not work for golang. > > Why? > > > Is there language-neutral protocol to communicate with Ignite server? > > REST, ODBC and JDBC only. > > — > Denis > > > > On Jun 28, 2017, at 12:43 PM, Aleksandr Sokolovskii > wrote: > > > > Denis, > > > > I take a look into ignite cpp core. > > Looks like it attaches to running jvm and calls java methods. > > Please tell me that I’m wrong! ))) > > > > It will not work for golang. > > Is there language-neutral protocol to communicate with Ignite server? > > > > Thanks, > > Aleksandr > > > > From: Aleksandr Sokolovskii > > Sent: 28 июня 2017 г. 22:26 > > To: dev@ignite.apache.org > > Subject: RE: golang client for Ignite > > > > Hi Deins, > > > > Thanks for answer. > > > > Link helps. > > > > It’s not good practice to call “c” functions from golang. > > That’s why I develop client by pure golang without Ignite “cpp” client > API. > > > > I already tested golang “handshake” using ODBC endpoint server:10800. > > It works great. > > But I don’t see support of transactions via ODBC... > > > > Yes, I would like to develop true native client. > > How can I get protocol specification to develop client BinaryMarshaller > with pure golang? > > > > Thanks, > > Aleksandr > > > > From: Denis Magda > > Sent: 28 июня 2017 г. 2:00 > > To: dev@ignite.apache.org > > Subject: Re: golang client for Ignite > > > > Hi Aleksandr, > > > > That looks really interesting to me. Personally, I would like to see a > dedicated Go module in Ignite. > > > > Do you support SQL API right now? If it’s so then you might want to > switch to Ignite JDBC driver instead that should outperform the REST > protocol. > > > > Otherwise, if the goal is to go beyond SQL boundaries adding key-value > and transactions to the list then we should either use the REST or create > a true native client. Read more about native clients development here: > > http://apache-ignite-developers.2346864.n4.nabble. > com/Read-this-if-you-want-to-integrate-Ignite-with-other- > platforms-Python-R-etc-td14006.html#a14028 > > > > — > > Denis > > > >> On Jun 27, 2017, at 2:29 PM, Aleksandr Sokolovskii > wrote: > >> > >> Dear Ignite team, > >> > >> I see there is no native client for golang. > >> I tried to fix this gap a little bit and developed golang SQL driver > for ignite/gridgain: https://github.com/amsokol/go-ignite-client. > >> Enjoy ))). > >> It's in beta phase now. I’m focusing on test coverage now. > >> > >> Driver uses HTTP REST API which is overhead. > >> Could you please provide me specification of ignite native > platform-independent client-server protocol. > >> I easy add it as well. > >> > >> I think many people tell us thanks for golang native client and SQL > driver ))) > >> > >> P.S.: If you are interesting in my project please let me know. I can > easy donate (and support) my code to your project. > >> > >> Thanks, > >> Aleksandr > >> > >> > > > > > > > >
Re: golang client for Ignite
Guys, Just an observation, but when I introduce Ignite to other developers, they get very interested. But when the question goes to the support of the language they use for development, often they look disappointed because a chunk of functionalities I introduce becomes inaccessible. I think from their NoSQL experience they expect a client protocol. As for Golang, from what I know, you have to write bindings in C for C++ and use them. There are performance concerns with cgo. -- Roman On Thursday, June 29, 2017 9:34 AM, Dmitriy Setrakyan wrote: Denis, Perhaps adding the same level of support for Golang as we have for .NET or C++ would be asking too much. The reason we start a JVM in .NET and C++ is because we implemented the full Ignite API support, even including transactions and executing .NET closures on remote Java servers. Perhaps, from Golang client standpoint, it is enough to implement it directly over the REST protocol initially. D. On Wed, Jun 28, 2017 at 3:59 PM, Denis Magda wrote: > Aleksandr, > > > I take a look into ignite cpp core. > > Looks like it attaches to running jvm and calls java methods. > > Please tell me that I’m wrong! ))) > > That’s a correct observation. Both C++ and .NET clients spawn a JVM > process underneath and redirect almost all the requests to it. That > approach allowed us to support these languages easily. Otherwise, it would > have taken ages to develop true C++ and .NET libs. > > > It will not work for golang. > > Why? > > > Is there language-neutral protocol to communicate with Ignite server? > > REST, ODBC and JDBC only. > > — > Denis > > > > On Jun 28, 2017, at 12:43 PM, Aleksandr Sokolovskii > wrote: > > > > Denis, > > > > I take a look into ignite cpp core. > > Looks like it attaches to running jvm and calls java methods. > > Please tell me that I’m wrong! ))) > > > > It will not work for golang. > > Is there language-neutral protocol to communicate with Ignite server? > > > > Thanks, > > Aleksandr > > > > From: Aleksandr Sokolovskii > > Sent: 28 июня 2017 г. 22:26 > > To: dev@ignite.apache.org > > Subject: RE: golang client for Ignite > > > > Hi Deins, > > > > Thanks for answer. > > > > Link helps. > > > > It’s not good practice to call “c” functions from golang. > > That’s why I develop client by pure golang without Ignite “cpp” client > API. > > > > I already tested golang “handshake” using ODBC endpoint server:10800. > > It works great. > > But I don’t see support of transactions via ODBC... > > > > Yes, I would like to develop true native client. > > How can I get protocol specification to develop client BinaryMarshaller > with pure golang? > > > > Thanks, > > Aleksandr > > > > From: Denis Magda > > Sent: 28 июня 2017 г. 2:00 > > To: dev@ignite.apache.org > > Subject: Re: golang client for Ignite > > > > Hi Aleksandr, > > > > That looks really interesting to me. Personally, I would like to see a > dedicated Go module in Ignite. > > > > Do you support SQL API right now? If it’s so then you might want to > switch to Ignite JDBC driver instead that should outperform the REST > protocol. > > > > Otherwise, if the goal is to go beyond SQL boundaries adding key-value > and transactions to the list then we should either use the REST or create > a true native client. Read more about native clients development here: > > http://apache-ignite-developers.2346864.n4.nabble. > com/Read-this-if-you-want-to-integrate-Ignite-with-other- > platforms-Python-R-etc-td14006.html#a14028 > > > > — > > Denis > > > >> On Jun 27, 2017, at 2:29 PM, Aleksandr Sokolovskii > wrote: > >> > >> Dear Ignite team, > >> > >> I see there is no native client for golang. > >> I tried to fix this gap a little bit and developed golang SQL driver > for ignite/gridgain: https://github.com/amsokol/go-ignite-client. > >> Enjoy ))). > >> It's in beta phase now. I’m focusing on test coverage now. > >> > >> Driver uses HTTP REST API which is overhead. > >> Could you please provide me specification of ignite native > platform-independent client-server protocol. > >> I easy add it as well. > >> > >> I think many people tell us thanks for golang native client and SQL > driver ))) > >> > >> P.S.: If you are interesting in my project please let me know. I can > easy donate (and support) my code to your project. > >> > >> Thanks, > >> Aleksandr > >> > >> > > > > > > > >
RE: golang client for Ignite
Guys, Thanks for your answers. So If users want to use Ignite DataGrid from “unsupported” language they have two choice for now: 1) REST API It supports: cache management, put-get, sql exec queries, fetch It does not support: sql transactions REST API is easy to use. REST API has a lot of overhead (HTTP, json marshalling/unmarshalling,.. you know) 2) develop pure SQL driver for Ignite ODBC endpoint It supports: sql exec queries, fetch It does not support: cache management, put-get, sql transactions Roman is right – using cgo is not very good idea. Honestly It’s not trivial task to develop pure SQL driver for Ignite ODBC endpoint. I spent some hours to remember how STL serializes std::string to unmarshall it in golang. ))) My last C++ experience was in 2004. ))) Another my concern is C++ wrapper over Ignite Java process. I try to explain: The main reason to develop something using golang is performance. People how use golang are enslaved by performance. They think how to avoid unnecessary memory allocation, avoid reflection, etc. Go provides goroutine instead of system threads to avoid memory allocation and cross-threads switching. Cross-process switching (cpp->jvm->cpp) on server side of in-memory database looks like “shot yourself in foot” for such people. ))) There is another way: to implement native client for H2 DB protocol. (If I’m not mistaken Ignite use H2 DB protocol for JDBC endpoint). Again it’s not trivial task to implement pure native H2 DB client for each necessary programming language. What do you think about to implement gRPC (or Apache Thrift, or…) platform-language-neutral protocol endpoint in Java core of Ignite? I resolves “unsupported” language client problem. Thanks, Aleksandr From: Roman Shtykh Sent: 29 июня 2017 г. 6:36 To: dev@ignite.apache.org Subject: Re: golang client for Ignite Guys, Just an observation, but when I introduce Ignite to other developers, they get very interested. But when the question goes to the support of the language they use for development, often they look disappointed because a chunk of functionalities I introduce becomes inaccessible. I think from their NoSQL experience they expect a client protocol. As for Golang, from what I know, you have to write bindings in C for C++ and use them. There are performance concerns with cgo. -- Roman On Thursday, June 29, 2017 9:34 AM, Dmitriy Setrakyan wrote: Denis, Perhaps adding the same level of support for Golang as we have for .NET or C++ would be asking too much. The reason we start a JVM in .NET and C++ is because we implemented the full Ignite API support, even including transactions and executing .NET closures on remote Java servers. Perhaps, from Golang client standpoint, it is enough to implement it directly over the REST protocol initially. D. On Wed, Jun 28, 2017 at 3:59 PM, Denis Magda wrote: > Aleksandr, > > > I take a look into ignite cpp core. > > Looks like it attaches to running jvm and calls java methods. > > Please tell me that I’m wrong! ))) > > That’s a correct observation. Both C++ and .NET clients spawn a JVM > process underneath and redirect almost all the requests to it. That > approach allowed us to support these languages easily. Otherwise, it would > have taken ages to develop true C++ and .NET libs. > > > It will not work for golang. > > Why? > > > Is there language-neutral protocol to communicate with Ignite server? > > REST, ODBC and JDBC only. > > — > Denis > > > > On Jun 28, 2017, at 12:43 PM, Aleksandr Sokolovskii > wrote: > > > > Denis, > > > > I take a look into ignite cpp core. > > Looks like it attaches to running jvm and calls java methods. > > Please tell me that I’m wrong! ))) > > > > It will not work for golang. > > Is there language-neutral protocol to communicate with Ignite server? > > > > Thanks, > > Aleksandr > > > > From: Aleksandr Sokolovskii > > Sent: 28 июня 2017 г. 22:26 > > To: dev@ignite.apache.org > > Subject: RE: golang client for Ignite > > > > Hi Deins, > > > > Thanks for answer. > > > > Link helps. > > > > It’s not good practice to call “c” functions from golang. > > That’s why I develop client by pure golang without Ignite “cpp” client > API. > > > > I already tested golang “handshake” using ODBC endpoint server:10800. > > It works great. > > But I don’t see support of transactions via ODBC... > > > > Yes, I would like to develop true native client. > > How can I get protocol specification to develop client BinaryMarshaller > with pure golang? > > > > Thanks, > > Aleksandr > > > > From: Denis Magda > > Sent: 28 июня 2017 г. 2:00 > > To: dev@ignite.apache.org > > Subject: R
Re: golang client for Ignite
Aleksandr, Ignite is distributed system. Typical call to get/put/sql is of micro- and millisecond magnitude. Java->CPP call takes several dozen nanoseconds. CPP->Java (so-called "reverse JNI") takes several hundred nanoseconds. Typical marshalling overhead is of nano- and micro-second scale as well. That is, total overhead of platform->Java->platform interaction is several orders of magnitude *smaller* than any call to remote node. So if Golang users are afraid of JNI, than any distributed system would scare them to death. We have a plan to implement platform-independent thin client protocol which could be re-used from many languages. But you should understand, that in most cases it will be *slower* than current implementation, because Java core contains essential logic for efficient request routing. Native thin client could be faster only if you port many and many thousands of relevant non-trivial Java code to native platform, which can be estimated not to man-months, but to man-years to complete. Having said that, I would recommend you to look closer to current JNI-based implementation :-) On Thu, Jun 29, 2017 at 9:11 AM, Aleksandr Sokolovskii wrote: > Guys, > > Thanks for your answers. > > So If users want to use Ignite DataGrid from “unsupported” language they > have two choice for now: > > 1) REST API > It supports: cache management, put-get, sql exec queries, fetch > It does not support: sql transactions > REST API is easy to use. > REST API has a lot of overhead (HTTP, json marshalling/unmarshalling,.. > you know) > > 2) develop pure SQL driver for Ignite ODBC endpoint > It supports: sql exec queries, fetch > It does not support: cache management, put-get, sql transactions > Roman is right – using cgo is not very good idea. > Honestly It’s not trivial task to develop pure SQL driver for Ignite ODBC > endpoint. > I spent some hours to remember how STL serializes std::string to > unmarshall it in golang. ))) > My last C++ experience was in 2004. ))) > Another my concern is C++ wrapper over Ignite Java process. > I try to explain: > The main reason to develop something using golang is performance. > People how use golang are enslaved by performance. > They think how to avoid unnecessary memory allocation, avoid reflection, > etc. > Go provides goroutine instead of system threads to avoid memory allocation > and cross-threads switching. > Cross-process switching (cpp->jvm->cpp) on server side of in-memory > database looks like “shot yourself in foot” for such people. ))) > > There is another way: to implement native client for H2 DB protocol. > (If I’m not mistaken Ignite use H2 DB protocol for JDBC endpoint). > Again it’s not trivial task to implement pure native H2 DB client for each > necessary programming language. > > What do you think about to implement gRPC (or Apache Thrift, or…) > platform-language-neutral protocol endpoint in Java core of Ignite? > I resolves “unsupported” language client problem. > > Thanks, > Aleksandr > > From: Roman Shtykh > Sent: 29 июня 2017 г. 6:36 > To: dev@ignite.apache.org > Subject: Re: golang client for Ignite > > Guys, > > Just an observation, but when I introduce Ignite to other developers, they > get very interested. But when the question goes to the support of the > language they use for development, often they look disappointed because a > chunk of functionalities I introduce becomes inaccessible. I think from > their NoSQL experience they expect a client protocol. > As for Golang, from what I know, you have to write bindings in C for C++ > and use them. There are performance concerns with cgo. > -- Roman > > > On Thursday, June 29, 2017 9:34 AM, Dmitriy Setrakyan < > dsetrak...@apache.org> wrote: > > > Denis, > > Perhaps adding the same level of support for Golang as we have for .NET or > C++ would be asking too much. The reason we start a JVM in .NET and C++ is > because we implemented the full Ignite API support, even including > transactions and executing .NET closures on remote Java servers. > > Perhaps, from Golang client standpoint, it is enough to implement it > directly over the REST protocol initially. > > D. > > On Wed, Jun 28, 2017 at 3:59 PM, Denis Magda wrote: > > > Aleksandr, > > > > > I take a look into ignite cpp core. > > > Looks like it attaches to running jvm and calls java methods. > > > Please tell me that I’m wrong! ))) > > > > That’s a correct observation. Both C++ and .NET clients spawn a JVM > > process underneath and redirect almost all the requests to it. That > > approach allowed us to support these languages easily. Otherwise, it > would > > have taken ages to develop true C
RE: golang client for Ignite
Dear Vladimir, Thanks for your answer. I’m a little bit confused. You write: We have a plan to implement platform-independent thin client protocol which could be re-used from many languages. But you should understand, that in most cases it will be *slower* than current implementation, because Java core contains essential logic for efficient request routing. Current C++ realization in Ignite now: Client[cpp]->network->Server[cpp->java] If you recommend use “Java core contains essential logic for efficient request routing” why you don’t implement the following for C++?: Client[cpp->java]->network->Server[java] Why platform-independent protocol: Client[cpp->]->network->Server[java] Will be slow than current?: Client[cpp]->network->Server[cpp->java] Thanks, Aleksandr From: Vladimir Ozerov Sent: 30 июня 2017 г. 0:25 To: dev@ignite.apache.org Cc: Roman Shtykh Subject: Re: golang client for Ignite Aleksandr, Ignite is distributed system. Typical call to get/put/sql is of micro- and millisecond magnitude. Java->CPP call takes several dozen nanoseconds. CPP->Java (so-called "reverse JNI") takes several hundred nanoseconds. Typical marshalling overhead is of nano- and micro-second scale as well. That is, total overhead of platform->Java->platform interaction is several orders of magnitude *smaller* than any call to remote node. So if Golang users are afraid of JNI, than any distributed system would scare them to death. We have a plan to implement platform-independent thin client protocol which could be re-used from many languages. But you should understand, that in most cases it will be *slower* than current implementation, because Java core contains essential logic for efficient request routing. Native thin client could be faster only if you port many and many thousands of relevant non-trivial Java code to native platform, which can be estimated not to man-months, but to man-years to complete. Having said that, I would recommend you to look closer to current JNI-based implementation :-) On Thu, Jun 29, 2017 at 9:11 AM, Aleksandr Sokolovskii wrote: > Guys, > > Thanks for your answers. > > So If users want to use Ignite DataGrid from “unsupported” language they > have two choice for now: > > 1) REST API > It supports: cache management, put-get, sql exec queries, fetch > It does not support: sql transactions > REST API is easy to use. > REST API has a lot of overhead (HTTP, json marshalling/unmarshalling,.. > you know) > > 2) develop pure SQL driver for Ignite ODBC endpoint > It supports: sql exec queries, fetch > It does not support: cache management, put-get, sql transactions > Roman is right – using cgo is not very good idea. > Honestly It’s not trivial task to develop pure SQL driver for Ignite ODBC > endpoint. > I spent some hours to remember how STL serializes std::string to > unmarshall it in golang. ))) > My last C++ experience was in 2004. ))) > Another my concern is C++ wrapper over Ignite Java process. > I try to explain: > The main reason to develop something using golang is performance. > People how use golang are enslaved by performance. > They think how to avoid unnecessary memory allocation, avoid reflection, > etc. > Go provides goroutine instead of system threads to avoid memory allocation > and cross-threads switching. > Cross-process switching (cpp->jvm->cpp) on server side of in-memory > database looks like “shot yourself in foot” for such people. ))) > > There is another way: to implement native client for H2 DB protocol. > (If I’m not mistaken Ignite use H2 DB protocol for JDBC endpoint). > Again it’s not trivial task to implement pure native H2 DB client for each > necessary programming language. > > What do you think about to implement gRPC (or Apache Thrift, or…) > platform-language-neutral protocol endpoint in Java core of Ignite? > I resolves “unsupported” language client problem. > > Thanks, > Aleksandr > > From: Roman Shtykh > Sent: 29 июня 2017 г. 6:36 > To: dev@ignite.apache.org > Subject: Re: golang client for Ignite > > Guys, > > Just an observation, but when I introduce Ignite to other developers, they > get very interested. But when the question goes to the support of the > language they use for development, often they look disappointed because a > chunk of functionalities I introduce becomes inaccessible. I think from > their NoSQL experience they expect a client protocol. > As for Golang, from what I know, you have to write bindings in C for C++ > and use them. There are performance concerns with cgo. > -- Roman > > > On Thursday, June 29, 2017 9:34 AM, Dmitriy Setrakyan < > dsetrak...@apache.org> wrote: > > > Denis, > > Perhaps adding the same level of support for Golang as we
Re: golang client for Ignite
Aleksandr, Currently Ignite.NET and Ignite C++ work as follows: >>> Client[platform->*jni->java*]->network->Server[java] Or as follows in some cases (e.g. calling remote .NET job from local .NET node): >>> Client[platform->*jni->java*]->network->Server[java->*jni->platform*] Possible thin client will work as follows (ODBC driver already works this way): >>> Client[platform]->*network*->EdgeNode[java]->network->Server[java] Note additional network hop from client to edge node because thin client lacks routing logic. On Fri, Jun 30, 2017 at 12:38 AM, Aleksandr Sokolovskii wrote: > Dear Vladimir, > > Thanks for your answer. > > I’m a little bit confused. > > You write: > We have a plan to implement platform-independent thin client protocol which > could be re-used from many languages. But you should understand, that in > most cases it will be *slower* than current implementation, because Java > core contains essential logic for efficient request routing. > > Current C++ realization in Ignite now: > Client[cpp]->network->Server[cpp->java] > > If you recommend use “Java core contains essential logic for efficient > request routing” why you don’t implement the following for C++?: > Client[cpp->java]->network->Server[java] > > > Why platform-independent protocol: > Client[cpp->]->network->Server[java] > > Will be slow than current?: > Client[cpp]->network->Server[cpp->java] > > Thanks, > Aleksandr > > From: Vladimir Ozerov > Sent: 30 июня 2017 г. 0:25 > To: dev@ignite.apache.org > Cc: Roman Shtykh > Subject: Re: golang client for Ignite > > Aleksandr, > > Ignite is distributed system. Typical call to get/put/sql is of micro- and > millisecond magnitude. Java->CPP call takes several dozen nanoseconds. > CPP->Java (so-called "reverse JNI") takes several hundred nanoseconds. > Typical marshalling overhead is of nano- and micro-second scale as well. > That is, total overhead of platform->Java->platform interaction is several > orders of magnitude *smaller* than any call to remote node. So if Golang > users are afraid of JNI, than any distributed system would scare them to > death. > > We have a plan to implement platform-independent thin client protocol which > could be re-used from many languages. But you should understand, that in > most cases it will be *slower* than current implementation, because Java > core contains essential logic for efficient request routing. Native thin > client could be faster only if you port many and many thousands of relevant > non-trivial Java code to native platform, which can be estimated not to > man-months, but to man-years to complete. > > Having said that, I would recommend you to look closer to current JNI-based > implementation :-) > > > On Thu, Jun 29, 2017 at 9:11 AM, Aleksandr Sokolovskii > wrote: > > > Guys, > > > > Thanks for your answers. > > > > So If users want to use Ignite DataGrid from “unsupported” language they > > have two choice for now: > > > > 1) REST API > > It supports: cache management, put-get, sql exec queries, fetch > > It does not support: sql transactions > > REST API is easy to use. > > REST API has a lot of overhead (HTTP, json marshalling/unmarshalling,.. > > you know) > > > > 2) develop pure SQL driver for Ignite ODBC endpoint > > It supports: sql exec queries, fetch > > It does not support: cache management, put-get, sql transactions > > Roman is right – using cgo is not very good idea. > > Honestly It’s not trivial task to develop pure SQL driver for Ignite ODBC > > endpoint. > > I spent some hours to remember how STL serializes std::string to > > unmarshall it in golang. ))) > > My last C++ experience was in 2004. ))) > > Another my concern is C++ wrapper over Ignite Java process. > > I try to explain: > > The main reason to develop something using golang is performance. > > People how use golang are enslaved by performance. > > They think how to avoid unnecessary memory allocation, avoid reflection, > > etc. > > Go provides goroutine instead of system threads to avoid memory > allocation > > and cross-threads switching. > > Cross-process switching (cpp->jvm->cpp) on server side of in-memory > > database looks like “shot yourself in foot” for such people. ))) > > > > There is another way: to implement native client for H2 DB protocol. > > (If I’m not mistaken Ignite use H2 DB protocol for JDBC endpoint). > > Again it’s not trivial task to implement pure native H2 DB client for > each > > necessa
RE: golang client for Ignite
Dear Vladimir, Thanks for your answer. Now I understand the difference between ‘client’ and ‘thin client’. Just to be clear I develop ‘thin client’ for DataGrid now: https://github.com/amsokol/go-ignite-client and all my questions are in context of ‘thin client’. So the questions are: 1) Does EdgeNode provide binary endpoint that supports cache management, put-get, sql exec queries, fetch, transactions and I can use it by ‘thin’ golang client? 2) If the answer to the question (1) is ‘NO’ – is it possible to add transactions to ODBC endpoint? 3) If the answer to the question (2) is ‘NO’ – is it possible to add transactions to REST API? Thanks, Aleksandr From: Vladimir Ozerov Sent: 30 июня 2017 г. 8:18 To: dev@ignite.apache.org Cc: Roman Shtykh Subject: Re: golang client for Ignite Aleksandr, Currently Ignite.NET and Ignite C++ work as follows: >>> Client[platform->*jni->java*]->network->Server[java] Or as follows in some cases (e.g. calling remote .NET job from local .NET node): >>> Client[platform->*jni->java*]->network->Server[java->*jni->platform*] Possible thin client will work as follows (ODBC driver already works this way): >>> Client[platform]->*network*->EdgeNode[java]->network->Server[java] Note additional network hop from client to edge node because thin client lacks routing logic. On Fri, Jun 30, 2017 at 12:38 AM, Aleksandr Sokolovskii wrote: > Dear Vladimir, > > Thanks for your answer. > > I’m a little bit confused. > > You write: > We have a plan to implement platform-independent thin client protocol which > could be re-used from many languages. But you should understand, that in > most cases it will be *slower* than current implementation, because Java > core contains essential logic for efficient request routing. > > Current C++ realization in Ignite now: > Client[cpp]->network->Server[cpp->java] > > If you recommend use “Java core contains essential logic for efficient > request routing” why you don’t implement the following for C++?: > Client[cpp->java]->network->Server[java] > > > Why platform-independent protocol: > Client[cpp->]->network->Server[java] > > Will be slow than current?: > Client[cpp]->network->Server[cpp->java] > > Thanks, > Aleksandr > > From: Vladimir Ozerov > Sent: 30 июня 2017 г. 0:25 > To: dev@ignite.apache.org > Cc: Roman Shtykh > Subject: Re: golang client for Ignite > > Aleksandr, > > Ignite is distributed system. Typical call to get/put/sql is of micro- and > millisecond magnitude. Java->CPP call takes several dozen nanoseconds. > CPP->Java (so-called "reverse JNI") takes several hundred nanoseconds. > Typical marshalling overhead is of nano- and micro-second scale as well. > That is, total overhead of platform->Java->platform interaction is several > orders of magnitude *smaller* than any call to remote node. So if Golang > users are afraid of JNI, than any distributed system would scare them to > death. > > We have a plan to implement platform-independent thin client protocol which > could be re-used from many languages. But you should understand, that in > most cases it will be *slower* than current implementation, because Java > core contains essential logic for efficient request routing. Native thin > client could be faster only if you port many and many thousands of relevant > non-trivial Java code to native platform, which can be estimated not to > man-months, but to man-years to complete. > > Having said that, I would recommend you to look closer to current JNI-based > implementation :-) > > > On Thu, Jun 29, 2017 at 9:11 AM, Aleksandr Sokolovskii > wrote: > > > Guys, > > > > Thanks for your answers. > > > > So If users want to use Ignite DataGrid from “unsupported” language they > > have two choice for now: > > > > 1) REST API > > It supports: cache management, put-get, sql exec queries, fetch > > It does not support: sql transactions > > REST API is easy to use. > > REST API has a lot of overhead (HTTP, json marshalling/unmarshalling,.. > > you know) > > > > 2) develop pure SQL driver for Ignite ODBC endpoint > > It supports: sql exec queries, fetch > > It does not support: cache management, put-get, sql transactions > > Roman is right – using cgo is not very good idea. > > Honestly It’s not trivial task to develop pure SQL driver for Ignite ODBC > > endpoint. > > I spent some hours to remember how STL serializes std::string to > > unmarshall it in golang. ))) > > My last C++ experience was in 2004. ))) > > Another my concern is C++ wrapper over Ignite Java process. > > I try to ex
Re: golang client for Ignite
Hi Aleksandr, Regarding transactions: now implementation of transactions assumes that transaction always belongs to some Ignite node, and it seems it is not simple task to support transactions for 'thin' clients. Thanks On Fri, Jun 30, 2017 at 9:55 AM, Aleksandr Sokolovskii wrote: > Dear Vladimir, > > Thanks for your answer. > > Now I understand the difference between ‘client’ and ‘thin client’. > > Just to be clear I develop ‘thin client’ for DataGrid now: > https://github.com/amsokol/go-ignite-client > and all my questions are in context of ‘thin client’. > > So the questions are: > 1) Does EdgeNode provide binary endpoint that supports cache management, > put-get, sql exec queries, fetch, transactions and I can use it by ‘thin’ > golang client? > 2) If the answer to the question (1) is ‘NO’ – is it possible to add > transactions to ODBC endpoint? > 3) If the answer to the question (2) is ‘NO’ – is it possible to add > transactions to REST API? > > Thanks, > Aleksandr > > From: Vladimir Ozerov > Sent: 30 июня 2017 г. 8:18 > To: dev@ignite.apache.org > Cc: Roman Shtykh > Subject: Re: golang client for Ignite > > Aleksandr, > > Currently Ignite.NET and Ignite C++ work as follows: > >>> Client[platform->*jni->java*]->network->Server[java] > > Or as follows in some cases (e.g. calling remote .NET job from local .NET > node): > >>> Client[platform->*jni->java*]->network->Server[java->*jni->platform*] > > Possible thin client will work as follows (ODBC driver already works this > way): > >>> Client[platform]->*network*->EdgeNode[java]->network->Server[java] > > Note additional network hop from client to edge node because thin client > lacks routing logic. > > > On Fri, Jun 30, 2017 at 12:38 AM, Aleksandr Sokolovskii > > wrote: > > > Dear Vladimir, > > > > Thanks for your answer. > > > > I’m a little bit confused. > > > > You write: > > We have a plan to implement platform-independent thin client protocol > which > > could be re-used from many languages. But you should understand, that in > > most cases it will be *slower* than current implementation, because Java > > core contains essential logic for efficient request routing. > > > > Current C++ realization in Ignite now: > > Client[cpp]->network->Server[cpp->java] > > > > If you recommend use “Java core contains essential logic for efficient > > request routing” why you don’t implement the following for C++?: > > Client[cpp->java]->network->Server[java] > > > > > > Why platform-independent protocol: > > Client[cpp->]->network->Server[java] > > > > Will be slow than current?: > > Client[cpp]->network->Server[cpp->java] > > > > Thanks, > > Aleksandr > > > > From: Vladimir Ozerov > > Sent: 30 июня 2017 г. 0:25 > > To: dev@ignite.apache.org > > Cc: Roman Shtykh > > Subject: Re: golang client for Ignite > > > > Aleksandr, > > > > Ignite is distributed system. Typical call to get/put/sql is of micro- > and > > millisecond magnitude. Java->CPP call takes several dozen nanoseconds. > > CPP->Java (so-called "reverse JNI") takes several hundred nanoseconds. > > Typical marshalling overhead is of nano- and micro-second scale as well. > > That is, total overhead of platform->Java->platform interaction is > several > > orders of magnitude *smaller* than any call to remote node. So if Golang > > users are afraid of JNI, than any distributed system would scare them to > > death. > > > > We have a plan to implement platform-independent thin client protocol > which > > could be re-used from many languages. But you should understand, that in > > most cases it will be *slower* than current implementation, because Java > > core contains essential logic for efficient request routing. Native thin > > client could be faster only if you port many and many thousands of > relevant > > non-trivial Java code to native platform, which can be estimated not to > > man-months, but to man-years to complete. > > > > Having said that, I would recommend you to look closer to current > JNI-based > > implementation :-) > > > > > > On Thu, Jun 29, 2017 at 9:11 AM, Aleksandr Sokolovskii < > amso...@gmail.com> > > wrote: > > > > > Guys, > > > > > > Thanks for your answers. > > > > > > So If users want to use Ignite DataGrid from “unsupported” language > they > > > have tw
RE: golang client for Ignite
Dear Semyon, Thanks. Understood. ((( Thanks, Aleksandr From: Semyon Boikov Sent: 30 июня 2017 г. 10:51 To: dev@ignite.apache.org Cc: Roman Shtykh Subject: Re: golang client for Ignite Hi Aleksandr, Regarding transactions: now implementation of transactions assumes that transaction always belongs to some Ignite node, and it seems it is not simple task to support transactions for 'thin' clients. Thanks On Fri, Jun 30, 2017 at 9:55 AM, Aleksandr Sokolovskii wrote: > Dear Vladimir, > > Thanks for your answer. > > Now I understand the difference between ‘client’ and ‘thin client’. > > Just to be clear I develop ‘thin client’ for DataGrid now: > https://github.com/amsokol/go-ignite-client > and all my questions are in context of ‘thin client’. > > So the questions are: > 1) Does EdgeNode provide binary endpoint that supports cache management, > put-get, sql exec queries, fetch, transactions and I can use it by ‘thin’ > golang client? > 2) If the answer to the question (1) is ‘NO’ – is it possible to add > transactions to ODBC endpoint? > 3) If the answer to the question (2) is ‘NO’ – is it possible to add > transactions to REST API? > > Thanks, > Aleksandr > > From: Vladimir Ozerov > Sent: 30 июня 2017 г. 8:18 > To: dev@ignite.apache.org > Cc: Roman Shtykh > Subject: Re: golang client for Ignite > > Aleksandr, > > Currently Ignite.NET and Ignite C++ work as follows: > >>> Client[platform->*jni->java*]->network->Server[java] > > Or as follows in some cases (e.g. calling remote .NET job from local .NET > node): > >>> Client[platform->*jni->java*]->network->Server[java->*jni->platform*] > > Possible thin client will work as follows (ODBC driver already works this > way): > >>> Client[platform]->*network*->EdgeNode[java]->network->Server[java] > > Note additional network hop from client to edge node because thin client > lacks routing logic. > > > On Fri, Jun 30, 2017 at 12:38 AM, Aleksandr Sokolovskii > > wrote: > > > Dear Vladimir, > > > > Thanks for your answer. > > > > I’m a little bit confused. > > > > You write: > > We have a plan to implement platform-independent thin client protocol > which > > could be re-used from many languages. But you should understand, that in > > most cases it will be *slower* than current implementation, because Java > > core contains essential logic for efficient request routing. > > > > Current C++ realization in Ignite now: > > Client[cpp]->network->Server[cpp->java] > > > > If you recommend use “Java core contains essential logic for efficient > > request routing” why you don’t implement the following for C++?: > > Client[cpp->java]->network->Server[java] > > > > > > Why platform-independent protocol: > > Client[cpp->]->network->Server[java] > > > > Will be slow than current?: > > Client[cpp]->network->Server[cpp->java] > > > > Thanks, > > Aleksandr > > > > From: Vladimir Ozerov > > Sent: 30 июня 2017 г. 0:25 > > To: dev@ignite.apache.org > > Cc: Roman Shtykh > > Subject: Re: golang client for Ignite > > > > Aleksandr, > > > > Ignite is distributed system. Typical call to get/put/sql is of micro- > and > > millisecond magnitude. Java->CPP call takes several dozen nanoseconds. > > CPP->Java (so-called "reverse JNI") takes several hundred nanoseconds. > > Typical marshalling overhead is of nano- and micro-second scale as well. > > That is, total overhead of platform->Java->platform interaction is > several > > orders of magnitude *smaller* than any call to remote node. So if Golang > > users are afraid of JNI, than any distributed system would scare them to > > death. > > > > We have a plan to implement platform-independent thin client protocol > which > > could be re-used from many languages. But you should understand, that in > > most cases it will be *slower* than current implementation, because Java > > core contains essential logic for efficient request routing. Native thin > > client could be faster only if you port many and many thousands of > relevant > > non-trivial Java code to native platform, which can be estimated not to > > man-months, but to man-years to complete. > > > > Having said that, I would recommend you to look closer to current > JNI-based > > implementation :-) > > > > > > On Thu, Jun 29, 2017 at 9:11 AM, Aleksandr Sokolovskii < > amso...@gmail.com> > > wrote: > > > > > Guy
