Ticket: https://issues.apache.org/jira/browse/IGNITE-4922 <https://issues.apache.org/jira/browse/IGNITE-4922>
— Denis > On Mar 30, 2017, at 7:06 PM, Denis Magda <dma...@apache.org> wrote: > > Folks, > > I had a chance to validate usability of both Ignite JDBC drivers at PGConf > USA today trying to connect to the cluster from the new JetBrains DataGrip > SQL tool [1]. Together with JetBrains folks we agreed on the following or > spotted the issues listed below. > > 1) Default JDBC driver, that connects via a client node using a Spring XML > config, is so counterintuitive and twisted that JetBrains folks couldn’t set > it up w/o my assistance. We force SQL tools users, that accustomed to connect > to DBs by hostname, to pass some weird Spring XML and add up to 15 (!) jars > to the classpath of a tool rather than one. This is huge +1 for the thin > client based driver that requires a hostname only. So, along with the thin > client protocol optimizations DML has to be supported for it as well [2]. > > 2) Regardless of JDBC driver version if you execute a query like “SELECT * > FROM PersonCache” you’ll get a deserialization exception for _val that is > implicitly added to the result set. Luckily, this will be no longer the issue > after this [3] improvement is released in 2.0. > > 3) It would be more user-friendly if the JDBC drivers are packaged under > single “ignite-jdbc.jar” and located in special directory of a release > bundle. All the dependencies have to be in this jar. The ticket is ready [4]. > > So I would encourage us to wrap up this discussion creating additional > tickets in order to renew the thin client based driver and promote it for > tools and non transactional use cases. > > [1] https://www.jetbrains.com/datagrip/ <https://www.jetbrains.com/datagrip/> > [2] https://issues.apache.org/jira/browse/IGNITE-4892 > <https://issues.apache.org/jira/browse/IGNITE-4892> > [3] https://issues.apache.org/jira/browse/IGNITE-3487 > <https://issues.apache.org/jira/browse/IGNITE-3487> > [4] https://issues.apache.org/jira/browse/IGNITE-4893 > <https://issues.apache.org/jira/browse/IGNITE-4893> > > — > Denis > >> On Mar 30, 2017, at 1:09 AM, Denis Magda <dma...@apache.org> wrote: >> >> Val, Igniters, >> >> I still believe the thin client has a right for living. It’s ideal for use >> cases when someone attempts to connect to Ignite from a tool or some sort of >> interface and query the data or update it in non transactional fashion. A >> TCP/IP address as a connection string to the cluster is ideal for this >> scenario. >> >> If someone decides to use JDBC programmatically and leverage from >> transactions then he will switch to the JDBC versions that starts a client >> node with a passed Ignite configuration. >> >> How do you like this? >> >> In general, it sounds like a right movement to replace REST with more >> flexible and, probably, unified protocol for thin client based JDBC >> implementation. But what if we extend REST a bit (supporting pagination for >> SELECTs and DML) at the first phase so that the thin client driver can be >> used right away in the scenarios I listed above? The rest of optimizations >> can be done after that. >> >> — >> Denis >> >>> On Mar 23, 2017, at 7:24 PM, Valentin Kulichenko >>> <valentin.kuliche...@gmail.com> wrote: >>> >>> I'm against keeping legacy thin client because: >>> >>> - Having two ways to configure driver is unnecessary complication and very >>> bad from usability standpoint. >>> - It is much slower than client node, performance was the main driver >>> behind its deprecation. >>> >>> What we should do, is improve usability of the client node, this will be >>> good improvement not only for JDBC driver. The list of required changes was >>> covered earlier in the thread, I will check if we have tickets for all of >>> them and provide a list. >>> >>> -Val >>> >>> On Thu, Mar 16, 2017 at 1:02 AM, Dmitriy Setrakyan <dsetrak...@apache.org> >>> wrote: >>> >>>> Denis, I completely agree that we should have a thin JDBC client. Can you >>>> file a ticket? >>>> >>>> On Wed, Mar 15, 2017 at 4:58 PM, Denis Magda <dma...@apache.org> wrote: >>>> >>>>> Frankly, during these two a couple of day I had a chance to play with Web >>>>> Console and Ignite JDBC driver. >>>>> >>>>> As many other users I referred to JDBC documentation in order to setup >>>> the >>>>> driver properly. However, then due to the recently reported issue I fell >>>>> back to JDBC v 1 (thin client based): >>>>> https://issues.apache.org/jira/browse/IGNITE-4829 < >>>>> https://issues.apache.org/jira/browse/IGNITE-4829> >>>>> >>>>> I was amazed how swift JDBC v 1 and sluggish JDBC v 2 which is default. >>>>> Unfortunately, JDBC v 1 doesn’t support DML this is why I had hard time >>>>> finding out a workaround for IGNITE-4829. But, in any case I fully >>>> support >>>>> reincarnation of JDBC v 1. >>>>> >>>>> *Val*, as one of the most experienced folks who participated in this >>>>> discussion, would you mind wrapping the discussion up in a form of a >>>> ticket >>>>> listing the design proposal? >>>>> >>>>> — >>>>> Denis >>>>> >>>>>> On Feb 10, 2017, at 4:47 PM, Dmitriy Setrakyan <dsetrak...@apache.org> >>>>> wrote: >>>>>> >>>>>> I generally like the idea of supporting thin JDBC clients. Often it is >>>>> not >>>>>> feasible to start a client node on JDBC side. >>>>>> >>>>>> On Fri, Feb 10, 2017 at 1:04 PM, Valentin Kulichenko < >>>>>> valentin.kuliche...@gmail.com> wrote: >>>>>> >>>>>>> Yakov, >>>>>>> >>>>>>> There are two separate aspects that we can improve: >>>>>>> >>>>>>> 1. Message routing as you described. I agree it can be useful in some >>>>>>> scenarios and I definitely not against this feature, but honestly I >>>>> haven't >>>>>>> seen a lot of requests for this so far. >>>>>>> 2. Server can initiate connection with client. This really blows >>>> users's >>>>>>> minds :) >>>>>>> >>>>>>> I only meant that the latter is much more critical in my view. >>>>> Configuring >>>>>>> port forwarding on the cluster can be complicated, but it absolutely >>>>> makes >>>>>>> sense, while doing the same on client just sounds crazy. Client should >>>>> be >>>>>>> able to just connect, without having public IP and without additional >>>>>>> configuration on the router. >>>>>>> >>>>>>> -Val >>>>>>> >>>>>>> On Fri, Feb 10, 2017 at 1:58 AM, Yakov Zhdanov <yzhda...@apache.org> >>>>>>> wrote: >>>>>>> >>>>>>>> Val, can you please explain your statement. You suggest users have >>>> port >>>>>>>> forwarding for dozens of nodes in their topologies so client can >>>>> initiate >>>>>>>> connect to any of them? I don't think this is possible and routing >>>>> seems >>>>>>> to >>>>>>>> be the only possible option. >>>>>>>> >>>>>>>> --Yakov >>>>>>>> >>>>>>>> 2017-02-10 0:04 GMT+03:00 Valentin Kulichenko < >>>>>>>> valentin.kuliche...@gmail.com >>>>>>>>> : >>>>>>>> >>>>>>>>> Yakov, >>>>>>>>> >>>>>>>>> 1. Yes, I will file a ticket. >>>>>>>>> >>>>>>>>> 4. I meant that server can currently initiate connection with >>>> client, >>>>>>> and >>>>>>>>> that's the main problem here. Is there a way to avoid this? Message >>>>>>>> routing >>>>>>>>> you're referring to can also be useful in some cases, but much less >>>>>>>>> critical. >>>>>>>>> >>>>>>>>> -Val >>>>>>>>> >>>>>>>>> On Wed, Feb 8, 2017 at 9:20 PM, Yakov Zhdanov < >>>> yzhda...@gridgain.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Val, >>>>>>>>>> >>>>>>>>>> 1. Our clients should stop require persistent store implementation >>>> if >>>>>>>>> they >>>>>>>>>> do not need it. Can you please file a ticket? I know you fixed some >>>>>>>>> places >>>>>>>>>> already. As an idea I would keep everything in binary format until >>>> we >>>>>>>>>> really need it. Will that work? >>>>>>>>>> 2. We can try adding the very first step to fetch the configuration >>>>>>> and >>>>>>>>>> then proceed with normal start. >>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4675 >>>>>>>>>> 3. Agree, but user needs to define the closures then. I would think >>>>>>> on >>>>>>>>> how >>>>>>>>>> to put this to a product. >>>>>>>>>> 4. This needs to be implemented :) I mean we can communicate to a >>>>>>>> client >>>>>>>>>> through server it is connected to. >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> -- >>>>>>>>>> Yakov Zhdanov, Director R&D >>>>>>>>>> *GridGain Systems* >>>>>>>>>> www.gridgain.com >>>>>>>>>> >>>>>>>>>> 2017-02-09 5:19 GMT+07:00 Valentin Kulichenko < >>>>>>>>>> valentin.kuliche...@gmail.com >>>>>>>>>>> : >>>>>>>>>> >>>>>>>>>>> Yakov, >>>>>>>>>>> >>>>>>>>>>> I agree that investing in the legacy client doesn't make sense - >>>>>>> it's >>>>>>>>>> slow >>>>>>>>>>> and outdated. Regarding your points: >>>>>>>>>>> >>>>>>>>>>> 1. This is just another build step, but the JAR is going to pretty >>>>>>>> fat >>>>>>>>> I >>>>>>>>>>> think (it will have to include Spring). Not ideal, but definitely >>>>>>>>> better >>>>>>>>>>> than what we have now. However, our clients also often require >>>> some >>>>>>>>> user >>>>>>>>>>> classes, like CacheStore implementations. This is also a problem. >>>>>>>>>>> >>>>>>>>>>> 2. That's a great idea! Actually, I'm not sure why we require to >>>>>>> have >>>>>>>>>> full >>>>>>>>>>> verbose config on client that is consistent with server. Why not >>>>>>>> fetch >>>>>>>>>> the >>>>>>>>>>> configuration from cluster during join? Not sure how hard this >>>>>>> change >>>>>>>>> is, >>>>>>>>>>> but it can be a very big usability improvement. And surely, JDBC >>>>>>>> driver >>>>>>>>>>> should be able to config with host:port without config file. >>>>>>>>>>> >>>>>>>>>>> 3. This can be already achieved with Compute Grid, no? I don't >>>>>>> think >>>>>>>> we >>>>>>>>>>> need to add anything here. >>>>>>>>>>> >>>>>>>>>>> Another issue with clients is that they currently can't work >>>> behind >>>>>>>> NAT >>>>>>>>>>> without additional config which is not very trivial >>>>>>>> (AddressResolver). >>>>>>>>> Is >>>>>>>>>>> it possible to avoid server->client connections in communication >>>>>>> SPI? >>>>>>>>>>> >>>>>>>>>>> -Val >>>>>>>>>>> >>>>>>>>>>> On Tue, Feb 7, 2017 at 9:09 PM, Yakov Zhdanov < >>>> yzhda...@apache.org >>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> "undeprecating" - lol :D >>>>>>>>>>>> Consider introducing @Un annotation which negates all annotations >>>>>>>> on >>>>>>>>>> the >>>>>>>>>>>> same level and below. >>>>>>>>>>>> >>>>>>>>>>>> I would probably agree with Val and Vova, but adding features to >>>>>>>>>>>> thin-client seems questionable to me. >>>>>>>>>>>> >>>>>>>>>>>> Is these possible: >>>>>>>>>>>> 1. avoid dependencies on client machine and require >>>>>>> ignite-jdbc.jar >>>>>>>>>> only >>>>>>>>>>>> (e.g. gathering dependencies inside the jar). >>>>>>>>>>>> 2. make it possible to provide just address and port to send join >>>>>>>>>> request >>>>>>>>>>>> to without providing the entire IgniteConfiguration. Client node >>>>>>>>> sends >>>>>>>>>>> join >>>>>>>>>>>> request to the cluster with flag that this is jdbc-driver >>>>>>>> connection >>>>>>>>>> and >>>>>>>>>>>> server-side topology omits configuration validation and forces >>>>>>>> client >>>>>>>>>> to >>>>>>>>>>>> set some properties if this is necessary (e.g. CommunicationSpi >>>>>>>>>>>> implementation class and settings) >>>>>>>>>>>> 3. add possibility to offload complex reduce processing to >>>>>>> server. >>>>>>>>>> Which >>>>>>>>>>>> may be very helpful for main client-server use cases when clients >>>>>>>>> being >>>>>>>>>>> run >>>>>>>>>>>> on much weaker machines. >>>>>>>>>>>> >>>>>>>>>>>> ? >>>>>>>>>>>> >>>>>>>>>>>> --Yakov >>>>>>>>>>>> >>>>>>>>>>>> 2017-02-07 14:30 GMT+07:00 Vladimir Ozerov <voze...@gridgain.com >>>>>>>> : >>>>>>>>>>>> >>>>>>>>>>>>> Big +1 to Val, not only about JDBC, but about our overall >>>>>>>> approach >>>>>>>>> to >>>>>>>>>>>>> clients. Starting a node with "client=true" is: >>>>>>>>>>>>> + Very reach feature set, which is cool >>>>>>>>>>>>> - Tons of dependencies >>>>>>>>>>>>> - Tons of threads >>>>>>>>>>>>> >>>>>>>>>>>>> It would be very cool if we have a true thin client with small >>>>>>>>> single >>>>>>>>>>>> JAR. >>>>>>>>>>>>> It should have: >>>>>>>>>>>>> - Failover >>>>>>>>>>>>> - Load-balance >>>>>>>>>>>>> - Optional server "stickyness" >>>>>>>>>>>>> >>>>>>>>>>>>> Once all these things are in place we will be able to provide >>>>>>> the >>>>>>>>>> same >>>>>>>>>>>> API >>>>>>>>>>>>> as in current client, but with predictable behavior and memory >>>>>>>>>>> footprint. >>>>>>>>>>>>> For instance our current client is not well-suited for running >>>>>>>>>>> map-reduce >>>>>>>>>>>>> (compute or SQL) because it moves large amount of data and >>>>>>>>> processing >>>>>>>>>>> to >>>>>>>>>>>>> the client, which is potentially a slow desktop machine. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Feb 7, 2017 at 3:44 AM, Valentin Kulichenko < >>>>>>>>>>>>> valentin.kuliche...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> There are two implementations of JDBC driver - based on >>>>>>> legacy >>>>>>>>> thin >>>>>>>>>>>>> client >>>>>>>>>>>>>> (jdbc package) and on client node (jdbc2). The first one was >>>>>>>>>>> deprecated >>>>>>>>>>>>>> when we introduced the latter, but now I tend to think that >>>>>>>> this >>>>>>>>>> was >>>>>>>>>>>> not >>>>>>>>>>>>> a >>>>>>>>>>>>>> right decision. Thin client driver provides worse >>>>>>> performance, >>>>>>>>> but >>>>>>>>>>> it's >>>>>>>>>>>>>> much easier to use, never requires additional dependencies >>>>>>> like >>>>>>>>>>> Spring >>>>>>>>>>>>> and >>>>>>>>>>>>>> can be used from any remote machine. Probably we can consider >>>>>>>>>>>>> undeprecating >>>>>>>>>>>>>> it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -Val >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Feb 6, 2017 at 2:02 AM, Sergi Vladykin < >>>>>>>>>>>> sergi.vlady...@gmail.com >>>>>>>>>>>>>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Guys, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We have 2 different packages: jdbc and jdbc2. Everything in >>>>>>>>> jdbc >>>>>>>>>> is >>>>>>>>>>>>>>> deprecated. Because of that new features like DML support >>>>>>>> were >>>>>>>>>> not >>>>>>>>>>>>> added >>>>>>>>>>>>>>> there. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This seems to cause some problems to our users. Can someone >>>>>>>>>>> clarify, >>>>>>>>>>>>> did >>>>>>>>>>>>>> we >>>>>>>>>>>>>>> deprecated these classes wrongly and we have to continue >>>>>>>>>> developing >>>>>>>>>>>>> them >>>>>>>>>>>>>> or >>>>>>>>>>>>>>> what? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Sergi >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>>> >> >