Folks, I had a chance to validate usability of both Ignite JDBC drivers at PGConf USA today trying to connect to the cluster from the new JetBrains DataGrip SQL tool [1]. Together with JetBrains folks we agreed on the following or spotted the issues listed below.
1) Default JDBC driver, that connects via a client node using a Spring XML config, is so counterintuitive and twisted that JetBrains folks couldn’t set it up w/o my assistance. We force SQL tools users, that accustomed to connect to DBs by hostname, to pass some weird Spring XML and add up to 15 (!) jars to the classpath of a tool rather than one. This is huge +1 for the thin client based driver that requires a hostname only. So, along with the thin client protocol optimizations DML has to be supported for it as well [2]. 2) Regardless of JDBC driver version if you execute a query like “SELECT * FROM PersonCache” you’ll get a deserialization exception for _val that is implicitly added to the result set. Luckily, this will be no longer the issue after this [3] improvement is released in 2.0. 3) It would be more user-friendly if the JDBC drivers are packaged under single “ignite-jdbc.jar” and located in special directory of a release bundle. All the dependencies have to be in this jar. The ticket is ready [4]. So I would encourage us to wrap up this discussion creating additional tickets in order to renew the thin client based driver and promote it for tools and non transactional use cases. [1] https://www.jetbrains.com/datagrip/ <https://www.jetbrains.com/datagrip/> [2] https://issues.apache.org/jira/browse/IGNITE-4892 <https://issues.apache.org/jira/browse/IGNITE-4892> [3] https://issues.apache.org/jira/browse/IGNITE-3487 <https://issues.apache.org/jira/browse/IGNITE-3487> [4] https://issues.apache.org/jira/browse/IGNITE-4893 <https://issues.apache.org/jira/browse/IGNITE-4893> — Denis > On Mar 30, 2017, at 1:09 AM, Denis Magda <dma...@apache.org> wrote: > > Val, Igniters, > > I still believe the thin client has a right for living. It’s ideal for use > cases when someone attempts to connect to Ignite from a tool or some sort of > interface and query the data or update it in non transactional fashion. A > TCP/IP address as a connection string to the cluster is ideal for this > scenario. > > If someone decides to use JDBC programmatically and leverage from > transactions then he will switch to the JDBC versions that starts a client > node with a passed Ignite configuration. > > How do you like this? > > In general, it sounds like a right movement to replace REST with more > flexible and, probably, unified protocol for thin client based JDBC > implementation. But what if we extend REST a bit (supporting pagination for > SELECTs and DML) at the first phase so that the thin client driver can be > used right away in the scenarios I listed above? The rest of optimizations > can be done after that. > > — > Denis > >> On Mar 23, 2017, at 7:24 PM, Valentin Kulichenko >> <valentin.kuliche...@gmail.com> wrote: >> >> I'm against keeping legacy thin client because: >> >> - Having two ways to configure driver is unnecessary complication and very >> bad from usability standpoint. >> - It is much slower than client node, performance was the main driver >> behind its deprecation. >> >> What we should do, is improve usability of the client node, this will be >> good improvement not only for JDBC driver. The list of required changes was >> covered earlier in the thread, I will check if we have tickets for all of >> them and provide a list. >> >> -Val >> >> On Thu, Mar 16, 2017 at 1:02 AM, Dmitriy Setrakyan <dsetrak...@apache.org> >> wrote: >> >>> Denis, I completely agree that we should have a thin JDBC client. Can you >>> file a ticket? >>> >>> On Wed, Mar 15, 2017 at 4:58 PM, Denis Magda <dma...@apache.org> wrote: >>> >>>> Frankly, during these two a couple of day I had a chance to play with Web >>>> Console and Ignite JDBC driver. >>>> >>>> As many other users I referred to JDBC documentation in order to setup >>> the >>>> driver properly. However, then due to the recently reported issue I fell >>>> back to JDBC v 1 (thin client based): >>>> https://issues.apache.org/jira/browse/IGNITE-4829 < >>>> https://issues.apache.org/jira/browse/IGNITE-4829> >>>> >>>> I was amazed how swift JDBC v 1 and sluggish JDBC v 2 which is default. >>>> Unfortunately, JDBC v 1 doesn’t support DML this is why I had hard time >>>> finding out a workaround for IGNITE-4829. But, in any case I fully >>> support >>>> reincarnation of JDBC v 1. >>>> >>>> *Val*, as one of the most experienced folks who participated in this >>>> discussion, would you mind wrapping the discussion up in a form of a >>> ticket >>>> listing the design proposal? >>>> >>>> — >>>> Denis >>>> >>>>> On Feb 10, 2017, at 4:47 PM, Dmitriy Setrakyan <dsetrak...@apache.org> >>>> wrote: >>>>> >>>>> I generally like the idea of supporting thin JDBC clients. Often it is >>>> not >>>>> feasible to start a client node on JDBC side. >>>>> >>>>> On Fri, Feb 10, 2017 at 1:04 PM, Valentin Kulichenko < >>>>> valentin.kuliche...@gmail.com> wrote: >>>>> >>>>>> Yakov, >>>>>> >>>>>> There are two separate aspects that we can improve: >>>>>> >>>>>> 1. Message routing as you described. I agree it can be useful in some >>>>>> scenarios and I definitely not against this feature, but honestly I >>>> haven't >>>>>> seen a lot of requests for this so far. >>>>>> 2. Server can initiate connection with client. This really blows >>> users's >>>>>> minds :) >>>>>> >>>>>> I only meant that the latter is much more critical in my view. >>>> Configuring >>>>>> port forwarding on the cluster can be complicated, but it absolutely >>>> makes >>>>>> sense, while doing the same on client just sounds crazy. Client should >>>> be >>>>>> able to just connect, without having public IP and without additional >>>>>> configuration on the router. >>>>>> >>>>>> -Val >>>>>> >>>>>> On Fri, Feb 10, 2017 at 1:58 AM, Yakov Zhdanov <yzhda...@apache.org> >>>>>> wrote: >>>>>> >>>>>>> Val, can you please explain your statement. You suggest users have >>> port >>>>>>> forwarding for dozens of nodes in their topologies so client can >>>> initiate >>>>>>> connect to any of them? I don't think this is possible and routing >>>> seems >>>>>> to >>>>>>> be the only possible option. >>>>>>> >>>>>>> --Yakov >>>>>>> >>>>>>> 2017-02-10 0:04 GMT+03:00 Valentin Kulichenko < >>>>>>> valentin.kuliche...@gmail.com >>>>>>>> : >>>>>>> >>>>>>>> Yakov, >>>>>>>> >>>>>>>> 1. Yes, I will file a ticket. >>>>>>>> >>>>>>>> 4. I meant that server can currently initiate connection with >>> client, >>>>>> and >>>>>>>> that's the main problem here. Is there a way to avoid this? Message >>>>>>> routing >>>>>>>> you're referring to can also be useful in some cases, but much less >>>>>>>> critical. >>>>>>>> >>>>>>>> -Val >>>>>>>> >>>>>>>> On Wed, Feb 8, 2017 at 9:20 PM, Yakov Zhdanov < >>> yzhda...@gridgain.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Val, >>>>>>>>> >>>>>>>>> 1. Our clients should stop require persistent store implementation >>> if >>>>>>>> they >>>>>>>>> do not need it. Can you please file a ticket? I know you fixed some >>>>>>>> places >>>>>>>>> already. As an idea I would keep everything in binary format until >>> we >>>>>>>>> really need it. Will that work? >>>>>>>>> 2. We can try adding the very first step to fetch the configuration >>>>>> and >>>>>>>>> then proceed with normal start. >>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4675 >>>>>>>>> 3. Agree, but user needs to define the closures then. I would think >>>>>> on >>>>>>>> how >>>>>>>>> to put this to a product. >>>>>>>>> 4. This needs to be implemented :) I mean we can communicate to a >>>>>>> client >>>>>>>>> through server it is connected to. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> -- >>>>>>>>> Yakov Zhdanov, Director R&D >>>>>>>>> *GridGain Systems* >>>>>>>>> www.gridgain.com >>>>>>>>> >>>>>>>>> 2017-02-09 5:19 GMT+07:00 Valentin Kulichenko < >>>>>>>>> valentin.kuliche...@gmail.com >>>>>>>>>> : >>>>>>>>> >>>>>>>>>> Yakov, >>>>>>>>>> >>>>>>>>>> I agree that investing in the legacy client doesn't make sense - >>>>>> it's >>>>>>>>> slow >>>>>>>>>> and outdated. Regarding your points: >>>>>>>>>> >>>>>>>>>> 1. This is just another build step, but the JAR is going to pretty >>>>>>> fat >>>>>>>> I >>>>>>>>>> think (it will have to include Spring). Not ideal, but definitely >>>>>>>> better >>>>>>>>>> than what we have now. However, our clients also often require >>> some >>>>>>>> user >>>>>>>>>> classes, like CacheStore implementations. This is also a problem. >>>>>>>>>> >>>>>>>>>> 2. That's a great idea! Actually, I'm not sure why we require to >>>>>> have >>>>>>>>> full >>>>>>>>>> verbose config on client that is consistent with server. Why not >>>>>>> fetch >>>>>>>>> the >>>>>>>>>> configuration from cluster during join? Not sure how hard this >>>>>> change >>>>>>>> is, >>>>>>>>>> but it can be a very big usability improvement. And surely, JDBC >>>>>>> driver >>>>>>>>>> should be able to config with host:port without config file. >>>>>>>>>> >>>>>>>>>> 3. This can be already achieved with Compute Grid, no? I don't >>>>>> think >>>>>>> we >>>>>>>>>> need to add anything here. >>>>>>>>>> >>>>>>>>>> Another issue with clients is that they currently can't work >>> behind >>>>>>> NAT >>>>>>>>>> without additional config which is not very trivial >>>>>>> (AddressResolver). >>>>>>>> Is >>>>>>>>>> it possible to avoid server->client connections in communication >>>>>> SPI? >>>>>>>>>> >>>>>>>>>> -Val >>>>>>>>>> >>>>>>>>>> On Tue, Feb 7, 2017 at 9:09 PM, Yakov Zhdanov < >>> yzhda...@apache.org >>>>>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> "undeprecating" - lol :D >>>>>>>>>>> Consider introducing @Un annotation which negates all annotations >>>>>>> on >>>>>>>>> the >>>>>>>>>>> same level and below. >>>>>>>>>>> >>>>>>>>>>> I would probably agree with Val and Vova, but adding features to >>>>>>>>>>> thin-client seems questionable to me. >>>>>>>>>>> >>>>>>>>>>> Is these possible: >>>>>>>>>>> 1. avoid dependencies on client machine and require >>>>>> ignite-jdbc.jar >>>>>>>>> only >>>>>>>>>>> (e.g. gathering dependencies inside the jar). >>>>>>>>>>> 2. make it possible to provide just address and port to send join >>>>>>>>> request >>>>>>>>>>> to without providing the entire IgniteConfiguration. Client node >>>>>>>> sends >>>>>>>>>> join >>>>>>>>>>> request to the cluster with flag that this is jdbc-driver >>>>>>> connection >>>>>>>>> and >>>>>>>>>>> server-side topology omits configuration validation and forces >>>>>>> client >>>>>>>>> to >>>>>>>>>>> set some properties if this is necessary (e.g. CommunicationSpi >>>>>>>>>>> implementation class and settings) >>>>>>>>>>> 3. add possibility to offload complex reduce processing to >>>>>> server. >>>>>>>>> Which >>>>>>>>>>> may be very helpful for main client-server use cases when clients >>>>>>>> being >>>>>>>>>> run >>>>>>>>>>> on much weaker machines. >>>>>>>>>>> >>>>>>>>>>> ? >>>>>>>>>>> >>>>>>>>>>> --Yakov >>>>>>>>>>> >>>>>>>>>>> 2017-02-07 14:30 GMT+07:00 Vladimir Ozerov <voze...@gridgain.com >>>>>>> : >>>>>>>>>>> >>>>>>>>>>>> Big +1 to Val, not only about JDBC, but about our overall >>>>>>> approach >>>>>>>> to >>>>>>>>>>>> clients. Starting a node with "client=true" is: >>>>>>>>>>>> + Very reach feature set, which is cool >>>>>>>>>>>> - Tons of dependencies >>>>>>>>>>>> - Tons of threads >>>>>>>>>>>> >>>>>>>>>>>> It would be very cool if we have a true thin client with small >>>>>>>> single >>>>>>>>>>> JAR. >>>>>>>>>>>> It should have: >>>>>>>>>>>> - Failover >>>>>>>>>>>> - Load-balance >>>>>>>>>>>> - Optional server "stickyness" >>>>>>>>>>>> >>>>>>>>>>>> Once all these things are in place we will be able to provide >>>>>> the >>>>>>>>> same >>>>>>>>>>> API >>>>>>>>>>>> as in current client, but with predictable behavior and memory >>>>>>>>>> footprint. >>>>>>>>>>>> For instance our current client is not well-suited for running >>>>>>>>>> map-reduce >>>>>>>>>>>> (compute or SQL) because it moves large amount of data and >>>>>>>> processing >>>>>>>>>> to >>>>>>>>>>>> the client, which is potentially a slow desktop machine. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Feb 7, 2017 at 3:44 AM, Valentin Kulichenko < >>>>>>>>>>>> valentin.kuliche...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> There are two implementations of JDBC driver - based on >>>>>> legacy >>>>>>>> thin >>>>>>>>>>>> client >>>>>>>>>>>>> (jdbc package) and on client node (jdbc2). The first one was >>>>>>>>>> deprecated >>>>>>>>>>>>> when we introduced the latter, but now I tend to think that >>>>>>> this >>>>>>>>> was >>>>>>>>>>> not >>>>>>>>>>>> a >>>>>>>>>>>>> right decision. Thin client driver provides worse >>>>>> performance, >>>>>>>> but >>>>>>>>>> it's >>>>>>>>>>>>> much easier to use, never requires additional dependencies >>>>>> like >>>>>>>>>> Spring >>>>>>>>>>>> and >>>>>>>>>>>>> can be used from any remote machine. Probably we can consider >>>>>>>>>>>> undeprecating >>>>>>>>>>>>> it. >>>>>>>>>>>>> >>>>>>>>>>>>> -Val >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Feb 6, 2017 at 2:02 AM, Sergi Vladykin < >>>>>>>>>>> sergi.vlady...@gmail.com >>>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Guys, >>>>>>>>>>>>>> >>>>>>>>>>>>>> We have 2 different packages: jdbc and jdbc2. Everything in >>>>>>>> jdbc >>>>>>>>> is >>>>>>>>>>>>>> deprecated. Because of that new features like DML support >>>>>>> were >>>>>>>>> not >>>>>>>>>>>> added >>>>>>>>>>>>>> there. >>>>>>>>>>>>>> >>>>>>>>>>>>>> This seems to cause some problems to our users. Can someone >>>>>>>>>> clarify, >>>>>>>>>>>> did >>>>>>>>>>>>> we >>>>>>>>>>>>>> deprecated these classes wrongly and we have to continue >>>>>>>>> developing >>>>>>>>>>>> them >>>>>>>>>>>>> or >>>>>>>>>>>>>> what? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sergi >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>>> >>> >
