Ivan, We are not going to adopt C++20 in the near future, as new standards are not adopted as fast in C++ world and if we'd do that most probably almost none of our users could use our client.
But yeah, if we'd use coroutines, I'd probably suggest that we only provide the async API. Best Regards, Igor On Thu, Sep 9, 2021 at 2:29 PM Ivan Daschinsky <ivanda...@gmail.com> wrote: > Igor, and what about C++20 and coroutines [1] > > [1] -- https://en.cppreference.com/w/cpp/language/coroutines > > чт, 9 сент. 2021 г. в 14:12, Igor Sapego <isap...@apache.org>: > > > Well, fortunately we do not provide a C client if you don't consider ODBC > > as one so we should not think about it. For C++ I believe we should use > > standard std::future+std::promise for async, but still can provide sync > > methods, > > built on top of async methods. There is no continuation problem in C++ > API > > as > > std::future do not provide continuation methods :) > > > > Now, for the sync/async problem you've mentioned. Can not we solve it by > > using different thread pools for completion and continuation of futures > or > > does > > CompletableFuture interface not allowing for that? > > > > Best Regards, > > Igor > > > > > > On Thu, Sep 9, 2021 at 1:15 PM Ivan Daschinsky <ivanda...@gmail.com> > > wrote: > > > > > And what about C/C++? There are so many options for them... > > > > > > чт, 9 сент. 2021 г. в 13:00, Pavel Tupitsyn <ptupit...@apache.org>: > > > > > > > Talked to Ivan in private, and came up with the following > per-language > > > > summary: > > > > > > > > * Java: sync and async > > > > *. NET: only async > > > > * Python: sync and async > > > > * JavaScript: only async (sync is not possible) > > > > > > > > Thin clients in other languages are to be discussed. > > > > > > > > On Thu, Sep 9, 2021 at 11:49 AM Ivan Daschinsky <ivanda...@gmail.com > > > > > > wrote: > > > > > > > > > >> You can mix them easily. > > > > > This is far from easily (you have already mentioned continuation > > > > problem), > > > > > but for i.e. in python it is absolutely not. > > > > > For kotlin it is a little bit easier, but also not fluent and a > > little > > > > bit > > > > > ugly. > > > > > > > > > > чт, 9 сент. 2021 г. в 11:37, Pavel Tupitsyn <ptupit...@apache.org > >: > > > > > > > > > > > Ivan, > > > > > > > > > > > > > Pavel, is it really true, that in .NET sync versions of > libraries > > > and > > > > > > tools > > > > > > > are completely eliminated? > > > > > > > > > > > > Far from being eliminated, but there is a movement in this > > direction. > > > > > > For example, HttpClient [1] only provides async variants for most > > of > > > > the > > > > > > methods. > > > > > > > > > > > > Exposing sync wrappers for async methods is not recommended [2]. > > > > > > There is a good explanation of potential threading issues there, > > and > > > it > > > > > can > > > > > > be applied to Java too. > > > > > > > > > > > > > > > > > > > you cannot mix both functions easily > > > > > > > > > > > > You can mix them easily. > > > > > > C#: table.GetAsync(k).Result or > > > > > table.GetAsync(k).GetAwaiter().GetResult() > > > > > > (different exception handling) > > > > > > Java: table.getAsync(k).get() > > > > > > > > > > > > The same thing we do in sync wrapper for async method is now in > the > > > > user > > > > > > code. > > > > > > Which is better for two reasons: > > > > > > - users won't accidentally use sync API when async API should be > > used > > > > > > - when they really have to use sync API, potentially dangerous > > > behavior > > > > > is > > > > > > not hidden > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=net-5.0#methods > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > > > > https://devblogs.microsoft.com/pfxteam/should-i-expose-synchronous-wrappers-for-asynchronous-methods/ > > > > > > > > > > > > On Thu, Sep 9, 2021 at 11:16 AM Ivan Daschinsky < > > ivanda...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > >> 2. In languages with proper async support (async-await, > etc.), > > > we > > > > > can > > > > > > > skip sync API altogether. > > > > > > > It sounds pretty strange, as for me. Usually you cannot mix > both > > > > > > functions > > > > > > > easily, it is called blue-red functions problem [1] > > > > > > > In python you definitely cannot do sync over async, for example > > > > > > > (principally can, but nobody do that because of mediocre > > > performance > > > > of > > > > > > > that solution). And many developers > > > > > > > still prefer writing code withou asyncio at all. > > > > > > > > > > > > > > Pavel, is it really true, that in .NET sync versions of > libraries > > > and > > > > > > tools > > > > > > > are completely eliminated? > > > > > > > > > > > > > > [1] -- > > > > > > > > > > > > > > > > https://elizarov.medium.com/how-do-you-color-your-functions-a6bb423d936d > > > > > > > > > > > > > > ср, 8 сент. 2021 г. в 22:33, Pavel Tupitsyn < > > ptupit...@apache.org > > > >: > > > > > > > > > > > > > > > To put it another way: > > > > > > > > - true sync operation completes by itself > > > > > > > > - sync-over-async operation requires another thread to > complete > > > > > > > > > > > > > > > > On Wed, Sep 8, 2021 at 10:15 PM Pavel Tupitsyn < > > > > ptupit...@apache.org > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Val, > > > > > > > > > > > > > > > > > > That's exactly what I have in mind. > > > > > > > > > Yes, we block the user thread, but then we should unblock > it > > by > > > > > > > > completing > > > > > > > > > the future. > > > > > > > > > We can't complete the future from a Netty thread [1], so > > we'll > > > > need > > > > > > > some > > > > > > > > > other thread from some executor. > > > > > > > > > If there are no threads available (because they are blocked > > by > > > > the > > > > > > sync > > > > > > > > > API above), the future won't complete => deadlock. > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/r528659381d983a177d779f56ef3d7da6fe17eb3504383f5f87727514%40%3Cdev.ignite.apache.org%3E > > > > > > > > > > > > > > > > > > On Wed, Sep 8, 2021 at 9:40 PM Valentin Kulichenko < > > > > > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > > > > > > >> Pavel, > > > > > > > > >> > > > > > > > > >> I might be missing something - could you please elaborate > a > > > > little > > > > > > > more? > > > > > > > > >> > > > > > > > > >> When I say "sync on top of async", I basically mean that > > (for > > > > > > example) > > > > > > > > >> 'insert(..)' is equivalent to 'insertAsync(..).join()'. In > > my > > > > > > > > >> understanding, it only blocks the user's thread. > > > > > > > > >> > > > > > > > > >> Am I wrong? Or you have a different implementation in > mind? > > > > > > > > >> > > > > > > > > >> -Val > > > > > > > > >> > > > > > > > > >> On Wed, Sep 8, 2021 at 12:50 AM Pavel Tupitsyn < > > > > > > ptupit...@apache.org> > > > > > > > > >> wrote: > > > > > > > > >> > > > > > > > > >> > Val, > > > > > > > > >> > > > > > > > > > >> > Agree with your points. > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > async API should be primary > > > > > > > > >> > > > > > > > > > >> > It should be noted that all our APIs are inherently > async, > > > > > > > > >> > because thin client is implemented asynchronously. > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > with the sync version build on top > > > > > > > > >> > > > > > > > > > >> > We should document somehow that sync APIs are based on > > async > > > > > ones, > > > > > > > > >> > because this may be dangerous in some use cases. > > > > > > > > >> > > > > > > > > > >> > For example, as a user, I may have a thread pool of 4 > > > threads > > > > > for > > > > > > > > >> > Ignite-related usage, that is also set as > > > > > > asyncContinuationExecutor > > > > > > > > [1]. > > > > > > > > >> > Now if I run a lot of concurrent Ignite requests using > > sync > > > > API, > > > > > > > all 4 > > > > > > > > >> > threads will end up blocked on CompletableFutures. > > > > > > > > >> > When one of the operations completes, we enqueue the > > > > completion > > > > > to > > > > > > > > that > > > > > > > > >> > same thread pool, but all threads are blocked on sync > > APIs, > > > > > > > resulting > > > > > > > > >> in a > > > > > > > > >> > deadlock. > > > > > > > > >> > > > > > > > > > >> > This can be prevented by using a different > > > > > > > asyncContinuationExecutor, > > > > > > > > >> but > > > > > > > > >> > sync API users won't be usually aware of this. > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > [1] https://issues.apache.org/jira/browse/IGNITE-15359 > > > > > > > > >> > > > > > > > > > >> > On Wed, Sep 8, 2021 at 10:04 AM Courtney Robinson < > > > > > > > > >> > courtney.robin...@hypi.io> > > > > > > > > >> > wrote: > > > > > > > > >> > > > > > > > > > >> > > Hi Val, > > > > > > > > >> > > > > > > > > > > >> > > I'd highly support an async first API based on > > > > CompletionStage > > > > > > > > >> > > < > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletionStage.html > > > > > > > > >> > > > > > > > > > > > >> > > or > > > > > > > > >> > > its subtypes like CompletableFuture. > > > > > > > > >> > > In Ignite 2 we've written a wrapper library around > > > > > IgniteFuture > > > > > > to > > > > > > > > >> > provide > > > > > > > > >> > > CompletionStage instead because many of the newer libs > > we > > > > use > > > > > > > > support > > > > > > > > >> > this. > > > > > > > > >> > > If Ignite 3 went this way it'd remove a lot of boiler > > > > > > > plate/wrapper > > > > > > > > >> that > > > > > > > > >> > we > > > > > > > > >> > > wrote to get what you're suggesting here. > > > > > > > > >> > > > > > > > > > > >> > > Regards, > > > > > > > > >> > > Courtney Robinson > > > > > > > > >> > > Founder and CEO, Hypi > > > > > > > > >> > > Tel: ++44 208 123 2413 (GMT+0) <https://hypi.io> > > > > > > > > >> > > > > > > > > > > >> > > <https://hypi.io> > > > > > > > > >> > > https://hypi.io > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > On Wed, Sep 8, 2021 at 12:44 AM Valentin Kulichenko < > > > > > > > > >> > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > >> > > > > > > > > > > >> > > > Igniters, > > > > > > > > >> > > > > > > > > > > > >> > > > I would like to gather some opinions on whether we > > want > > > to > > > > > > focus > > > > > > > > on > > > > > > > > >> > sync > > > > > > > > >> > > vs > > > > > > > > >> > > > async APIs in Ignite 3. > > > > > > > > >> > > > > > > > > > > > >> > > > Here are some initial considerations that I have: > > > > > > > > >> > > > 1. Ignite 2.x is essentially "sync first". Async > APIs > > > > exist, > > > > > > but > > > > > > > > >> they > > > > > > > > >> > use > > > > > > > > >> > > > non-standard IgniteFuture and provide > counterintuitive > > > > > > > guarantees. > > > > > > > > >> In > > > > > > > > >> > my > > > > > > > > >> > > > experience, they significantly lack usability, and > > > because > > > > > of > > > > > > > that > > > > > > > > >> are > > > > > > > > >> > > > rarely used. > > > > > > > > >> > > > 2. In general, however, async execution becomes more > > and > > > > > more > > > > > > > > >> > prominent. > > > > > > > > >> > > > Something we can't ignore if we want to create a > > modern > > > > > > > framework. > > > > > > > > >> > > > 3. Still, async support in Java is very limited > > > > (especially > > > > > if > > > > > > > > >> compared > > > > > > > > >> > > to > > > > > > > > >> > > > other languages, like C# for example). > > > > > > > > >> > > > > > > > > > > > >> > > > My current position is the following (happy to > > discuss): > > > > > > > > >> > > > 1. We should pay more attention to async APIs. As a > > > > general > > > > > > > rule, > > > > > > > > >> async > > > > > > > > >> > > API > > > > > > > > >> > > > should be primary, with the sync version build on > top. > > > > > > > > >> > > > 2. In languages with proper async support > > (async-await, > > > > > etc.), > > > > > > > we > > > > > > > > >> can > > > > > > > > >> > > skip > > > > > > > > >> > > > sync API altogether. As an example of this, you can > > look > > > > at > > > > > > the > > > > > > > > >> first > > > > > > > > >> > > > version of the .NET client [1]. It exposes only > async > > > > > methods, > > > > > > > and > > > > > > > > >> it > > > > > > > > >> > > > doesn't look like sync counterparts are really > needed. > > > > > > > > >> > > > 3. In Java (as well as other languages where > > > applicable), > > > > we > > > > > > > will > > > > > > > > >> add > > > > > > > > >> > > sync > > > > > > > > >> > > > APIs that simply delegate to async APIs. This will > > help > > > > > users > > > > > > to > > > > > > > > >> avoid > > > > > > > > >> > > > CompletableFuture if they don't want to use it. > > > > > > > > >> > > > > > > > > > > > >> > > > [1] https://github.com/apache/ignite-3/pull/306 > > > > > > > > >> > > > > > > > > > > > >> > > > Please share your thoughts. > > > > > > > > >> > > > > > > > > > > > >> > > > -Val > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > > > > > > > > > > > -- > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > -- > Sincerely yours, Ivan Daschinskiy >
