Igor, and what about C++20 and coroutines [1] [1] -- https://en.cppreference.com/w/cpp/language/coroutines
чт, 9 сент. 2021 г. в 14:12, Igor Sapego <isap...@apache.org>: > Well, fortunately we do not provide a C client if you don't consider ODBC > as one so we should not think about it. For C++ I believe we should use > standard std::future+std::promise for async, but still can provide sync > methods, > built on top of async methods. There is no continuation problem in C++ API > as > std::future do not provide continuation methods :) > > Now, for the sync/async problem you've mentioned. Can not we solve it by > using different thread pools for completion and continuation of futures or > does > CompletableFuture interface not allowing for that? > > Best Regards, > Igor > > > On Thu, Sep 9, 2021 at 1:15 PM Ivan Daschinsky <ivanda...@gmail.com> > wrote: > > > And what about C/C++? There are so many options for them... > > > > чт, 9 сент. 2021 г. в 13:00, Pavel Tupitsyn <ptupit...@apache.org>: > > > > > Talked to Ivan in private, and came up with the following per-language > > > summary: > > > > > > * Java: sync and async > > > *. NET: only async > > > * Python: sync and async > > > * JavaScript: only async (sync is not possible) > > > > > > Thin clients in other languages are to be discussed. > > > > > > On Thu, Sep 9, 2021 at 11:49 AM Ivan Daschinsky <ivanda...@gmail.com> > > > wrote: > > > > > > > >> You can mix them easily. > > > > This is far from easily (you have already mentioned continuation > > > problem), > > > > but for i.e. in python it is absolutely not. > > > > For kotlin it is a little bit easier, but also not fluent and a > little > > > bit > > > > ugly. > > > > > > > > чт, 9 сент. 2021 г. в 11:37, Pavel Tupitsyn <ptupit...@apache.org>: > > > > > > > > > Ivan, > > > > > > > > > > > Pavel, is it really true, that in .NET sync versions of libraries > > and > > > > > tools > > > > > > are completely eliminated? > > > > > > > > > > Far from being eliminated, but there is a movement in this > direction. > > > > > For example, HttpClient [1] only provides async variants for most > of > > > the > > > > > methods. > > > > > > > > > > Exposing sync wrappers for async methods is not recommended [2]. > > > > > There is a good explanation of potential threading issues there, > and > > it > > > > can > > > > > be applied to Java too. > > > > > > > > > > > > > > > > you cannot mix both functions easily > > > > > > > > > > You can mix them easily. > > > > > C#: table.GetAsync(k).Result or > > > > table.GetAsync(k).GetAwaiter().GetResult() > > > > > (different exception handling) > > > > > Java: table.getAsync(k).get() > > > > > > > > > > The same thing we do in sync wrapper for async method is now in the > > > user > > > > > code. > > > > > Which is better for two reasons: > > > > > - users won't accidentally use sync API when async API should be > used > > > > > - when they really have to use sync API, potentially dangerous > > behavior > > > > is > > > > > not hidden > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=net-5.0#methods > > > > > [2] > > > > > > > > > > > > > > > > > > > > https://devblogs.microsoft.com/pfxteam/should-i-expose-synchronous-wrappers-for-asynchronous-methods/ > > > > > > > > > > On Thu, Sep 9, 2021 at 11:16 AM Ivan Daschinsky < > ivanda...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > >> 2. In languages with proper async support (async-await, etc.), > > we > > > > can > > > > > > skip sync API altogether. > > > > > > It sounds pretty strange, as for me. Usually you cannot mix both > > > > > functions > > > > > > easily, it is called blue-red functions problem [1] > > > > > > In python you definitely cannot do sync over async, for example > > > > > > (principally can, but nobody do that because of mediocre > > performance > > > of > > > > > > that solution). And many developers > > > > > > still prefer writing code withou asyncio at all. > > > > > > > > > > > > Pavel, is it really true, that in .NET sync versions of libraries > > and > > > > > tools > > > > > > are completely eliminated? > > > > > > > > > > > > [1] -- > > > > > > > > > > > > https://elizarov.medium.com/how-do-you-color-your-functions-a6bb423d936d > > > > > > > > > > > > ср, 8 сент. 2021 г. в 22:33, Pavel Tupitsyn < > ptupit...@apache.org > > >: > > > > > > > > > > > > > To put it another way: > > > > > > > - true sync operation completes by itself > > > > > > > - sync-over-async operation requires another thread to complete > > > > > > > > > > > > > > On Wed, Sep 8, 2021 at 10:15 PM Pavel Tupitsyn < > > > ptupit...@apache.org > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Val, > > > > > > > > > > > > > > > > That's exactly what I have in mind. > > > > > > > > Yes, we block the user thread, but then we should unblock it > by > > > > > > > completing > > > > > > > > the future. > > > > > > > > We can't complete the future from a Netty thread [1], so > we'll > > > need > > > > > > some > > > > > > > > other thread from some executor. > > > > > > > > If there are no threads available (because they are blocked > by > > > the > > > > > sync > > > > > > > > API above), the future won't complete => deadlock. > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/r528659381d983a177d779f56ef3d7da6fe17eb3504383f5f87727514%40%3Cdev.ignite.apache.org%3E > > > > > > > > > > > > > > > > On Wed, Sep 8, 2021 at 9:40 PM Valentin Kulichenko < > > > > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > > > > >> Pavel, > > > > > > > >> > > > > > > > >> I might be missing something - could you please elaborate a > > > little > > > > > > more? > > > > > > > >> > > > > > > > >> When I say "sync on top of async", I basically mean that > (for > > > > > example) > > > > > > > >> 'insert(..)' is equivalent to 'insertAsync(..).join()'. In > my > > > > > > > >> understanding, it only blocks the user's thread. > > > > > > > >> > > > > > > > >> Am I wrong? Or you have a different implementation in mind? > > > > > > > >> > > > > > > > >> -Val > > > > > > > >> > > > > > > > >> On Wed, Sep 8, 2021 at 12:50 AM Pavel Tupitsyn < > > > > > ptupit...@apache.org> > > > > > > > >> wrote: > > > > > > > >> > > > > > > > >> > Val, > > > > > > > >> > > > > > > > > >> > Agree with your points. > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > async API should be primary > > > > > > > >> > > > > > > > > >> > It should be noted that all our APIs are inherently async, > > > > > > > >> > because thin client is implemented asynchronously. > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > with the sync version build on top > > > > > > > >> > > > > > > > > >> > We should document somehow that sync APIs are based on > async > > > > ones, > > > > > > > >> > because this may be dangerous in some use cases. > > > > > > > >> > > > > > > > > >> > For example, as a user, I may have a thread pool of 4 > > threads > > > > for > > > > > > > >> > Ignite-related usage, that is also set as > > > > > asyncContinuationExecutor > > > > > > > [1]. > > > > > > > >> > Now if I run a lot of concurrent Ignite requests using > sync > > > API, > > > > > > all 4 > > > > > > > >> > threads will end up blocked on CompletableFutures. > > > > > > > >> > When one of the operations completes, we enqueue the > > > completion > > > > to > > > > > > > that > > > > > > > >> > same thread pool, but all threads are blocked on sync > APIs, > > > > > > resulting > > > > > > > >> in a > > > > > > > >> > deadlock. > > > > > > > >> > > > > > > > > >> > This can be prevented by using a different > > > > > > asyncContinuationExecutor, > > > > > > > >> but > > > > > > > >> > sync API users won't be usually aware of this. > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > [1] https://issues.apache.org/jira/browse/IGNITE-15359 > > > > > > > >> > > > > > > > > >> > On Wed, Sep 8, 2021 at 10:04 AM Courtney Robinson < > > > > > > > >> > courtney.robin...@hypi.io> > > > > > > > >> > wrote: > > > > > > > >> > > > > > > > > >> > > Hi Val, > > > > > > > >> > > > > > > > > > >> > > I'd highly support an async first API based on > > > CompletionStage > > > > > > > >> > > < > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletionStage.html > > > > > > > >> > > > > > > > > > > >> > > or > > > > > > > >> > > its subtypes like CompletableFuture. > > > > > > > >> > > In Ignite 2 we've written a wrapper library around > > > > IgniteFuture > > > > > to > > > > > > > >> > provide > > > > > > > >> > > CompletionStage instead because many of the newer libs > we > > > use > > > > > > > support > > > > > > > >> > this. > > > > > > > >> > > If Ignite 3 went this way it'd remove a lot of boiler > > > > > > plate/wrapper > > > > > > > >> that > > > > > > > >> > we > > > > > > > >> > > wrote to get what you're suggesting here. > > > > > > > >> > > > > > > > > > >> > > Regards, > > > > > > > >> > > Courtney Robinson > > > > > > > >> > > Founder and CEO, Hypi > > > > > > > >> > > Tel: ++44 208 123 2413 (GMT+0) <https://hypi.io> > > > > > > > >> > > > > > > > > > >> > > <https://hypi.io> > > > > > > > >> > > https://hypi.io > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > On Wed, Sep 8, 2021 at 12:44 AM Valentin Kulichenko < > > > > > > > >> > > valentin.kuliche...@gmail.com> wrote: > > > > > > > >> > > > > > > > > > >> > > > Igniters, > > > > > > > >> > > > > > > > > > > >> > > > I would like to gather some opinions on whether we > want > > to > > > > > focus > > > > > > > on > > > > > > > >> > sync > > > > > > > >> > > vs > > > > > > > >> > > > async APIs in Ignite 3. > > > > > > > >> > > > > > > > > > > >> > > > Here are some initial considerations that I have: > > > > > > > >> > > > 1. Ignite 2.x is essentially "sync first". Async APIs > > > exist, > > > > > but > > > > > > > >> they > > > > > > > >> > use > > > > > > > >> > > > non-standard IgniteFuture and provide counterintuitive > > > > > > guarantees. > > > > > > > >> In > > > > > > > >> > my > > > > > > > >> > > > experience, they significantly lack usability, and > > because > > > > of > > > > > > that > > > > > > > >> are > > > > > > > >> > > > rarely used. > > > > > > > >> > > > 2. In general, however, async execution becomes more > and > > > > more > > > > > > > >> > prominent. > > > > > > > >> > > > Something we can't ignore if we want to create a > modern > > > > > > framework. > > > > > > > >> > > > 3. Still, async support in Java is very limited > > > (especially > > > > if > > > > > > > >> compared > > > > > > > >> > > to > > > > > > > >> > > > other languages, like C# for example). > > > > > > > >> > > > > > > > > > > >> > > > My current position is the following (happy to > discuss): > > > > > > > >> > > > 1. We should pay more attention to async APIs. As a > > > general > > > > > > rule, > > > > > > > >> async > > > > > > > >> > > API > > > > > > > >> > > > should be primary, with the sync version build on top. > > > > > > > >> > > > 2. In languages with proper async support > (async-await, > > > > etc.), > > > > > > we > > > > > > > >> can > > > > > > > >> > > skip > > > > > > > >> > > > sync API altogether. As an example of this, you can > look > > > at > > > > > the > > > > > > > >> first > > > > > > > >> > > > version of the .NET client [1]. It exposes only async > > > > methods, > > > > > > and > > > > > > > >> it > > > > > > > >> > > > doesn't look like sync counterparts are really needed. > > > > > > > >> > > > 3. In Java (as well as other languages where > > applicable), > > > we > > > > > > will > > > > > > > >> add > > > > > > > >> > > sync > > > > > > > >> > > > APIs that simply delegate to async APIs. This will > help > > > > users > > > > > to > > > > > > > >> avoid > > > > > > > >> > > > CompletableFuture if they don't want to use it. > > > > > > > >> > > > > > > > > > > >> > > > [1] https://github.com/apache/ignite-3/pull/306 > > > > > > > >> > > > > > > > > > > >> > > > Please share your thoughts. > > > > > > > >> > > > > > > > > > > >> > > > -Val > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > > > > > > -- > > Sincerely yours, Ivan Daschinskiy > > > -- Sincerely yours, Ivan Daschinskiy
