Re: IEP-90 Ignite 3 Client Lifecycle
Igor, ok, agreed. On Mon, Jun 27, 2022 at 11:27 PM Igor Sapego wrote: > Pavel, let's postpone developing a sessions part until all the corner cases > are clarified. I'll let you know when new the proposal is ready for > discussion > again. > > Best Regards, > Igor > > > On Mon, May 23, 2022 at 12:25 PM Andrey Gura wrote: > > > Thanks for the answers, Igor. > > > > On Thu, May 19, 2022 at 10:55 PM Igor Sapego wrote: > > > > > > Andrey, > > > > > > 1. If a server generates a UUID that already exists it can check and > just > > > re-generate it straight away > > > as a check is just a simple map lookup. > > > > > > 2. Well, yes. This proposal does not consider monitoring, but with > > current > > > approach it will be definitely > > > easier to implement it properly. > > > > > > Pavel, > > > > > > Wow, that's really cool that you've already started working on it. > > > > > > Regarding your proposal I like it - it will really make a procedure > > easier > > > and faster for a client. I'll change > > > the IEP accordingly. > > > > > > Best Regards, > > > Igor > > > > > > > > > On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn > > > wrote: > > > > > > > Igor, > > > > > > > > I've started working on the initial sessions implementation [1], > > > > and I'd like to clarify the procedure of logical connection restore. > > > > > > > > > client in its turn tries to establish a new *node connection* > > > > > and restore *logical connection *using ConnectionRestoreReq > > > > > > > > This implies either an additional request, or something that replaces > > > > normal handshake. > > > > However, we need to handle two cases here: > > > > - Logical connection is restored > > > > - Logical connection is not restored (timed out, server restarted, > > etc), in > > > > which case we still establish the connection and use it. > > > > > > > > To account for the second case, we should always start with a regular > > > > handshake. > > > > I propose to add a nullable UUID field to the handshake request to > > cover > > > > both cases: > > > > - connectionId is null or not found on server: proceed with normal > > > > handshake, return newly generated connectionId. > > > > - connectionId is not null and found on server: restore logical > > connection, > > > > return the same connectionId > > > > > > > > Client checks if returned connectionId equals to the original and > > > > understands whether logical connection was restored or not. > > > > > > > > Thoughts? > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-16928 > > > > > > > > On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn < > ptupit...@apache.org> > > > > wrote: > > > > > > > > > Andrey, > > > > > > > > > > > different connections on different client instances theoretically > > > > > > could generate an already existing connection ID > > > > > > > > > > Do you mean an UUID collision? > > > > > > > > > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura > > wrote: > > > > > > > > > >> Igor, > > > > >> > > > > >> Thanks for the proposal. > > > > >> > > > > >> I understand that such a situation is almost impossible but "if > > > > >> anything can go wrong, it will". Does the protocol take into > account > > > > >> that different connections on different client instances > > theoretically > > > > >> could generate an already existing connection ID? > > > > >> > > > > >> Also, do I understand correctly that a server has enough > information > > > > >> about client connections so it will be possible to observe a > > > > >> connections list on the server? It would be useful for cluster > > > > >> monitoring purposes. > > > > >> > > > > >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego > > wrote: > > > > >> > > > > > >> > 1. I've tried to clarify IDs part; > > > > >> > 2. Maybe you are right, though in this case we'd need to use > > > > >> authentication > > > > >> > in ConnectionRestoreReq. Which sounds reasonable to me. > > > > >> > > > > > >> > Best Regards, > > > > >> > Igor > > > > >> > > > > > >> > > > > > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn < > > ptupit...@apache.org > > > > > > > > > >> > wrote: > > > > >> > > > > > >> > > Igor, > > > > >> > > > > > > >> > > The proposal looks good to me. Very detailed! > > > > >> > > > > > > >> > > A couple comments: > > > > >> > > > > > > >> > > 1. There is a bit of a term mixup with "Connection ID", "Node > > ID", > > > > >> "Token" > > > > >> > > - can you please review those? > > > > >> > > > > > > >> > > 2. > The Connection ID should be generated using a proper > secure > > > > >> algorithm > > > > >> > > (additional research is required here) to make sure an > intruder > > can > > > > >> not > > > > >> > > generate an existing Connection ID > > > > >> > > Not sure about the reasoning here. I think randomUUID() should > > be > > > > >> enough: > > > > >> > > - In the case of an unsecured cluster it does not matter, > > because > > > > >> anyone > > > > >> > > can do anything. > > > > >> > > - In the case of
Re: IEP-90 Ignite 3 Client Lifecycle
Pavel, let's postpone developing a sessions part until all the corner cases are clarified. I'll let you know when new the proposal is ready for discussion again. Best Regards, Igor On Mon, May 23, 2022 at 12:25 PM Andrey Gura wrote: > Thanks for the answers, Igor. > > On Thu, May 19, 2022 at 10:55 PM Igor Sapego wrote: > > > > Andrey, > > > > 1. If a server generates a UUID that already exists it can check and just > > re-generate it straight away > > as a check is just a simple map lookup. > > > > 2. Well, yes. This proposal does not consider monitoring, but with > current > > approach it will be definitely > > easier to implement it properly. > > > > Pavel, > > > > Wow, that's really cool that you've already started working on it. > > > > Regarding your proposal I like it - it will really make a procedure > easier > > and faster for a client. I'll change > > the IEP accordingly. > > > > Best Regards, > > Igor > > > > > > On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn > > wrote: > > > > > Igor, > > > > > > I've started working on the initial sessions implementation [1], > > > and I'd like to clarify the procedure of logical connection restore. > > > > > > > client in its turn tries to establish a new *node connection* > > > > and restore *logical connection *using ConnectionRestoreReq > > > > > > This implies either an additional request, or something that replaces > > > normal handshake. > > > However, we need to handle two cases here: > > > - Logical connection is restored > > > - Logical connection is not restored (timed out, server restarted, > etc), in > > > which case we still establish the connection and use it. > > > > > > To account for the second case, we should always start with a regular > > > handshake. > > > I propose to add a nullable UUID field to the handshake request to > cover > > > both cases: > > > - connectionId is null or not found on server: proceed with normal > > > handshake, return newly generated connectionId. > > > - connectionId is not null and found on server: restore logical > connection, > > > return the same connectionId > > > > > > Client checks if returned connectionId equals to the original and > > > understands whether logical connection was restored or not. > > > > > > Thoughts? > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-16928 > > > > > > On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn > > > wrote: > > > > > > > Andrey, > > > > > > > > > different connections on different client instances theoretically > > > > > could generate an already existing connection ID > > > > > > > > Do you mean an UUID collision? > > > > > > > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura > wrote: > > > > > > > >> Igor, > > > >> > > > >> Thanks for the proposal. > > > >> > > > >> I understand that such a situation is almost impossible but "if > > > >> anything can go wrong, it will". Does the protocol take into account > > > >> that different connections on different client instances > theoretically > > > >> could generate an already existing connection ID? > > > >> > > > >> Also, do I understand correctly that a server has enough information > > > >> about client connections so it will be possible to observe a > > > >> connections list on the server? It would be useful for cluster > > > >> monitoring purposes. > > > >> > > > >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego > wrote: > > > >> > > > > >> > 1. I've tried to clarify IDs part; > > > >> > 2. Maybe you are right, though in this case we'd need to use > > > >> authentication > > > >> > in ConnectionRestoreReq. Which sounds reasonable to me. > > > >> > > > > >> > Best Regards, > > > >> > Igor > > > >> > > > > >> > > > > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn < > ptupit...@apache.org > > > > > > > >> > wrote: > > > >> > > > > >> > > Igor, > > > >> > > > > > >> > > The proposal looks good to me. Very detailed! > > > >> > > > > > >> > > A couple comments: > > > >> > > > > > >> > > 1. There is a bit of a term mixup with "Connection ID", "Node > ID", > > > >> "Token" > > > >> > > - can you please review those? > > > >> > > > > > >> > > 2. > The Connection ID should be generated using a proper secure > > > >> algorithm > > > >> > > (additional research is required here) to make sure an intruder > can > > > >> not > > > >> > > generate an existing Connection ID > > > >> > > Not sure about the reasoning here. I think randomUUID() should > be > > > >> enough: > > > >> > > - In the case of an unsecured cluster it does not matter, > because > > > >> anyone > > > >> > > can do anything. > > > >> > > - In the case of a secured cluster it does not matter, because > > > >> > > authentication/authorization keeps intruders out. > > > >> > > > > > >> > > > > > >> > > On Mon, May 16, 2022 at 11:07 PM Igor Sapego < > isap...@apache.org> > > > >> wrote: > > > >> > > > > > >> > > > Hi, Igniters > > > >> > > > > > > >> > > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The > main > > > >> idea is > > > >>
Re: IEP-90 Ignite 3 Client Lifecycle
Thanks for the answers, Igor. On Thu, May 19, 2022 at 10:55 PM Igor Sapego wrote: > > Andrey, > > 1. If a server generates a UUID that already exists it can check and just > re-generate it straight away > as a check is just a simple map lookup. > > 2. Well, yes. This proposal does not consider monitoring, but with current > approach it will be definitely > easier to implement it properly. > > Pavel, > > Wow, that's really cool that you've already started working on it. > > Regarding your proposal I like it - it will really make a procedure easier > and faster for a client. I'll change > the IEP accordingly. > > Best Regards, > Igor > > > On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn > wrote: > > > Igor, > > > > I've started working on the initial sessions implementation [1], > > and I'd like to clarify the procedure of logical connection restore. > > > > > client in its turn tries to establish a new *node connection* > > > and restore *logical connection *using ConnectionRestoreReq > > > > This implies either an additional request, or something that replaces > > normal handshake. > > However, we need to handle two cases here: > > - Logical connection is restored > > - Logical connection is not restored (timed out, server restarted, etc), in > > which case we still establish the connection and use it. > > > > To account for the second case, we should always start with a regular > > handshake. > > I propose to add a nullable UUID field to the handshake request to cover > > both cases: > > - connectionId is null or not found on server: proceed with normal > > handshake, return newly generated connectionId. > > - connectionId is not null and found on server: restore logical connection, > > return the same connectionId > > > > Client checks if returned connectionId equals to the original and > > understands whether logical connection was restored or not. > > > > Thoughts? > > > > [1] https://issues.apache.org/jira/browse/IGNITE-16928 > > > > On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn > > wrote: > > > > > Andrey, > > > > > > > different connections on different client instances theoretically > > > > could generate an already existing connection ID > > > > > > Do you mean an UUID collision? > > > > > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura wrote: > > > > > >> Igor, > > >> > > >> Thanks for the proposal. > > >> > > >> I understand that such a situation is almost impossible but "if > > >> anything can go wrong, it will". Does the protocol take into account > > >> that different connections on different client instances theoretically > > >> could generate an already existing connection ID? > > >> > > >> Also, do I understand correctly that a server has enough information > > >> about client connections so it will be possible to observe a > > >> connections list on the server? It would be useful for cluster > > >> monitoring purposes. > > >> > > >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego wrote: > > >> > > > >> > 1. I've tried to clarify IDs part; > > >> > 2. Maybe you are right, though in this case we'd need to use > > >> authentication > > >> > in ConnectionRestoreReq. Which sounds reasonable to me. > > >> > > > >> > Best Regards, > > >> > Igor > > >> > > > >> > > > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn > > > > >> > wrote: > > >> > > > >> > > Igor, > > >> > > > > >> > > The proposal looks good to me. Very detailed! > > >> > > > > >> > > A couple comments: > > >> > > > > >> > > 1. There is a bit of a term mixup with "Connection ID", "Node ID", > > >> "Token" > > >> > > - can you please review those? > > >> > > > > >> > > 2. > The Connection ID should be generated using a proper secure > > >> algorithm > > >> > > (additional research is required here) to make sure an intruder can > > >> not > > >> > > generate an existing Connection ID > > >> > > Not sure about the reasoning here. I think randomUUID() should be > > >> enough: > > >> > > - In the case of an unsecured cluster it does not matter, because > > >> anyone > > >> > > can do anything. > > >> > > - In the case of a secured cluster it does not matter, because > > >> > > authentication/authorization keeps intruders out. > > >> > > > > >> > > > > >> > > On Mon, May 16, 2022 at 11:07 PM Igor Sapego > > >> wrote: > > >> > > > > >> > > > Hi, Igniters > > >> > > > > > >> > > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main > > >> idea is > > >> > > to > > >> > > > define client lifecycle as well as core algorithms and mechanisms > > >> used by > > >> > > > clients. This proposal can be used as a reference for > > >> implementation of a > > >> > > > new client for Ignite when dealing with such problems as: > > >> > > > > > >> > > >- Resolving of user-provided addresses; > > >> > > >- Initial connection to a cluster; > > >> > > >- Maintaining cluster connection; > > >> > > >- Connection recovery; > > >> > > >- Connection break handling. > > >> > > > > > >> > > > So take a look and let me know
Re: IEP-90 Ignite 3 Client Lifecycle
One more corner case: In case of a connection loss, even if the message was successfully written to the socket, there is no guarantee it was read on the other side For example, a client could have sent a request with ID 10, but the server did not get it. Or the server sent a reply to request 9, but the client did not get it. So when a logical connection is restored, we need to compare pending client requests (TcpClientChannel.pendingReqs) with the server outbox. To do that, I propose to send a list of known requestIds from the server to the client. The client then completes missing requests with an error and RetryPolicy will deal with them up the call stack. On Fri, May 20, 2022 at 12:02 PM Pavel Tupitsyn wrote: > Another point: > > With the current proposal, there is no way for the server to distinguish > a broken connection from a normal user-initiated connection close. > > So in either case we'll wait for some configured timeout before cleaning > up session resources. > I think this is fine, at least for now. > > On Thu, May 19, 2022 at 10:55 PM Igor Sapego wrote: > >> Andrey, >> >> 1. If a server generates a UUID that already exists it can check and just >> re-generate it straight away >> as a check is just a simple map lookup. >> >> 2. Well, yes. This proposal does not consider monitoring, but with current >> approach it will be definitely >> easier to implement it properly. >> >> Pavel, >> >> Wow, that's really cool that you've already started working on it. >> >> Regarding your proposal I like it - it will really make a procedure easier >> and faster for a client. I'll change >> the IEP accordingly. >> >> Best Regards, >> Igor >> >> >> On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn >> wrote: >> >> > Igor, >> > >> > I've started working on the initial sessions implementation [1], >> > and I'd like to clarify the procedure of logical connection restore. >> > >> > > client in its turn tries to establish a new *node connection* >> > > and restore *logical connection *using ConnectionRestoreReq >> > >> > This implies either an additional request, or something that replaces >> > normal handshake. >> > However, we need to handle two cases here: >> > - Logical connection is restored >> > - Logical connection is not restored (timed out, server restarted, >> etc), in >> > which case we still establish the connection and use it. >> > >> > To account for the second case, we should always start with a regular >> > handshake. >> > I propose to add a nullable UUID field to the handshake request to cover >> > both cases: >> > - connectionId is null or not found on server: proceed with normal >> > handshake, return newly generated connectionId. >> > - connectionId is not null and found on server: restore logical >> connection, >> > return the same connectionId >> > >> > Client checks if returned connectionId equals to the original and >> > understands whether logical connection was restored or not. >> > >> > Thoughts? >> > >> > [1] https://issues.apache.org/jira/browse/IGNITE-16928 >> > >> > On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn >> > wrote: >> > >> > > Andrey, >> > > >> > > > different connections on different client instances theoretically >> > > > could generate an already existing connection ID >> > > >> > > Do you mean an UUID collision? >> > > >> > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura wrote: >> > > >> > >> Igor, >> > >> >> > >> Thanks for the proposal. >> > >> >> > >> I understand that such a situation is almost impossible but "if >> > >> anything can go wrong, it will". Does the protocol take into account >> > >> that different connections on different client instances >> theoretically >> > >> could generate an already existing connection ID? >> > >> >> > >> Also, do I understand correctly that a server has enough information >> > >> about client connections so it will be possible to observe a >> > >> connections list on the server? It would be useful for cluster >> > >> monitoring purposes. >> > >> >> > >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego >> wrote: >> > >> > >> > >> > 1. I've tried to clarify IDs part; >> > >> > 2. Maybe you are right, though in this case we'd need to use >> > >> authentication >> > >> > in ConnectionRestoreReq. Which sounds reasonable to me. >> > >> > >> > >> > Best Regards, >> > >> > Igor >> > >> > >> > >> > >> > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn < >> ptupit...@apache.org >> > > >> > >> > wrote: >> > >> > >> > >> > > Igor, >> > >> > > >> > >> > > The proposal looks good to me. Very detailed! >> > >> > > >> > >> > > A couple comments: >> > >> > > >> > >> > > 1. There is a bit of a term mixup with "Connection ID", "Node >> ID", >> > >> "Token" >> > >> > > - can you please review those? >> > >> > > >> > >> > > 2. > The Connection ID should be generated using a proper secure >> > >> algorithm >> > >> > > (additional research is required here) to make sure an intruder >> can >> > >> not >> > >> > > generate an existing Connection ID >>
Re: IEP-90 Ignite 3 Client Lifecycle
Another point: With the current proposal, there is no way for the server to distinguish a broken connection from a normal user-initiated connection close. So in either case we'll wait for some configured timeout before cleaning up session resources. I think this is fine, at least for now. On Thu, May 19, 2022 at 10:55 PM Igor Sapego wrote: > Andrey, > > 1. If a server generates a UUID that already exists it can check and just > re-generate it straight away > as a check is just a simple map lookup. > > 2. Well, yes. This proposal does not consider monitoring, but with current > approach it will be definitely > easier to implement it properly. > > Pavel, > > Wow, that's really cool that you've already started working on it. > > Regarding your proposal I like it - it will really make a procedure easier > and faster for a client. I'll change > the IEP accordingly. > > Best Regards, > Igor > > > On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn > wrote: > > > Igor, > > > > I've started working on the initial sessions implementation [1], > > and I'd like to clarify the procedure of logical connection restore. > > > > > client in its turn tries to establish a new *node connection* > > > and restore *logical connection *using ConnectionRestoreReq > > > > This implies either an additional request, or something that replaces > > normal handshake. > > However, we need to handle two cases here: > > - Logical connection is restored > > - Logical connection is not restored (timed out, server restarted, etc), > in > > which case we still establish the connection and use it. > > > > To account for the second case, we should always start with a regular > > handshake. > > I propose to add a nullable UUID field to the handshake request to cover > > both cases: > > - connectionId is null or not found on server: proceed with normal > > handshake, return newly generated connectionId. > > - connectionId is not null and found on server: restore logical > connection, > > return the same connectionId > > > > Client checks if returned connectionId equals to the original and > > understands whether logical connection was restored or not. > > > > Thoughts? > > > > [1] https://issues.apache.org/jira/browse/IGNITE-16928 > > > > On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn > > wrote: > > > > > Andrey, > > > > > > > different connections on different client instances theoretically > > > > could generate an already existing connection ID > > > > > > Do you mean an UUID collision? > > > > > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura wrote: > > > > > >> Igor, > > >> > > >> Thanks for the proposal. > > >> > > >> I understand that such a situation is almost impossible but "if > > >> anything can go wrong, it will". Does the protocol take into account > > >> that different connections on different client instances theoretically > > >> could generate an already existing connection ID? > > >> > > >> Also, do I understand correctly that a server has enough information > > >> about client connections so it will be possible to observe a > > >> connections list on the server? It would be useful for cluster > > >> monitoring purposes. > > >> > > >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego > wrote: > > >> > > > >> > 1. I've tried to clarify IDs part; > > >> > 2. Maybe you are right, though in this case we'd need to use > > >> authentication > > >> > in ConnectionRestoreReq. Which sounds reasonable to me. > > >> > > > >> > Best Regards, > > >> > Igor > > >> > > > >> > > > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn < > ptupit...@apache.org > > > > > >> > wrote: > > >> > > > >> > > Igor, > > >> > > > > >> > > The proposal looks good to me. Very detailed! > > >> > > > > >> > > A couple comments: > > >> > > > > >> > > 1. There is a bit of a term mixup with "Connection ID", "Node ID", > > >> "Token" > > >> > > - can you please review those? > > >> > > > > >> > > 2. > The Connection ID should be generated using a proper secure > > >> algorithm > > >> > > (additional research is required here) to make sure an intruder > can > > >> not > > >> > > generate an existing Connection ID > > >> > > Not sure about the reasoning here. I think randomUUID() should be > > >> enough: > > >> > > - In the case of an unsecured cluster it does not matter, because > > >> anyone > > >> > > can do anything. > > >> > > - In the case of a secured cluster it does not matter, because > > >> > > authentication/authorization keeps intruders out. > > >> > > > > >> > > > > >> > > On Mon, May 16, 2022 at 11:07 PM Igor Sapego > > >> wrote: > > >> > > > > >> > > > Hi, Igniters > > >> > > > > > >> > > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main > > >> idea is > > >> > > to > > >> > > > define client lifecycle as well as core algorithms and > mechanisms > > >> used by > > >> > > > clients. This proposal can be used as a reference for > > >> implementation of a > > >> > > > new client for Ignite when dealing with such problems as: > > >> > > > >
Re: IEP-90 Ignite 3 Client Lifecycle
Andrey, 1. If a server generates a UUID that already exists it can check and just re-generate it straight away as a check is just a simple map lookup. 2. Well, yes. This proposal does not consider monitoring, but with current approach it will be definitely easier to implement it properly. Pavel, Wow, that's really cool that you've already started working on it. Regarding your proposal I like it - it will really make a procedure easier and faster for a client. I'll change the IEP accordingly. Best Regards, Igor On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn wrote: > Igor, > > I've started working on the initial sessions implementation [1], > and I'd like to clarify the procedure of logical connection restore. > > > client in its turn tries to establish a new *node connection* > > and restore *logical connection *using ConnectionRestoreReq > > This implies either an additional request, or something that replaces > normal handshake. > However, we need to handle two cases here: > - Logical connection is restored > - Logical connection is not restored (timed out, server restarted, etc), in > which case we still establish the connection and use it. > > To account for the second case, we should always start with a regular > handshake. > I propose to add a nullable UUID field to the handshake request to cover > both cases: > - connectionId is null or not found on server: proceed with normal > handshake, return newly generated connectionId. > - connectionId is not null and found on server: restore logical connection, > return the same connectionId > > Client checks if returned connectionId equals to the original and > understands whether logical connection was restored or not. > > Thoughts? > > [1] https://issues.apache.org/jira/browse/IGNITE-16928 > > On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn > wrote: > > > Andrey, > > > > > different connections on different client instances theoretically > > > could generate an already existing connection ID > > > > Do you mean an UUID collision? > > > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura wrote: > > > >> Igor, > >> > >> Thanks for the proposal. > >> > >> I understand that such a situation is almost impossible but "if > >> anything can go wrong, it will". Does the protocol take into account > >> that different connections on different client instances theoretically > >> could generate an already existing connection ID? > >> > >> Also, do I understand correctly that a server has enough information > >> about client connections so it will be possible to observe a > >> connections list on the server? It would be useful for cluster > >> monitoring purposes. > >> > >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego wrote: > >> > > >> > 1. I've tried to clarify IDs part; > >> > 2. Maybe you are right, though in this case we'd need to use > >> authentication > >> > in ConnectionRestoreReq. Which sounds reasonable to me. > >> > > >> > Best Regards, > >> > Igor > >> > > >> > > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn > > >> > wrote: > >> > > >> > > Igor, > >> > > > >> > > The proposal looks good to me. Very detailed! > >> > > > >> > > A couple comments: > >> > > > >> > > 1. There is a bit of a term mixup with "Connection ID", "Node ID", > >> "Token" > >> > > - can you please review those? > >> > > > >> > > 2. > The Connection ID should be generated using a proper secure > >> algorithm > >> > > (additional research is required here) to make sure an intruder can > >> not > >> > > generate an existing Connection ID > >> > > Not sure about the reasoning here. I think randomUUID() should be > >> enough: > >> > > - In the case of an unsecured cluster it does not matter, because > >> anyone > >> > > can do anything. > >> > > - In the case of a secured cluster it does not matter, because > >> > > authentication/authorization keeps intruders out. > >> > > > >> > > > >> > > On Mon, May 16, 2022 at 11:07 PM Igor Sapego > >> wrote: > >> > > > >> > > > Hi, Igniters > >> > > > > >> > > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main > >> idea is > >> > > to > >> > > > define client lifecycle as well as core algorithms and mechanisms > >> used by > >> > > > clients. This proposal can be used as a reference for > >> implementation of a > >> > > > new client for Ignite when dealing with such problems as: > >> > > > > >> > > >- Resolving of user-provided addresses; > >> > > >- Initial connection to a cluster; > >> > > >- Maintaining cluster connection; > >> > > >- Connection recovery; > >> > > >- Connection break handling. > >> > > > > >> > > > So take a look and let me know what you think guys. > >> > > > > >> > > > [1] - > >> > > > > >> > > > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle > >> > > > > >> > > > Best Regards, > >> > > > Igor > >> > > > > >> > > > >> > > >
Re: IEP-90 Ignite 3 Client Lifecycle
Igor, I've started working on the initial sessions implementation [1], and I'd like to clarify the procedure of logical connection restore. > client in its turn tries to establish a new *node connection* > and restore *logical connection *using ConnectionRestoreReq This implies either an additional request, or something that replaces normal handshake. However, we need to handle two cases here: - Logical connection is restored - Logical connection is not restored (timed out, server restarted, etc), in which case we still establish the connection and use it. To account for the second case, we should always start with a regular handshake. I propose to add a nullable UUID field to the handshake request to cover both cases: - connectionId is null or not found on server: proceed with normal handshake, return newly generated connectionId. - connectionId is not null and found on server: restore logical connection, return the same connectionId Client checks if returned connectionId equals to the original and understands whether logical connection was restored or not. Thoughts? [1] https://issues.apache.org/jira/browse/IGNITE-16928 On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn wrote: > Andrey, > > > different connections on different client instances theoretically > > could generate an already existing connection ID > > Do you mean an UUID collision? > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura wrote: > >> Igor, >> >> Thanks for the proposal. >> >> I understand that such a situation is almost impossible but "if >> anything can go wrong, it will". Does the protocol take into account >> that different connections on different client instances theoretically >> could generate an already existing connection ID? >> >> Also, do I understand correctly that a server has enough information >> about client connections so it will be possible to observe a >> connections list on the server? It would be useful for cluster >> monitoring purposes. >> >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego wrote: >> > >> > 1. I've tried to clarify IDs part; >> > 2. Maybe you are right, though in this case we'd need to use >> authentication >> > in ConnectionRestoreReq. Which sounds reasonable to me. >> > >> > Best Regards, >> > Igor >> > >> > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn >> > wrote: >> > >> > > Igor, >> > > >> > > The proposal looks good to me. Very detailed! >> > > >> > > A couple comments: >> > > >> > > 1. There is a bit of a term mixup with "Connection ID", "Node ID", >> "Token" >> > > - can you please review those? >> > > >> > > 2. > The Connection ID should be generated using a proper secure >> algorithm >> > > (additional research is required here) to make sure an intruder can >> not >> > > generate an existing Connection ID >> > > Not sure about the reasoning here. I think randomUUID() should be >> enough: >> > > - In the case of an unsecured cluster it does not matter, because >> anyone >> > > can do anything. >> > > - In the case of a secured cluster it does not matter, because >> > > authentication/authorization keeps intruders out. >> > > >> > > >> > > On Mon, May 16, 2022 at 11:07 PM Igor Sapego >> wrote: >> > > >> > > > Hi, Igniters >> > > > >> > > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main >> idea is >> > > to >> > > > define client lifecycle as well as core algorithms and mechanisms >> used by >> > > > clients. This proposal can be used as a reference for >> implementation of a >> > > > new client for Ignite when dealing with such problems as: >> > > > >> > > >- Resolving of user-provided addresses; >> > > >- Initial connection to a cluster; >> > > >- Maintaining cluster connection; >> > > >- Connection recovery; >> > > >- Connection break handling. >> > > > >> > > > So take a look and let me know what you think guys. >> > > > >> > > > [1] - >> > > > >> > > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle >> > > > >> > > > Best Regards, >> > > > Igor >> > > > >> > > >> >
Re: IEP-90 Ignite 3 Client Lifecycle
Andrey, > different connections on different client instances theoretically > could generate an already existing connection ID Do you mean an UUID collision? On Thu, May 19, 2022 at 1:09 AM Andrey Gura wrote: > Igor, > > Thanks for the proposal. > > I understand that such a situation is almost impossible but "if > anything can go wrong, it will". Does the protocol take into account > that different connections on different client instances theoretically > could generate an already existing connection ID? > > Also, do I understand correctly that a server has enough information > about client connections so it will be possible to observe a > connections list on the server? It would be useful for cluster > monitoring purposes. > > On Tue, May 17, 2022 at 3:11 PM Igor Sapego wrote: > > > > 1. I've tried to clarify IDs part; > > 2. Maybe you are right, though in this case we'd need to use > authentication > > in ConnectionRestoreReq. Which sounds reasonable to me. > > > > Best Regards, > > Igor > > > > > > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn > > wrote: > > > > > Igor, > > > > > > The proposal looks good to me. Very detailed! > > > > > > A couple comments: > > > > > > 1. There is a bit of a term mixup with "Connection ID", "Node ID", > "Token" > > > - can you please review those? > > > > > > 2. > The Connection ID should be generated using a proper secure > algorithm > > > (additional research is required here) to make sure an intruder can not > > > generate an existing Connection ID > > > Not sure about the reasoning here. I think randomUUID() should be > enough: > > > - In the case of an unsecured cluster it does not matter, because > anyone > > > can do anything. > > > - In the case of a secured cluster it does not matter, because > > > authentication/authorization keeps intruders out. > > > > > > > > > On Mon, May 16, 2022 at 11:07 PM Igor Sapego > wrote: > > > > > > > Hi, Igniters > > > > > > > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main > idea is > > > to > > > > define client lifecycle as well as core algorithms and mechanisms > used by > > > > clients. This proposal can be used as a reference for implementation > of a > > > > new client for Ignite when dealing with such problems as: > > > > > > > >- Resolving of user-provided addresses; > > > >- Initial connection to a cluster; > > > >- Maintaining cluster connection; > > > >- Connection recovery; > > > >- Connection break handling. > > > > > > > > So take a look and let me know what you think guys. > > > > > > > > [1] - > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle > > > > > > > > Best Regards, > > > > Igor > > > > > > > >
Re: IEP-90 Ignite 3 Client Lifecycle
Igor, Thanks for the proposal. I understand that such a situation is almost impossible but "if anything can go wrong, it will". Does the protocol take into account that different connections on different client instances theoretically could generate an already existing connection ID? Also, do I understand correctly that a server has enough information about client connections so it will be possible to observe a connections list on the server? It would be useful for cluster monitoring purposes. On Tue, May 17, 2022 at 3:11 PM Igor Sapego wrote: > > 1. I've tried to clarify IDs part; > 2. Maybe you are right, though in this case we'd need to use authentication > in ConnectionRestoreReq. Which sounds reasonable to me. > > Best Regards, > Igor > > > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn > wrote: > > > Igor, > > > > The proposal looks good to me. Very detailed! > > > > A couple comments: > > > > 1. There is a bit of a term mixup with "Connection ID", "Node ID", "Token" > > - can you please review those? > > > > 2. > The Connection ID should be generated using a proper secure algorithm > > (additional research is required here) to make sure an intruder can not > > generate an existing Connection ID > > Not sure about the reasoning here. I think randomUUID() should be enough: > > - In the case of an unsecured cluster it does not matter, because anyone > > can do anything. > > - In the case of a secured cluster it does not matter, because > > authentication/authorization keeps intruders out. > > > > > > On Mon, May 16, 2022 at 11:07 PM Igor Sapego wrote: > > > > > Hi, Igniters > > > > > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main idea is > > to > > > define client lifecycle as well as core algorithms and mechanisms used by > > > clients. This proposal can be used as a reference for implementation of a > > > new client for Ignite when dealing with such problems as: > > > > > >- Resolving of user-provided addresses; > > >- Initial connection to a cluster; > > >- Maintaining cluster connection; > > >- Connection recovery; > > >- Connection break handling. > > > > > > So take a look and let me know what you think guys. > > > > > > [1] - > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle > > > > > > Best Regards, > > > Igor > > > > >
Re: IEP-90 Ignite 3 Client Lifecycle
1. I've tried to clarify IDs part; 2. Maybe you are right, though in this case we'd need to use authentication in ConnectionRestoreReq. Which sounds reasonable to me. Best Regards, Igor On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn wrote: > Igor, > > The proposal looks good to me. Very detailed! > > A couple comments: > > 1. There is a bit of a term mixup with "Connection ID", "Node ID", "Token" > - can you please review those? > > 2. > The Connection ID should be generated using a proper secure algorithm > (additional research is required here) to make sure an intruder can not > generate an existing Connection ID > Not sure about the reasoning here. I think randomUUID() should be enough: > - In the case of an unsecured cluster it does not matter, because anyone > can do anything. > - In the case of a secured cluster it does not matter, because > authentication/authorization keeps intruders out. > > > On Mon, May 16, 2022 at 11:07 PM Igor Sapego wrote: > > > Hi, Igniters > > > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main idea is > to > > define client lifecycle as well as core algorithms and mechanisms used by > > clients. This proposal can be used as a reference for implementation of a > > new client for Ignite when dealing with such problems as: > > > >- Resolving of user-provided addresses; > >- Initial connection to a cluster; > >- Maintaining cluster connection; > >- Connection recovery; > >- Connection break handling. > > > > So take a look and let me know what you think guys. > > > > [1] - > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle > > > > Best Regards, > > Igor > > >
Re: IEP-90 Ignite 3 Client Lifecycle
Igor, The proposal looks good to me. Very detailed! A couple comments: 1. There is a bit of a term mixup with "Connection ID", "Node ID", "Token" - can you please review those? 2. > The Connection ID should be generated using a proper secure algorithm (additional research is required here) to make sure an intruder can not generate an existing Connection ID Not sure about the reasoning here. I think randomUUID() should be enough: - In the case of an unsecured cluster it does not matter, because anyone can do anything. - In the case of a secured cluster it does not matter, because authentication/authorization keeps intruders out. On Mon, May 16, 2022 at 11:07 PM Igor Sapego wrote: > Hi, Igniters > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main idea is to > define client lifecycle as well as core algorithms and mechanisms used by > clients. This proposal can be used as a reference for implementation of a > new client for Ignite when dealing with such problems as: > >- Resolving of user-provided addresses; >- Initial connection to a cluster; >- Maintaining cluster connection; >- Connection recovery; >- Connection break handling. > > So take a look and let me know what you think guys. > > [1] - > https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle > > Best Regards, > Igor >
IEP-90 Ignite 3 Client Lifecycle
Hi, Igniters I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main idea is to define client lifecycle as well as core algorithms and mechanisms used by clients. This proposal can be used as a reference for implementation of a new client for Ignite when dealing with such problems as: - Resolving of user-provided addresses; - Initial connection to a cluster; - Maintaining cluster connection; - Connection recovery; - Connection break handling. So take a look and let me know what you think guys. [1] - https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle Best Regards, Igor
