Hi Yakov, Great to have you here! :) Thanks for the inputs. My comments are below.
*Alexey*, could you please comment on the points 3 and 4? -Val On Sat, Oct 10, 2020 at 4:36 PM Yakov Zhdanov <[email protected]> wrote: > Hi! > I am back! > > Here are several ideas on top of my mind for Ignite 3.0 > 1. Client nodes should take the config from servers. Basically it should be > enough to provide some cluster identifier or any known IP address to start > a client. > [Val] We will have a meta-store that will store the cluster-wide configuration (among other things). Whether it's a server or a client, any new node will be able to fetch the required information from there. In practice, it will work how you describe - just provide an IP address and you are good to go. > > 2. Thread per partition. Again. I strongly recommend taking a look at how > Scylla DB operates. I think this is the best distributed database threading > model and can be a perfect fit for Ignite. Main idea is in "share nothing" > approach - they keep thread switches to the necessary minimum - messages > reading or updating data are processed within the thread that reads them > (of course, the sender should properly route the message, i.e. send it to > the correct socket). Blocking operations such as fsync happen outside of > worker (shard) threads. > This will require to split indexes per partition which should be quite ok > for most of the use cases in my view. Edge cases with high-selectivity > indexes selecting 1-2 rows for a value can be sped up with hash indexes or > even by secondary index backed by another cache. > > 3. Replicate physical updates instead of logical. This will simplify logic > running on backups to zero. Read a page sent by the primary node and apply > it locally. Most probably this change will require pure thread per > partition described above. > 4. Merge (all?) network components (communication, disco, REST, etc?) and > listen to one port. > [Val] This is in plans at least for the communication and disco. We'll see how it goes for other components. > > 5. Revisit transaction concurrency and isolation settings. Currently some > of the combinations do not work as users may expect them to and some look > just weird. > [Val] Also planned. > > Ready to discuss the above and other points. > > Thanks! > Yakov >
