Hi Denis, This IEP is mostly about how we work with our own indexes and pages. So 3rd party DB is out of question.
On Thu, Jun 21, 2018 at 10:38 PM Denis Magda <dma...@apache.org> wrote: > Vladimir, > > As I see from the IEP, this data loading technique is supposed to be used > for deployments with Ignite persistence enabled. Is it possible to > generalize this solution and use for pure in-memory and in-memory + 3rd > party DB scenarios? > > -- > Denis > > On Wed, Jun 20, 2018 at 8:08 AM Vladimir Ozerov <voze...@gridgain.com> > wrote: > > > Igniters, > > > > Initial data load is one of the most important use cases for our product. > > This is one the first things user try to do with Ignite. And if it takes > > too much time, it is very likely that user will look for other solutions. > > > > We did good progress in this area recently. Specifically - a set of > > internal improvements on our indexes, steaming mode for JDBC driver, COPY > > command. But our internals are still not very efficient - every single > > update goes through the whole set of Ignite components, such as page > cache, > > free-lists, BTrees, etc.. > > > > I created IEP-22 [1]. It's goal is to implement special direct data load > > mode which will bypass our page cache and use alternative algorithm for > > index updates. Together with COPY command and streaming this improvement > > will allow Ignite to load data with very high speed. > > > > Please review the IEP and share your comments. > > > > Vladimir. > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-22%3A+Direct+Data+Load > > >
