Re: Ignite cache partition size
Cool, thanks .
I have a quite big cluster with 100 nodes, so I guess 25600 will be good
partition?
I am using ignite as database.
I am creating a table with 20 million rows,each row has 1450 columns.
Is this a good idea?
Thank you!
On Sat, Oct 26, 2019 at 9:10 AM Prasad Bhalerao <
prasadbhalerao1...@gmail.com> wrote:
> Please check this link...
>
>
> https://apacheignite.readme.io/docs/affinity-collocation#section-affinity-function
>
>
> Example from the ignite doc.
>
>
> // Preparing Apache Ignite node configuration.IgniteConfiguration cfg = new
> IgniteConfiguration();
> // Creating a cache configuration.CacheConfiguration cacheCfg = new
> CacheConfiguration("myCache");
> // Creating the affinity function with custom
> setting.RendezvousAffinityFunction affFunc = new RendezvousAffinityFunction();
> affFunc.setExcludeNeighbors(true);
> affFunc.setPartitions(2048);
> // Applying the affinity function configuration.cacheCfg.setAffinity(affFunc);
> // Setting the cache
> configuration.cfg.setCacheConfiguration(cacheCfg);
>
>
>
> On Sat 26 Oct, 2019, 7:42 PM Andrey Dolmatov
>> Try to implement you're own affinity function. If it map any key to
>> numbers from 1 to n, you have n partitions
>>
>> On Sat, Oct 26, 2019, 09:46 codeboyyong wrote:
>>
>>> Hi can you please tell me how to config this ?
>>> "number of cache partitions"
>>>
>>> Thank You
>>> Yong
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>
Re: Ignite cache partition size
Please check this link...
https://apacheignite.readme.io/docs/affinity-collocation#section-affinity-function
Example from the ignite doc.
// Preparing Apache Ignite node configuration.IgniteConfiguration cfg
= new IgniteConfiguration();
// Creating a cache configuration.CacheConfiguration cacheCfg
= new CacheConfiguration("myCache");
// Creating the affinity function with custom
setting.RendezvousAffinityFunction affFunc = new
RendezvousAffinityFunction();
affFunc.setExcludeNeighbors(true);
affFunc.setPartitions(2048);
// Applying the affinity function configuration.cacheCfg.setAffinity(affFunc);
// Setting the cache configuration.cfg.setCacheConfiguration(cacheCfg);
On Sat 26 Oct, 2019, 7:42 PM Andrey Dolmatov Try to implement you're own affinity function. If it map any key to
> numbers from 1 to n, you have n partitions
>
> On Sat, Oct 26, 2019, 09:46 codeboyyong wrote:
>
>> Hi can you please tell me how to config this ?
>> "number of cache partitions"
>>
>> Thank You
>> Yong
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
Throttling getAll
Hello, I've benchmarked my grid for users (clients) to do getAll with upto 100 keys at a time. My value objects tend to be quite large and my worry is if there are errant clients might at times do a getAll with a larger number of keys - say 1000. If that happens I worry about GC issues/humongous objects/OOM on the grid. Is there a way to configure the grid to auto-split these requests into smaller batches (smaller number of keys per batch) or rejecting them? Thanks, Abhishek
Re: Drop index do not release memory used ?
Yes the data pages in reuse list are reused for new objects. If you drop table, you get back all pages used in reuse list. But it seems that it's not true for drop indexes Le sam. 26 oct. 2019 à 16:18, Andrey Dolmatov a écrit : > Ignite believe that Ignite will reuse that memory. But it is a question, > does Ignite reuse index data blocks for data blocks. > > On Fri, Oct 25, 2019, 15:28 yann.blaz...@externe.bnpparibas.com < > yann.blaz...@externe.bnpparibas.com> wrote: > >> Hello all. >> >> If you can remember, I found a way to compute the real size of memory >> used in offheap, using the reuselist size. >> >> As I'm facing some limits on my hardware, I'm trying to optimize my >> memory consumption, in pure memory, no persistence on hdd or ssd. >> >> For that as I have to execute plenty of request on my stored data, I saw >> that indexes consumes a lot of memory. >> >> To improve that, in my algorithm I tried to create tables with only pk, >> no indexes at first. >> >> Then before each request, I tried to create indexes , execute request, >> then drop indexes. >> >> What I see, is that drop index do not release memory... >> >> Everything is release only when we drop table. >> >> Is this normal ? >> >> >> Thnaks and regards. >> This message and any attachments (the "message") is >> intended solely for the intended addressees and is confidential. >> If you receive this message in error,or are not the intended >> recipient(s), >> please delete it and any copies from your systems and immediately notify >> the sender. Any unauthorized view, use that does not comply with its >> purpose, >> dissemination or disclosure, either whole or partial, is prohibited. >> Since the internet >> cannot guarantee the integrity of this message which may not be reliable, >> BNP PARIBAS >> (and its subsidiaries) shall not be liable for the message if modified, >> changed or falsified. >> Do not print this message unless it is necessary, consider the >> environment. >> >> >> -- >> >> Ce message et toutes les pieces jointes (ci-apres le "message") >> sont etablis a l'intention exclusive de ses destinataires et sont >> confidentiels. >> Si vous recevez ce message par erreur ou s'il ne vous est pas destine, >> merci de le detruire ainsi que toute copie de votre systeme et d'en >> avertir >> immediatement l'expediteur. Toute lecture non autorisee, toute >> utilisation de >> ce message qui n'est pas conforme a sa destination, toute diffusion ou >> toute >> publication, totale ou partielle, est interdite. L'Internet ne permettant >> pas d'assurer >> l'integrite de ce message electronique susceptible d'alteration, BNP >> Paribas >> (et ses filiales) decline(nt) toute responsabilite au titre de ce message >> dans l'hypothese >> ou il aurait ete modifie, deforme ou falsifie. >> N'imprimez ce message que si necessaire, pensez a l'environnement. >> >>
Re: Drop index do not release memory used ?
Ignite believe that Ignite will reuse that memory. But it is a question, does Ignite reuse index data blocks for data blocks. On Fri, Oct 25, 2019, 15:28 yann.blaz...@externe.bnpparibas.com < yann.blaz...@externe.bnpparibas.com> wrote: > Hello all. > > If you can remember, I found a way to compute the real size of memory used > in offheap, using the reuselist size. > > As I'm facing some limits on my hardware, I'm trying to optimize my memory > consumption, in pure memory, no persistence on hdd or ssd. > > For that as I have to execute plenty of request on my stored data, I saw > that indexes consumes a lot of memory. > > To improve that, in my algorithm I tried to create tables with only pk, no > indexes at first. > > Then before each request, I tried to create indexes , execute request, > then drop indexes. > > What I see, is that drop index do not release memory... > > Everything is release only when we drop table. > > Is this normal ? > > > Thnaks and regards. > This message and any attachments (the "message") is > intended solely for the intended addressees and is confidential. > If you receive this message in error,or are not the intended recipient(s), > please delete it and any copies from your systems and immediately notify > the sender. Any unauthorized view, use that does not comply with its > purpose, > dissemination or disclosure, either whole or partial, is prohibited. Since > the internet > cannot guarantee the integrity of this message which may not be reliable, > BNP PARIBAS > (and its subsidiaries) shall not be liable for the message if modified, > changed or falsified. > Do not print this message unless it is necessary, consider the environment. > > > -- > > Ce message et toutes les pieces jointes (ci-apres le "message") > sont etablis a l'intention exclusive de ses destinataires et sont > confidentiels. > Si vous recevez ce message par erreur ou s'il ne vous est pas destine, > merci de le detruire ainsi que toute copie de votre systeme et d'en avertir > immediatement l'expediteur. Toute lecture non autorisee, toute utilisation > de > ce message qui n'est pas conforme a sa destination, toute diffusion ou > toute > publication, totale ou partielle, est interdite. L'Internet ne permettant > pas d'assurer > l'integrite de ce message electronique susceptible d'alteration, BNP > Paribas > (et ses filiales) decline(nt) toute responsabilite au titre de ce message > dans l'hypothese > ou il aurait ete modifie, deforme ou falsifie. > N'imprimez ce message que si necessaire, pensez a l'environnement. > >
Re: Ignite cache partition size
Try to implement you're own affinity function. If it map any key to numbers from 1 to n, you have n partitions On Sat, Oct 26, 2019, 09:46 codeboyyong wrote: > Hi can you please tell me how to config this ? > "number of cache partitions" > > Thank You > Yong > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Ignite cache partition size
Hi can you please tell me how to config this ? "number of cache partitions" Thank You Yong -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
