If there are too many data related to one affinity key, even more than the
capacity of one node.
Will Ignite automatically split that data and stored them in several nodes?
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/How-to-avoid-data-skew-in-collocate-data-
Got you. thanks.
On 27 October 2016 at 09:30, vkulichenko
wrote:
> I changed the definition of the class (generic types, in particular).
> Compilation will be broken for those who already use KafkaStreamer.
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x
No, for that you will have to implement your own affinity function. But there
are a lot of cases that you should keep in mind. What if the node fails, how
the data will be redistributed? What if new node joins, will it take some of
the data (in other words, will you be able to scale)? Etc.
-Val
Looks like you provided everything except the model classes I asked for :)
Anyway, that's a lot of code and it's very hard to read it here. Can you
create a small project on GitHub that I can run to reproduce the problem?
This way I will be able to help you.
-Val
--
View this message in context
Do you mean that company in the object is just the company ID, not the actual
company object? If so, then this is the correct model and I don't see any
issues with loading it. Just load persons and companies separately and then
join using SQL queries.
-Val
--
View this message in context:
http
I changed the definition of the class (generic types, in particular).
Compilation will be broken for those who already use KafkaStreamer.
-Val
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Kafka-Streamer-tp8432p8532.html
Sent from the Apache Ignite Users maili
HI Val,
Thanks for your reply. Relation will be one to many.
My underlying data base is hbase. currently i am fetching the regions which
needs to loaded into ignite and broadcasting the region information.
Each IgniteCallable instance scan a region and load into Ignite using data
streamer.
Get
Looks perfect. Could you please elaborate "this change breaks API
compatibility" ? Thanks
On 27 October 2016 at 01:42, vkulichenko
wrote:
> Hi,
>
> OK, I see now what you mean and it looks like this is not supported right
> now. I created a ticket [1], however the way I propose it will break API
Hi,
1.8 is supposed to be released in November. You may want to subscribe to the
dev list and keep an eye on "Apache Ignite 1.8 Release” discussion going there.
—
Denis
> On Oct 26, 2016, at 6:16 PM, Shawn Du wrote:
>
> Hi Andrey Mashenkov,
>
> Thanks, I will have a try and apply these loca
I would like to pre-distribution of data according to a certain algorithm to
the designated node.
Is there an API can be used to put data to a specified node?
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/How-to-avoid-data-skew-in-collocate-data-with-data-tp84
Hi Andrey Mashenkov,
Thanks, I will have a try and apply these locally.
Do you know 1.8’s release plan?
Thanks
Shawn
发件人: Andrey Mashenkov [mailto:amashen...@gridgain.com]
发送时间: 2016年10月26日 20:53
收件人: user@ignite.apache.org
主题: Re: 答复: ignite used too much memory
Hi, Shawn D
Here is the ticket for this issue:
https://issues.apache.org/jira/browse/IGNITE-4142
-Val
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/AssertionError-tp8321p8525.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Hi,
OK, I see now what you mean and it looks like this is not supported right
now. I created a ticket [1], however the way I propose it will break API
compatibility, so can be done only in 2.0. If you have any other
suggestions, feel free to add comments.
For now you can create your own consumer
Hi,
Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.
Y.G wrote
> C:\Project\IgniteTest\apache-ignite-1.7.0-src\mod
Hi,
Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.
navy.chi wrote
> when I injected the data through IgniteDataS
Hi AL,
If there is heap memory consumption, that doesn't actually mean that the
memory is consumed by Ignite data. JVM will generally try to avoid GC unless
this is really needed or some thresholds are reached, so it can be just some
garbage that will be eventually removed. If you don't have any o
Hi,
If you have such data distribution, then most likely your collocation
strategy is incorrect or at least not optimal. If your number of partitions
is much bigger than number of nodes and number of unique affinity keys is
much bigger than number of partitions, such skew will be minimized. "Much
Hi Abhishek,
If you disable GC, you will run out of memory very quickly. Can you please
clarify why you want to do this?
-Val
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/jvm-setting-to-turn-off-the-Garbage-collector-tp8508p8520.html
Sent from the Apache Ign
Hi,
You can always deserialize a BinaryObject by calling the deserialize()
method.
-Val
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Cant-events-listen-from-client-Node-tp8470p8519.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Hi Anil,
Is there one-to-one relation between person and address/company? If yes,
then you just load three rows from different tables and create an object
based on fetch information. You can load data via IgniteDataStreamer or
within your implementation of CacheStore:
https://apacheignite.readme.i
HI,
What is the best way to load a normalized data from a db into flatten
object into ignite cache.
Ex :
CacheObject {
// person details
// person address
// person comany infomraiton
}
both in RDBMS and nosql point of view
Thanks.
Hello.
Thank you for your help. I saw that problem was in my servers configs. Now I
enable event on all servers and start get events on client, but now i have
another problem.
In listener in event.newValue() I get BinaryObjectImpl and I want to get
value of my Class type.
How could I convert this?
Hi,
No, it will not help. Because affinity function have not information about
data volume into partition.
The purpose of the function (RendezvousAffinityFunction or
FairAffinityFunction) is distribute evenly partition quantity between nodes
(but not data). You can to deeper understand affinity co
For some reason, I can not choose more selective affinty key than current.
If I increase number of partition, will it help me to avoid the case that
keys 553, 551, 554, 550 hitting into one node?
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/How-to-avoid-data-
Hi,
Could you please, provide your gradle file?
On Mon, Oct 24, 2016 at 9:13 PM, chevy wrote:
> *My current dependency looks like below [1](I am getting error pop up in
> eclipse when I use name in exclude as you have suggested). But I still get
> the same exception [2] mentioned below -*
>
>
>
Thanks Vladislav
From: Vladislav Pyatkov [mailto:vldpyat...@gmail.com]
Sent: Wednesday, October 26, 2016 5:08 AM
To: user@ignite.apache.org
Subject: Re: Evicted entry appears in Write-behind cache
Hi,
Yes, write behind buffer gets locks on entries which stores between loading to
CacheStore. So
Hi, Shawn Du
It seems you faced next 2 bugs.
First bug: High memory utilization using OffHeap with ExpirePolicy. Issue
has a fix, but it is not merged to master yet. See:
https://issues.apache.org/jira/browse/IGNITE-3840.
Second bug: TTL Manager continue track evicted (and removed) entries,
https
Hi,
Is there any specific setting to turn off the Garbage collector in JVM that
is invoked from C++ application?
Regards
Abhishek
Hi experts,
Can anyone help to explain ignite memory model?
Now I tried following ways, but no effect.
1) Remove all indexes.
2) enable swap. I see more than 800M’s data are stored in swapspace
directory.
3) Don’t cache short life entries. All cache with fifo evict poli
Hi,
You can increase number of partition through affinity function
configuration, but default value (1024 partitions) would be enough.
...
I do not shure than FairAffinityFunction will help to you, in case if keys
553, 551, 554, 550 will hit into one node.
How about choose more sel
This has been resolved Val. Thanks
On 26 October 2016 at 14:58, vdpyatkov wrote:
> Hi Anil,
>
> I doubt, about this fields can serialize correctly:
>
> private Scan scan;
> private QueryPlan queryPlan;
>
> You need will get rid of this fields from serialized object.
>
>
>
> --
> View this messag
Hi Anil,
I doubt, about this fields can serialize correctly:
private Scan scan;
private QueryPlan queryPlan;
You need will get rid of this fields from serialized object.
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Loading-Hbase-data-into-Ignite-tp8209p8502
Hi
This is the output of jmap histo:live .
Any useful information? There are about 300k cache entries. Also I enable
swap.
Each entry costs 10K memory in average. But each entry only has 50 bytes at
most.
Any help will be appreciated.
num #instances #bytes class
Hi,
Yes, write behind buffer gets locks on entries which stores between loading
to CacheStore. So that this entries cannot be evicted.
If "get" invocation can not find value in cache, it will try to get value
from CacheStore.
On Fri, Oct 14, 2016 at 6:55 PM, Pradeep Badiger
wrote:
> I have som
Hi Bob,
Make sure ignite-indexing attends in classpath.
And try to remove ccfg.setQueryEntities(qryEntities) from cache
configuration, when you using cfg.setIndexedTypes(Kc21Key.class,
Kc21.class).
On Wed, Oct 26, 2016 at 6:19 AM, 胡永亮/Bob wrote:
> Hi Vladislav
>
> First, thank you for you
35 matches
Mail list logo
