Re: .NET ContinuousQuery lose cache entries
Hello again!
I have filed a ticket: https://issues.apache.org/jira/browse/IGNITE-9787
Regards,
--
Ilya Kasnacheev
вт, 2 окт. 2018 г. в 20:16, Ilya Kasnacheev :
> Hello!
>
> I have spent some time tinkering with your case, and I think that an issue
> exists.
>
> If you have a partitioned cache with non-local continuous query, it works.
> If you have a replicated cache with non-local continuous query, it works.
> If you have a replicated cache with local continuous query and synchronous
> rebalancing, it also works.
> But if you have replicated cache with local continuous query and
> non-synchronous rebalancing (as you do), you will see the gap in data that
> you are observing.
>
> I still the best course of action in your case is to set,
> RebalanceMode = CacheRebalanceMode.Sync
> for "Test" cache.
>
> However, I'm not 100% confident so I recommended running your test in a
> loop for a couple minutes.
>
> I will ponder if this is actually a bug, and maybe fill an issue against
> Apache Ignite.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 1 окт. 2018 г. в 14:36, Alew :
>
>> Hi!
>>
>> Tried, but the issue still exists.
>>
>> https://monosnap.com/file/UUbYX4RUyXPZyPxKx97hTwSUSvNtye
>>
>> On 01/10/2018 13:06, Ilya Kasnacheev wrote:
>>
>> Hello!
>>
>> Setting BufferSize to 1 seems to fix your reproducer's problem:
>>
>> new ContinuousQuery(new CacheListener(skippedItems,
>> doubleDelete), true) { BufferSize = 1}
>>
>> but the recommendation is to avoid doing anything synchronous from
>> Continuous Query's body. Better offload any non-trivial processing to other
>> threads operating asynchronously.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> сб, 29 сент. 2018 г. в 5:18, Alew :
>>
>>> Hi, attached a reproducer.
>>> Turn off logs to fix the issue. Slow log is not the only reason. More
>>> nodes in a cluster lead to the same behaviour.
>>> Who is responsible for the behavior? Is it .net, java, bad docs or me?
>>>
>>> On 24/09/2018 20:03, Alew wrote:
>>> > Hi!
>>> >
>>> > I need a way to consistently get all entries in a replicated cache and
>>> > then all updates for them while application is working.
>>> >
>>> > I use ContinuousQuery for it.
>>> >
>>> > var cursor = cache.QueryContinuous(new ContinuousQuery>> > byte[]>(new CacheListener(), true),
>>> > new ScanQuery>> byte[]>()).GetInitialQueryCursor();
>>> >
>>> > But I have some issues with it.
>>> >
>>> > Sometimes cursor returns only part of entries in a cache and cache
>>> > listener does not return them either.
>>> >
>>> > Sometimes cursor and cache listener return the same entry both.
>>> >
>>> > Issue somehow related to amount of work the nodes have to do and
>>> > amount of time between start of the publisher node and subscriber node.
>>> >
>>> > There are more problems if nodes start at the same time.
>>> >
>>> > Is there a reliable way to do it without controling order of node
>>> > start and pauses between them?
>>> >
>>> >
>>>
>>>
>>
Re: .NET ContinuousQuery lose cache entries
Hello!
I have spent some time tinkering with your case, and I think that an issue
exists.
If you have a partitioned cache with non-local continuous query, it works.
If you have a replicated cache with non-local continuous query, it works.
If you have a replicated cache with local continuous query and synchronous
rebalancing, it also works.
But if you have replicated cache with local continuous query and
non-synchronous rebalancing (as you do), you will see the gap in data that
you are observing.
I still the best course of action in your case is to set,
RebalanceMode = CacheRebalanceMode.Sync
for "Test" cache.
However, I'm not 100% confident so I recommended running your test in a
loop for a couple minutes.
I will ponder if this is actually a bug, and maybe fill an issue against
Apache Ignite.
Regards,
--
Ilya Kasnacheev
пн, 1 окт. 2018 г. в 14:36, Alew :
> Hi!
>
> Tried, but the issue still exists.
>
> https://monosnap.com/file/UUbYX4RUyXPZyPxKx97hTwSUSvNtye
>
> On 01/10/2018 13:06, Ilya Kasnacheev wrote:
>
> Hello!
>
> Setting BufferSize to 1 seems to fix your reproducer's problem:
>
> new ContinuousQuery(new CacheListener(skippedItems,
> doubleDelete), true) { BufferSize = 1}
>
> but the recommendation is to avoid doing anything synchronous from
> Continuous Query's body. Better offload any non-trivial processing to other
> threads operating asynchronously.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> сб, 29 сент. 2018 г. в 5:18, Alew :
>
>> Hi, attached a reproducer.
>> Turn off logs to fix the issue. Slow log is not the only reason. More
>> nodes in a cluster lead to the same behaviour.
>> Who is responsible for the behavior? Is it .net, java, bad docs or me?
>>
>> On 24/09/2018 20:03, Alew wrote:
>> > Hi!
>> >
>> > I need a way to consistently get all entries in a replicated cache and
>> > then all updates for them while application is working.
>> >
>> > I use ContinuousQuery for it.
>> >
>> > var cursor = cache.QueryContinuous(new ContinuousQuery> > byte[]>(new CacheListener(), true),
>> > new ScanQuery> byte[]>()).GetInitialQueryCursor();
>> >
>> > But I have some issues with it.
>> >
>> > Sometimes cursor returns only part of entries in a cache and cache
>> > listener does not return them either.
>> >
>> > Sometimes cursor and cache listener return the same entry both.
>> >
>> > Issue somehow related to amount of work the nodes have to do and
>> > amount of time between start of the publisher node and subscriber node.
>> >
>> > There are more problems if nodes start at the same time.
>> >
>> > Is there a reliable way to do it without controling order of node
>> > start and pauses between them?
>> >
>> >
>>
>>
>
Re: .NET ContinuousQuery lose cache entries
Hi!
Tried, but the issue still exists.
https://monosnap.com/file/UUbYX4RUyXPZyPxKx97hTwSUSvNtye
On 01/10/2018 13:06, Ilya Kasnacheev wrote:
Hello!
Setting BufferSize to 1 seems to fix your reproducer's problem:
newContinuousQuery(newCacheListener(skippedItems,doubleDelete),true){BufferSize=1}
but the recommendation is to avoid doing anything synchronous from
Continuous Query's body. Better offload any non-trivial processing to
other threads operating asynchronously.
Regards,
--
Ilya Kasnacheev
сб, 29 сент. 2018 г. в 5:18, Alew >:
Hi, attached a reproducer.
Turn off logs to fix the issue. Slow log is not the only reason. More
nodes in a cluster lead to the same behaviour.
Who is responsible for the behavior? Is it .net, java, bad docs or me?
On 24/09/2018 20:03, Alew wrote:
> Hi!
>
> I need a way to consistently get all entries in a replicated
cache and
> then all updates for them while application is working.
>
> I use ContinuousQuery for it.
>
> var cursor = cache.QueryContinuous(new ContinuousQuery byte[]>(new CacheListener(), true),
> new ScanQuery()).GetInitialQueryCursor();
>
> But I have some issues with it.
>
> Sometimes cursor returns only part of entries in a cache and cache
> listener does not return them either.
>
> Sometimes cursor and cache listener return the same entry both.
>
> Issue somehow related to amount of work the nodes have to do and
> amount of time between start of the publisher node and
subscriber node.
>
> There are more problems if nodes start at the same time.
>
> Is there a reliable way to do it without controling order of node
> start and pauses between them?
>
>
Re: .NET ContinuousQuery lose cache entries
Hello!
Setting BufferSize to 1 seems to fix your reproducer's problem:
new ContinuousQuery(new CacheListener(skippedItems,
doubleDelete), true) { BufferSize = 1}
but the recommendation is to avoid doing anything synchronous from
Continuous Query's body. Better offload any non-trivial processing to other
threads operating asynchronously.
Regards,
--
Ilya Kasnacheev
сб, 29 сент. 2018 г. в 5:18, Alew :
> Hi, attached a reproducer.
> Turn off logs to fix the issue. Slow log is not the only reason. More
> nodes in a cluster lead to the same behaviour.
> Who is responsible for the behavior? Is it .net, java, bad docs or me?
>
> On 24/09/2018 20:03, Alew wrote:
> > Hi!
> >
> > I need a way to consistently get all entries in a replicated cache and
> > then all updates for them while application is working.
> >
> > I use ContinuousQuery for it.
> >
> > var cursor = cache.QueryContinuous(new ContinuousQuery > byte[]>(new CacheListener(), true),
> > new ScanQuery()).GetInitialQueryCursor();
> >
> > But I have some issues with it.
> >
> > Sometimes cursor returns only part of entries in a cache and cache
> > listener does not return them either.
> >
> > Sometimes cursor and cache listener return the same entry both.
> >
> > Issue somehow related to amount of work the nodes have to do and
> > amount of time between start of the publisher node and subscriber node.
> >
> > There are more problems if nodes start at the same time.
> >
> > Is there a reliable way to do it without controling order of node
> > start and pauses between them?
> >
> >
>
>
Re: .NET ContinuousQuery lose cache entries
Hi, attached a reproducer. Turn off logs to fix the issue. Slow log is not the only reason. More nodes in a cluster lead to the same behaviour. Who is responsible for the behavior? Is it .net, java, bad docs or me? On 24/09/2018 20:03, Alew wrote: Hi! I need a way to consistently get all entries in a replicated cache and then all updates for them while application is working. I use ContinuousQuery for it. var cursor = cache.QueryContinuous(new ContinuousQuerybyte[]>(new CacheListener(), true), new ScanQuery()).GetInitialQueryCursor(); But I have some issues with it. Sometimes cursor returns only part of entries in a cache and cache listener does not return them either. Sometimes cursor and cache listener return the same entry both. Issue somehow related to amount of work the nodes have to do and amount of time between start of the publisher node and subscriber node. There are more problems if nodes start at the same time. Is there a reliable way to do it without controling order of node start and pauses between them? <>
Re: .NET ContinuousQuery lose cache entries
Hello! Do you have some kind of reproducer project? It's hard to say otherwise what is happening in your case. Regards, -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
.NET ContinuousQuery lose cache entries
Hi! I need a way to consistently get all entries in a replicated cache and then all updates for them while application is working. I use ContinuousQuery for it. var cursor = cache.QueryContinuous(new ContinuousQuerybyte[]>(new CacheListener(), true), new ScanQuery()).GetInitialQueryCursor(); But I have some issues with it. Sometimes cursor returns only part of entries in a cache and cache listener does not return them either. Sometimes cursor and cache listener return the same entry both. Issue somehow related to amount of work the nodes have to do and amount of time between start of the publisher node and subscriber node. There are more problems if nodes start at the same time. Is there a reliable way to do it without controling order of node start and pauses between them?
