Ilya -
Thank you for your kind help.
Do you mind sharing your server configuration? I re-run with your
configuration, it cost more than 60 minutes to load 4000 records.
And I increased data region size and checkout frequency, they improve a bit,
but still too slow.
According to my test, the
I have looked at the documentation and the code samples and nothing is
doing what I am trying to do. I want to be able to use the datastreamer to
load 3 or 4 TABLES in a cache for an application that we use. If I create
the tables using a create table syntax how do attach a datastreamer to the
di
Hello Antonio!
I canonly observe 'timeout' checkpoints which is good news - you are not
running out of checkpoint buffer.
Otherwise, maybe you are hitting actual performance limit, i.e., your
system is saturated for good at this point. What is total amount of data
per node at this time? What's th
Hello again!
For added benefits, ScanQuery can filter entries on data node for you, and
return only the IDs needed for pruning.
Regards,
--
Ilya Kasnacheev
пт, 1 мар. 2019 г. в 20:08, Ilya Kasnacheev :
> Hello!
>
> 1. I have no idea about your use case so it's up to you.
> 2. Please refer to
Hello!
1. I have no idea about your use case so it's up to you.
2. Please refer to ScanQuery(int) constructor.
3. ScanQueries are pretty solid, one partition should have around 40k
records assuming 1024 partitions/cluster, that's peanuts.
Regards,
--
Ilya Kasnacheev
пт, 1 мар. 2019 г. в 17:57,
Hello!
I assume we're still talking about your reproducer
https://github.com/RedBlackTreei/streamer.git
With your code and reduced data set of 2500
Total time:628120ms when using
cacheCfg.setSqlIndexMaxInlineSize(64);
devIdIdx.setInlineSize(96);
as opposed to Total time:820821ms with your s
I have tried to increase QueryIndex.setInlineSize and
CacheConfiguration.setSqlIndexMaxInlineSize to 128 256 and 512, but the
performace became worse.
Do I miss some configuration?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
I have tried to increase QueryIndex.setInlineSize and
CacheConfiguration.setSqlIndexMaxInlineSize to 128 256 and 512, but the
performace became worse.
Do I miss some configuration?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Hello!
>From the shared logs it seems that you spend time building indexes (which
are possibly not-inlined as we discussed) and I can see nothing related to
backups here.
Regards,
--
Ilya Kasnacheev
пт, 1 мар. 2019 г. в 17:55, Justin Ji :
> Thank for your reply!
> 1. No, I did not use FULL_SY
Ilya -
First of all, thank for your reply.
For your suggest, I have some questions:
1.Do you mean I should disable eviction policies?
2.How to scan partition by ScanQuery, I did not find an example in ignite.
3.If we have more 40 million records, does this manner has good
performance?
--
Sent f
Thank for your reply!
1. No, I did not use FULL_SYNC, because it will wait for write or commit to
complete on all participating remote nodes (primary and backup), so it may
lead to a drop of write performance, am I right? But I will try it.
2. Yes, please refer to the attachment, I dumped thread st
Hello Ilya.
I ran again the test from scratch with fixed rate at 4000 msg/sec. with the
environment variable IGNITE_MAX_INDEX_PAYLOAD_SIZE=66 and Cache:
1) PARTITIONED
2) TRANSACTIONAL
3) persistence enabled
4) backup=0
5) indexes on key and value
6) Data region 8 GB
7) Checkpoint buffer size 2 G
Hello!
Do you use FULL_SYNC for a chance? Can you provide thread dumps during
slowdown?
Regards,
--
Ilya Kasnacheev
пт, 1 мар. 2019 г. в 12:53, BinaryTree :
> Hi Igniters -
>
> I know backups will impact the performance of the cluster:
>
> If you use a PARTITIONED cache and the data loss is n
Hello!
You can do scanQuery on every partition periodically and remove old entries
along with their dependencies. This might be easier than this eviction
policy business. Just my 5c.
Regard,
--
Ilya Kasnacheev
пт, 1 мар. 2019 г. в 15:41, BinaryTree :
> I have a cache, it contains many datapoi
Hello!
If you are using two rows as @AffinityKeyMapped, you can join other tables
which use same two rows, BUT you can't join tables which use only first row
as @AffinityKeyMapped or only second row.
That's why you can join invoice_line to fact_purhcase_line - I guess they
both have invoiceId and
I have a cache, it contains many datapoint, the datapoint looks like:
dpId integer
devId String
name String
The datapoint relates to device, their relationship is one-to-many, and they
are connected according to devId, so the devId is the affinityKey.
The cache key is:
//key=devId + "_" + dpId
pr
Hi,
Most likely heap size is too low.
Try to increase Xmx up to 4Gb or higher or avoid G1GC usage on small heaps
as it is very sensitive to free heap memory.
Looks like you have Visor node (or may be web-console) in grid. Is OOM
happened only when Visor attached to grid?
On Fri, Mar 1, 2019 at 7
If you set the localPortRange to zero (property in the TcpDiscoverySpi), Ignite
will only start on the port number you specific. That way, if you bring up
another node it will fail to start. Though automating how your environment is
configured so this could never happen would probably be a bette
Hi Igniters -
I know backups will impact the performance of the cluster:
If you use a PARTITIONED cache and the data loss is not critical for you (for
example, when you have a backing cache store), consider disabling backups for
the cache. When backups are enabled, the cache engine has to main
Thank for your reply, the project was created, you can refer to :
https://github.com/RedBlackTreei/streamer.git
Related post:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Data-Streamer-Hung-after-a-period-td21161.html
Looking forward to your reply!
--
Sent from: http://apache-ignite-u
Hello Ilya,
you are right from 65681 to 82206 and the corresponding duration of
checkpoint from 1,6 sec to 9,6 sec it is not catastrophic. I just wanted to
highlight a noticeable increase in dirty pages in a short time despite the
constant rate, suggesting that in the long run the performance would
Thank for your reply, the project was created, you can refer to :
https://github.com/RedBlackTreei/streamer.git
Related post:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Data-Streamer-Hung-after-a-period-td21161.html
Looking forward to your reply!
--
Sent from: http://apache-ignite-u
22 matches
Mail list logo
