SQL insert if that would be possible.
Mikael
On 2020-11-14 23:35, Wolfgang Meyerle wrote:
Hi,
I have a question in regards to Apache Ignite performance tuning.
This is my config so far:
https://pastebin.com/NWDzY3RK
I plan to store 2 billion entries in the database. The machine that I
have
Hi,
I have a question in regards to Apache Ignite performance tuning.
This is my config so far:
https://pastebin.com/NWDzY3RK
I plan to store 2 billion entries in the database. The machine that I
have is not great, 32gb ram nothing more.
I wonder how I can tune the performance to speed up
Hi,
The root cause of the performance looks like in the change the default
count of atomic backups [1] on the 2.4 and later.
Please try to use IgniteAtomicSequence instead of the IgniteAtomicLong
to generate IDs.
[1]. org.apache.ignite.configuration.AtomicConfiguration#DFLT_BACKUPS
18.09.201
Hello!
Please collect profiling (e.g. using JFR) from both runs and search for hot
spots in both scenarios. If you find anything suspicious, please file a
ticket.
We can also look at profiling results for you.
Unfortunately I'm not that good with yardstick to run your benchmarks
locally. It woul
Hi
Does anyone have any idea why there is such a drop in performance between
ignite versions?
_
This communication is intended only for the addressee(s) and may contain
confidential information. We do not waive any confidentiality by misd
:* Re: Ignite Performance - Adding new node is not improving
SQL query performance
Hi!
If the new node is not part of the baseline topology it will not have
any persisted data stored so any SQL query will not be of any use on
the node as it does not have any of the data (at least that is how I
query, the data will load to memory of
all these 5 nodes and will use the computing power of all. But now it seems, it
is not working like that.
Regards,
Favas
From: Mikael
Sent: Monday, September 9, 2019 3:21 PM
To: user@ignite.apache.org
Subject: Re: Ignite Performance - Adding new node is not
Hi!
If the new node is not part of the baseline topology it will not have
any persisted data stored so any SQL query will not be of any use on the
node as it does not have any of the data (at least that is how I
understand it, I could be wrong here).
If so you would need to add the new node
Hi,
I have an ignite cluster with 4 node(each with 8 core, 32 GB RAM and 30 GB
Disk) with native persistence enabled and added to baseline topology. There
are two sql table created and loaded with 120 GB data.
One of my test sql query is taking 8 second with this set up. Currently I am
trying
Hi shahidv,
Could you post please the details for configurations of caches, ideas of
queries and activation logs?
It's not easy to blindly give some recommendations to follow as there is
usually no silver bullet for SQL queries execution in a distributed
environment as you have to work on each sp
Hi,
*-when restarted node, It took 6 hrs to activate again, *
If you activated cluster on startup, before data loading, then after
restart, cluster is active.
*-queries are very slower, *
it depends. What SQL queries do you have? JOIN, AGG?
*-out of memory issue when loading data. *
Show your Cac
Hi,
We are looking for new database technologies because of some performance
issues in existing.
So we tried ignite with 1 node cluster (28gb ram), loaded 90 million records
to the cache.
But we faced several issues like,
-when restarted node, It took 6 hrs to activate again,
-queries are very s
Hello!
Can you provide EXPLAIN output (all rows) for this query?
I don't see @AffinityKeyMapped in your examples and doubt it will work with
QueryEntities anyway. Please try to use CacheKeyConfiguration instead to
specify affinity fields. However, I'm not sure if you can use two filelds
as affini
I have 250,000 records for each cache store and when I tried to join they are
pretty slow.
Is annotation -> @affinityKey, @SqlQueryField works in java version? I am
using JDK1.8 version AND NOT Spring version.
Can someone help how to use @affinitykey in group for below example in
'PARTIONED' wit
Hello!
Let's talk about persistence performance.
First of all, you have to understand how checkpointing works. It works by
writing a whole 4k page if you happen to add or update an object to it.
This means when you have a lot of updates there will be a lot of writes.
When you have a write-heavy p
I'm Using 2.6 on AWS. Like I mentioned, my Ignite cluster is running on i3
instances which have local storage, so burst shouldn't be a problem.
The trend I've noticed is that my writes-per-second increases, while the
size of each write decreases, and the number of PUT operations per second
and CPU
Hello!
This is somewhat expected since memory grids are very fast. Disk-based
database can't match memory grid's in-memory performance numbers.
Still, it should not degrade endlessly. What is the version that you are
on? I guess you may see improvements in the coming Apache Ignite 2.7.
DataStrea
I'm trying to seed about 500 million rows from a Spark DataFrame into a clean
Ignite database, but running into serious performance issues once Ignite
runs out of durable memory. I'm running 4 Ignite Nodes on Kubernetes cluster
backed by AWS i3.2xl instances (8 CPUs per node, 60 GB Memory, 2TB SSD
Hello!
I think you should expect that. When you do more parallel operations,
there's more locking and throughput decreases.
However, we don't know enough about your specific workload to really
understand what's going on.
Regards,
--
Ilya Kasnacheev
вт, 27 нояб. 2018 г. в 16:24, Naveen :
> Th
These are settings we have
maxSize=#{80L * 1024 * 1024 * 1024}
checkpointPageBufferSize=#{8L * 1024 * 1024 * 1024}
pageSize=#{4 * 1024}
Any clues we have for this dip in TPS
Thanks
Naveen
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Hi
We are using ignite 2.6
We are doing performance testing with JMeter, when we run only Query service
per instance, we were getting 1000 TPS and when we run both upserts and
queries, its starts with the same TPS, but comes down to 500 TPS which is
50% dip in TPS.
What could be the reason, we
Hi Andrew,
I have found what is causing problem here and moved it into a more specific
thread:
http://apache-ignite-users.70518.x6.nabble.com/QueryCursor-getAll-performance-is-very-very-low-td21691.html
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Hi Andrew,
I had analyzed this further and found out that actually ther cursor getAll
is making it slow. I have pointed this out in the thread below:
http://apache-ignite-users.70518.x6.nabble.com/QueryCursor-getAll-performance-is-very-very-low-td21691.html
--
Sent from: http://apache-ignite-u
Hi,
Why you think the performance is slow?
Ignite SQL has an H2 underneath and have some overhead as it is a distributed
system at first. So, local query latency can be slightly higher.
How do you measure query latency?
You may need to warm grid up to all components involved in query initialize.
Hi,
I have 200 caches loaded into a single data region. Among that I have a
customerCache which has 300K records. It has many indexed fields, including
customer number.
When I query using *SQLFieldsQuery *on this particular cache (where
condition has just customer number = ?), it takes around *28
Hi,
As I know Ignite provides its own benchmarks that used Yardstick framework.
Documentation:
https://apacheignite.readme.io/docs/perfomance-benchmarking
yardstick:
https://github.com/gridgain/yardstick
In Ignite sources exists IgniteAbstractBenchmark class that could be used
for creation of
Just like the subject title,has anyone tested the Ignite performance by
bigbench or other kind of benchmark util?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Hi Val,
Thanks for the quick response. I was checking at the network throughput for
the redis cache for one minute and the traffic is around 5GB, I can bet it
is not optimised, with one farm which has got 10 nodes.
Lets say if we have an Ignite cluster of 100 nodes how you would and check
the ma
Hi luqmanahmed,
Could you please tell me why do need 1200 nodes to get 8 gb in memory?
Is it replicated cache? Do you need 1200 nodes to serve high number of
requests/sec?
If yes then does each node act as a primary for whole data? If yes, do you
use load balancer to reroute the requests?
Thanks,
Hi Luqman,
I don't see why not. It will probably require pretty big cluster, but looks
like your Redis cluster is not very small either :) Ignite is a highly
scalable system, so you can test with smaller clusters of different sizes,
check what maximum throughput they provide and then extrapolate t
Hi Igniters,
I will try to keep it short as much as I can. We have 8 caches total size is
around 6GB to 8GB in redis and the cluster is made up of 1200 nodes. All the
operations are read-only. It is a very low latency system where each request
doesn't need to be more than 2ms, in very rare cases 3
o
> leverage with ignite.
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Apache-ignite-performance-tp12274p12328.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
:
http://apache-ignite-users.70518.x6.nabble.com/Apache-ignite-performance-tp12274p12328.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Hello Sweta Das!
1. It's not recommended to perform any performance tests on virtual
environment, because this environments have too many points where something
can work not as expected: memory swapping, virtual file systems, driver
problems and the effects from other virtual machines on the same h
Hi
We are evaluating apache ignite for compute and data grid on virtual machines.
Can anyone give me idea or inputs on below points-
1. Performance or drawbacks of ignite on virtual machines versus ignite
deployed on physical servers? Has anyone seen any overcommitting memory in VMs ?
2. Is there
[mailto:valentin.kuliche...@gmail.com]
Sent: 06 August 2016 06:20
To: user@ignite.apache.org
Subject: RE: Ignite performance
Hi Manuel,
Are the results in the latest table in milliseconds? I don't see significant
growth with the number of elements there, am I missing something? Actually your
query calcu
will depend on the
cache size. Makes sense?
-Val
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-tp6703p6825.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
ilto:yzhda...@apache.org]
Sent: 03 August 2016 15:28
To: Piubelli, Manuel [ICG-IT]
Cc: user@ignite.apache.org
Subject: Re: Ignite performance
How many concurrent queries do you have? Or in other words - how many threads
does your executor have? If it has several ones then I understand why CPU load
gt;
>
>
> Ignite configuration:
>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
>
>
>
>
>
>
>
>
>
>
> class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"
127.0.0.1:47500..47509
From: Yakov Zhdanov [mailto:yzhda...@apache.org]
Sent: 03 August 2016 13:37
To: user@ignite.apache.org
Subject: Re: Ignit
Manuel,
The numbers you are seeing are pretty strange to me.
Is it correct that you run your query in a loop giving enough time for the
whole cluster to warmup and only then take the final measurements?
I also do not understand why CPU load is 400% which may be interpreted as
full (correct?). Thi
is there some obvious setting i am missing?
I could not find benchmarks on similar operations.
2.What is the advised method to run fork-join style computations on ignite
without moving data?
Thank you
Manuel
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignit
Hello,
I am currently benchmarking Apache Ignite for a near real-time application and
simple operations seem to be excessively slow for a relatively small sample
size. The following is giving the setup details and timings - please see 2
questions at the bottom.
Setup:
* Cache mode: Parti
the confusion.
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-improvements-tp5623p5742.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> In this case if I use the same key in different transactiosn would it impact
> performance or it will have no impact?
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-improvements-tp5623p5710.html
> S
yes, I understand that.
In production we plan to use that, however one question:-
In this case if I use the same key in different transactiosn would it impact
performance or it will have no impact?
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite
yes, I understand that.
In production we plan to use that, however one question:-
In this case if I use the same key in different transactiosn would it impact
performance or it will have no impact?
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite
TIC Serializable give better performance?
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-improvements-tp5623p5684.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Does OPTIMISTIC Serializable give better performance?
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-improvements-tp5623p5684.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
gt; TPS).
>
> My Access point can take in 1 million per second so its not a bottleneck. I
> am expecting more than 5000 and for transactions to scale as I keep adding
> nodes.
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-igni
gt;
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-improvements-tp5623p5680.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Vladislav Pyatkov
-users.70518.x6.nabble.com/Ignite-performance-improvements-tp5623p5680.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
ta. is there any work around for this?
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-improvements-tp5623p5632.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Vladislav Pyatkov
message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-improvements-tp5623p5632.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
SYNC enabled, All our transactional caches are partitioned.
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-improvements-tp5623.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Thank you for your reply.
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-about-put-get-tp2782p2786.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
[1] https://ignite.apache.org/benchmarks/ignite-vs-hazelcast.html
[2] https://github.com/apacheignite/yardstick-ignite
Regards,
Denis
On 1/30/2016 8:33 AM, ght230 wrote:
Who can tell me the Ignite Performance about put/get TPS
(key: 16-char-long java.lang.String, value: java.lang.Integer)
and the time spent
Who can tell me the Ignite Performance about put/get TPS
(key: 16-char-long java.lang.String, value: java.lang.Integer)
and the time spent of getting one value?
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-about-put-get-tp2782.html
Sent
, Ignite is designed and implemented in a way that will allow to
use all the CPU and other resources available on a machine.
Regards,
Denis
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-performance-on-multicore-tp2020p2024.html
Sent from the Apache Ignite
59 matches
Mail list logo
