Re:Re: Re: Parallel execution CreateTable
Thanks , Stephen Based on your suggestion, we are optimizing for the first and second issues. For the third suggestion, our business has many 'temporary tables', which have different column and be detroyed when used. So I'm afraid it is hard for us to avoid create/destory tables. The column is dynamic so it cannot be written in XML either . I’ not sure whether createCaches() can meets the requirements or not . Thank you for your help！ At 2023-10-17 17:18:56, "Stephen Darlington" wrote: There's a lot going on there, and I don't have the time to fully analyse, but I will note a few things: You have a lot of very poorly optimised queries (full table scans) I see a few "long JVM pauses" which suggests poorly configured garbage collector Ignite is not designed for large numbers of cache creation/destruction. Lots of simultaneous queries are fine, because those do not require a global lock If you need to create a lot of caches at once, you can use XML or a programming language and use the createCaches() method On Tue, 17 Oct 2023 at 10:43, y wrote: Here is cluster log（from one node） and createtable sql（Appendix）. The Cluster has 10 nodes and do stress testing with 50 thread .It means there 50 query or createtable operations at the same time. Notice that when create table，the cluster node log has many ‘Thread - WAITING’. That's not normal, right? At 2023-10-17 15:19:33, "Stephen Darlington" wrote: Can you share some more information about your cluster? There is no way that creating a cache should take so long. On Tue, 17 Oct 2023 at 03:51, y wrote: Hello. Everyone！ Creating table statements is executed synchronously and will block other DDL statements. There are serious performance issues in concurrent environments. It takes one minute to create one table. If there is any way to solve this problem, such as changing it to serial execution？ Thanks, Tianyu-Hu
Re: Re: Parallel execution CreateTable
There's a lot going on there, and I don't have the time to fully analyse, but I will note a few things: - You have a lot of very poorly optimised queries (full table scans) - I see a few "long JVM pauses" which suggests poorly configured garbage collector - Ignite is not designed for large numbers of cache creation/destruction. Lots of simultaneous queries are fine, because those do not require a global lock - If you need to create a lot of caches at once, you can use XML or a programming language and use the createCaches() method On Tue, 17 Oct 2023 at 10:43, y wrote: > > Here is cluster log（from one node） and createtable sql（Appendix）. > The Cluster has 10 nodes and do stress testing with 50 thread .It means > there 50 query or createtable operations at the same time. Notice that when > create table，the cluster node log has many ‘Thread - WAITING’. That's not > normal, right? > > > > At 2023-10-17 15:19:33, "Stephen Darlington" > wrote: > > Can you share some more information about your cluster? There is no way > that creating a cache should take so long. > > On Tue, 17 Oct 2023 at 03:51, y wrote: > >> Hello. >> Everyone！ >> >> Creating table statements is executed synchronously and will block >> other DDL statements. There are serious performance issues in concurrent >> environments. It takes one minute to create one table. >>If there is any way to solve this problem, such as changing it to >> serial execution？ >> >> >> Thanks, >> >> Tianyu-Hu >> >> >> >> >
Re: Parallel execution CreateTable
Can you share some more information about your cluster? There is no way that creating a cache should take so long. On Tue, 17 Oct 2023 at 03:51, y wrote: > Hello. > Everyone！ > > Creating table statements is executed synchronously and will block other > DDL statements. There are serious performance issues in concurrent > environments. It takes one minute to create one table. >If there is any way to solve this problem, such as changing it to > serial execution？ > > > Thanks, > > Tianyu-Hu > > > >
Parallel execution CreateTable
Hello. Everyone！ Creating table statements is executed synchronously and will block other DDL statements. There are serious performance issues in concurrent environments. It takes one minute to create one table. If there is any way to solve this problem, such as changing it to serial execution？ Thanks, Tianyu-Hu
