Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi folks, On Tue, May 8, 2018 at 1:21 PM, Guillaume Smet wrote: > As mentioned, I'll get back to you soon. > I know you were all impatiently waiting for me to start working on this so here we are. I created 2 PRs: 1/ https://github.com/hibernate/hibernate-orm/pull/2277 2/ https://github.com/hibernate/hibernate-orm/pull/2278 1/ was my initial idea before y'all told me what I should do ;). I think it's pretty safe. 2/ contains 2 other changes, one to generalize 1/ and the other to lazy load the loaders per lock mode that might be more risky I added a lot of explanations/comments in the PRs (especially the second one) so let's continue the discussion in the PRs (as long as it's related to these PRs of course). Feel free to shout other ideas here, they are obviously very welcome. Thanks. -- Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
>> Deferring initialization of these batch loaders is one option, but that just really delays the memory consumption. Personally, if it were my app I'd rather that sizing be done up front rather than over the course of the next few hours or weeks as the system continues to run. Not sure how that is advocating that we delay the generation ;) Anyway, that email thread is what led me to add the different batch modes. That helps with the memory consumption (I think - I was waiting on someone to test it). Have you tried one of the alternative batch modes? I think DYNAMIC is the only one that will improve memory usage, however it incurs the overhead of calculating the SQL each and every time. I have always had the idea to come back and delay the creation of these. IMO we could calculate the most likely used Loaders up-front (LockMode.NONE, LockMode.READ) and delay the more less-used ones. I think this is part of the solution. I think the bigger solution is along the lines of what Guillaume is planning on looking at. In fact this is one of the design aspects of 6.0. Loader is designed to bind the SQL and the "load plan" into one representation. The problem here is that it is designed to do this by doing both tasks internally - it templates building the SQL via its JoinWalkers, which also builds the "load plan". So both happen at the same time as part of the same process. The real solution is to break this up so that generation of the SQL and determination of the "load plan" can be separate operations - namely so that we can share the "load plan" graph with all of the nested loaders for a given batch loader (sharing the load plan across all batch loaders for a given entity is not feasible - some different fetch grapghs. Anyway, like I said - this is one of the design goals with 6.0 and is actually already in place there. I think ultimately a combination of delayed (and then cached) generation of these and separately creatable and passable "load plans" (especially in combination with batch strategies) is the ultimate solution. I'm just not sure how feasible the "separately creatable and passable load plans" portion is in 5.x. Hopefully Guillaume has luck solving that one. On Tue, May 8, 2018 at 7:27 AM Andrej Golovnin wrote: > Hi Sanne, > > > sounds great. Do you plan to contribute your updated patch version? > > I don't think it is needed. I'm sure Guillaume will provide a good > solution. And if not, then I will still have the best Hibernate > version ever. :-) (OK, I will have in any case the best Hibernate > version ever because I have more patches. But this is another story. > :-) ) > > Best regards, > Andrej Golovnin > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi Sanne, > sounds great. Do you plan to contribute your updated patch version? I don't think it is needed. I'm sure Guillaume will provide a good solution. And if not, then I will still have the best Hibernate version ever. :-) (OK, I will have in any case the best Hibernate version ever because I have more patches. But this is another story. :-) ) Best regards, Andrej Golovnin ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi Andrej, sounds great. Do you plan to contribute your updated patch version? Thanks, Sanne On 8 May 2018 at 12:06, Andrej Golovnin wrote: > Hi Sanne, > >> you're right, I found the most interesting email from you now (reading >> the whole thread) : >> - http://lists.jboss.org/pipermail/hibernate-dev/2012-May/008410.html >> >> I'm not sure why the discussion stopped, I'm sorry. >> Let's try to fix it this time, to make sure of that we need a JIRA. >> Someone created one? Guillaume? >> >> summary: >> - problem introduced by https://hibernate.atlassian.net/browse/HHH-4546 >> - great results applying Andrej's >> https://github.com/golovnin/hibernate-orm/commit/1cc8a23d1d5f4079fb090cca63a44e8660f05c7f >> - the patch could be improved today with Java8 >>-- I'd like to have some degree of upfront validation at boostrap still >>-- hopefully explore an array based structure >>-- do not extend CHM >>-- ideally have no read barriers at all at runtime >> > > This is an old patch. The current version does not extend CHM and even > does not use it. It adds ca. 30 lines of code and modifies two classes > to achieve the most memory savings. > > And Sanne, it is not needed to be sorry. Thanks to GIT I can apply my > patches to the new version of Hibernate without big problems and have > the best Hibernate version ever. :-) > > Best regards, > Andrej Golovnin ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi, On Tue, May 8, 2018 at 11:42 AM, Sanne Grinovero wrote: > I'm not sure why the discussion stopped, I'm sorry. > Let's try to fix it this time, to make sure of that we need a JIRA. > Someone created one? Guillaume? > No, I didn't. I plan to work on it tomorrow (today is a public holiday here... and well Thursday too so I have my Friday booked for it too) and will create the appropriate JIRAs once I have a good grasp of what needs to be done. Lazy loading the loaders is one thing but I think we should also try to reduce the amount of memory consumed by them. Otherwise you can still have the same issue under load. And it might be even more problematic as you don't see the issue at startup. As mentioned, I'll get back to you soon. -- Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi Sanne, > you're right, I found the most interesting email from you now (reading > the whole thread) : > - http://lists.jboss.org/pipermail/hibernate-dev/2012-May/008410.html > > I'm not sure why the discussion stopped, I'm sorry. > Let's try to fix it this time, to make sure of that we need a JIRA. > Someone created one? Guillaume? > > summary: > - problem introduced by https://hibernate.atlassian.net/browse/HHH-4546 > - great results applying Andrej's > https://github.com/golovnin/hibernate-orm/commit/1cc8a23d1d5f4079fb090cca63a44e8660f05c7f > - the patch could be improved today with Java8 >-- I'd like to have some degree of upfront validation at boostrap still >-- hopefully explore an array based structure >-- do not extend CHM >-- ideally have no read barriers at all at runtime > This is an old patch. The current version does not extend CHM and even does not use it. It adds ca. 30 lines of code and modifies two classes to achieve the most memory savings. And Sanne, it is not needed to be sorry. Thanks to GIT I can apply my patches to the new version of Hibernate without big problems and have the best Hibernate version ever. :-) Best regards, Andrej Golovnin ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi Andrej, you're right, I found the most interesting email from you now (reading the whole thread) : - http://lists.jboss.org/pipermail/hibernate-dev/2012-May/008410.html I'm not sure why the discussion stopped, I'm sorry. Let's try to fix it this time, to make sure of that we need a JIRA. Someone created one? Guillaume? summary: - problem introduced by https://hibernate.atlassian.net/browse/HHH-4546 - great results applying Andrej's https://github.com/golovnin/hibernate-orm/commit/1cc8a23d1d5f4079fb090cca63a44e8660f05c7f - the patch could be improved today with Java8 -- I'd like to have some degree of upfront validation at boostrap still -- hopefully explore an array based structure -- do not extend CHM -- ideally have no read barriers at all at runtime Thanks, Sanne On 6 May 2018 at 22:03, Andrej Golovnin wrote: > Hi, > >> So, I analyzed the dumps yesterday evening. The problem is real, meaning >> his SessionFactory is consuming more than 1GB of memory for 600+ tables, >> some with a lot of attributes. So for sure, the model is a big one, … > > No, it is still a small one. I work on a project with 2145 tables. And the > SessionFactory > consumes on production systems ca 300-400MB with Hibernate 5.2. I haven’t > tested the project with Hibernate 5.3. But I doubt it would consume more > memory. > Except the Hibernate team broke something again. > >> but it >> would be nice to be more gentle with this type of configuration. I don't >> think it's something new to 5.3 though as it's not the first time we have >> this type of reports. >> >>> From my observations, the problem mostly comes from: >> LegacyBatchingEntityLoader >> \_ loaders - EntityLoader >> \_ staticLoadQuery - EntityLoadQueryDetails >> \_ rootReturn - EntityReturnImpl >> >> The largest LegacyBatchingEntityLoader I have in the dump takes more than 2 >> MB. >> >> Keep in mind that with a batch size of 50, we have 13 EntityLoaders for >> batch sizes of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 20 and 50, each loader >> taking ~ 200 KB. >> >> We discussed some ideas yesterday with Steve. > > And I have discussed it with the Hibernate team 6 years ago: > > http://lists.jboss.org/pipermail/hibernate-dev/2012-May/008341.html > > And my suggestion to create things lazily were ignored/rejected. > And I have ignored the opinion of the Hibernate team and implemented > my suggestions in my version of Hibernate and saved a lot of memory > in my project. > > Btw. please ask the user whether he has a lot of abstract entity classes. > When yes, then ask him if it would be possible to convert this entity classes > to mapped supper-classes. This may help to reduce memory consumption too. > > Best regards, > Andrej Golovnin ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi Guillaume, >> > It's not only the number of tables but also the number of columns per >> > table. >> > The table I take as an example in my tests has 200+ columns, which >> > definitely doesn't help. >> >> I have also a lot of tables with high number of columns. The biggest >> one has 254 columns. So what? > > > So it's not the average 20 columns table. > > Not saying it's not a case we should be grateful with. > > -- > Guillaume > What you should really do is write a generator that would generate models with a configurable number of entities, abstract entities, mapped supper-classes, embeddables, relationships, fields, and so on. You could integrate it into the build process to verify memory consumption after every change in the Hibernate code. This would allow you to find problems before you ship a new version of Hibernate. It should be a good job for a student. :-) Best regards, Andrej Golovnin ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
On Mon, May 7, 2018 at 2:52 PM, Andrej Golovnin wrote: > > It's not only the number of tables but also the number of columns per > table. > > The table I take as an example in my tests has 200+ columns, which > > definitely doesn't help. > > I have also a lot of tables with high number of columns. The biggest > one has 254 columns. So what? > So it's not the average 20 columns table. Not saying it's not a case we should be grateful with. -- Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi Guillaume, > It's not only the number of tables but also the number of columns per table. > The table I take as an example in my tests has 200+ columns, which > definitely doesn't help. I have also a lot of tables with high number of columns. The biggest one has 254 columns. So what? > > If we can come up with some patches, it would be nice if you could test them > with your workload. When you are ready, sent me links to the pull requests and I'll test them. Best regards, Andrej Golovnin ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi Andrej, On Sun, May 6, 2018 at 11:03 PM, Andrej Golovnin wrote: > > So, I analyzed the dumps yesterday evening. The problem is real, meaning > > his SessionFactory is consuming more than 1GB of memory for 600+ tables, > > some with a lot of attributes. So for sure, the model is a big one, … > > No, it is still a small one. I work on a project with 2145 tables. And the > SessionFactory > consumes on production systems ca 300-400MB with Hibernate 5.2. I haven’t > tested the project with Hibernate 5.3. But I doubt it would consume more > memory. > Except the Hibernate team broke something again > It's not only the number of tables but also the number of columns per table. The table I take as an example in my tests has 200+ columns, which definitely doesn't help. And I have discussed it with the Hibernate team 6 years ago: > > http://lists.jboss.org/pipermail/hibernate-dev/2012-May/008341.html > > And my suggestion to create things lazily were ignored/rejected. > And I have ignored the opinion of the Hibernate team and implemented > my suggestions in my version of Hibernate and saved a lot of memory > in my project. > See, we are improving. I see a lot of conjectures in the thread but we should have analyzed the memory dump more closely to understand exactly what was going on. I think it would have led to a different outcome. > Btw. please ask the user whether he has a lot of abstract entity classes. > When yes, then ask him if it would be possible to convert this entity > classes > to mapped supper-classes. This may help to reduce memory consumption too. > Yeah, first, I would like to see if we can improve the situation by changing the ORM code. I'm more for us improving the situation globally rather than tweaking the model of each user. If we can come up with some patches, it would be nice if you could test them with your workload. -- Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi, > So, I analyzed the dumps yesterday evening. The problem is real, meaning > his SessionFactory is consuming more than 1GB of memory for 600+ tables, > some with a lot of attributes. So for sure, the model is a big one, … No, it is still a small one. I work on a project with 2145 tables. And the SessionFactory consumes on production systems ca 300-400MB with Hibernate 5.2. I haven’t tested the project with Hibernate 5.3. But I doubt it would consume more memory. Except the Hibernate team broke something again. > but it > would be nice to be more gentle with this type of configuration. I don't > think it's something new to 5.3 though as it's not the first time we have > this type of reports. > >> From my observations, the problem mostly comes from: > LegacyBatchingEntityLoader > \_ loaders - EntityLoader >\_ staticLoadQuery - EntityLoadQueryDetails >\_ rootReturn - EntityReturnImpl > > The largest LegacyBatchingEntityLoader I have in the dump takes more than 2 > MB. > > Keep in mind that with a batch size of 50, we have 13 EntityLoaders for > batch sizes of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 20 and 50, each loader > taking ~ 200 KB. > > We discussed some ideas yesterday with Steve. And I have discussed it with the Hibernate team 6 years ago: http://lists.jboss.org/pipermail/hibernate-dev/2012-May/008341.html And my suggestion to create things lazily were ignored/rejected. And I have ignored the opinion of the Hibernate team and implemented my suggestions in my version of Hibernate and saved a lot of memory in my project. Btw. please ask the user whether he has a lot of abstract entity classes. When yes, then ask him if it would be possible to convert this entity classes to mapped supper-classes. This may help to reduce memory consumption too. Best regards, Andrej Golovnin ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Loaders were designed to be an encapsulation of the query (the SQL, how to build and execute the PreparedStatement) and the "load plan" (how the ResultSet is processed). 4.2 changed how this last part is represented, but overall the design is the same it always has been. The problem is that Loader was designed such that it itself builds all of these pieces. This is what you'd have to change - specifically to split the creation of the "load plan" outside of the Loader, or at least have the ability to pass in an externally built one On Fri, May 4, 2018 at 3:50 PM Guillaume Smet wrote: > As already mentioned, I plan to take a look at this next week. > > I also thought we could share most of the info between loaders but Steve > was thinking it might not be possible without massive changes. > > I'll take a look and report back to all of you. I might be way over my head > as I don't know the codebase very well but I already did a very similar > work for HV with great results so it's worth a shot. > > -- > Guillaume > > On Fri, May 4, 2018 at 10:31 PM, Gail Badner wrote: > > > I *think* that most, if not all, of the loaders for a particular entity > > could be reworked to share most of the same information. > > > > IIRC, the only real differences are the batch size and the actual query > > that gets generated. > > > > I'm happy to look into this, but I don't want to duplicate what others > are > > doing. It a bit difficult collaborating on this since our time zones > don't > > overlap very well (and, unfortunately, I'm not a morning person). > > > > I have some other work I need to attend to today. Please let me know if I > > should dig into this next week. > > > > On Fri, May 4, 2018 at 12:30 PM, Guillaume Smet < > guillaume.s...@gmail.com> > > wrote: > > > >> Don't know about the history but with the entity I checked out in the > >> OP's case, each EntityLoader is approximately 200 KB. > >> > >> The entity contains a lot of columns so that's no surprise. > >> > >> What doesn't help is that I don't think the 13 EntityLoaders/entity > share > >> a lot of things so they are just adding up. > >> > >> On Fri, May 4, 2018 at 9:15 PM, Sanne Grinovero > >> wrote: > >> > >>> On 4 May 2018 at 19:45, Gail Badner wrote: > >>> > I don't think the batching strategies actually changed when we moved > >>> to load > >>> > plans. > >>> > > >>> > Is the problem that each loader is taking considerably more memory? > >>> > >>> Yes that's right. > >>> > >>> > > >>> > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet < > >>> guillaume.s...@gmail.com> > >>> > wrote: > >>> >> > >>> >> On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero < > sa...@hibernate.org > >>> > > >>> >> wrote: > >>> >> > >>> >> > thanks for investigating. Steve in chat mentioned the option of > >>> >> > creating these EntityLoaders lazily: as while they need to be > >>> reused, > >>> >> > it's possible many of them are never really used. > >>> >> > > >>> >> > I think that's a very interesting observation, is that what you > >>> plan to > >>> >> > explore? > >>> >> > > >>> >> > >>> >> Yes, it's one of the things we should explore. > >>> >> > >>> >> > >>> >> > I see two difficulties: > >>> >> > - you'd need thredsafe code, yet we don't want barriers on this > >>> area. > >>> >> > Will probably need features such as lazy set, yet these would need > >>> >> > even more objects allocated if you're not careful. > >>> >> > - we always prefer fail-at-boot validation, so we'd need to make > >>> sure > >>> >> > even if you don't build them all, we can at least make sure we'll > be > >>> >> > able to build them later. > >>> >> > > >>> >> > >>> >> Well, the idea is to prototype something on the memory usage front, > >>> then > >>> >> we > >>> >> can discuss further what we want to do with it and if it's worth > >>> refining > >>> >> it and spending more time on it. > >>> >> > >>> >> If we end up choosing this path, be sure we'll be bugging you for > >>> advice > >>> >> on > >>> >> concurrency and performance :). > >>> >> > >>> >> > >>> >> > Additionally there's the idea to see if any such Loader instance > >>> could > >>> >> > be reused - probably using a cache which we clear at end of > >>> bootstrap > >>> >> > - yet since Steve mentioned this would be a significant change > let's > >>> >> > not try to push it in 5.3. > >>> >> > > >>> >> > Finally, a simple one: I have the impression that loader's Map > could > >>> >> > be made into a plain array, or a custom structure which would then > >>> be > >>> >> > ideal to support lazy creation of loaders. With the exception of > two > >>> >> > special strings - which will needt to be treated specially - all > >>> other > >>> >> > keys are enums. > >>> >> > > >>> >> > >>> >> Yeah, I'll play with a few ideas and see what I can do. I'm not yet > >>> sure > >>> >> if > >>> >> not knowing the code base will be a thing in my favor or not. At > >>> least I > >>> >> don't have preconceived ideas :]. > >>> >> > >>> >> > >>> >> > Whateve
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
OK, that's fine. I did a lot of the work converting to load plans, so feel free to ask if you have questions. On Fri, May 4, 2018 at 1:37 PM, Guillaume Smet wrote: > As already mentioned, I plan to take a look at this next week. > > I also thought we could share most of the info between loaders but Steve > was thinking it might not be possible without massive changes. > > I'll take a look and report back to all of you. I might be way over my > head as I don't know the codebase very well but I already did a very > similar work for HV with great results so it's worth a shot. > > -- > Guillaume > > On Fri, May 4, 2018 at 10:31 PM, Gail Badner wrote: > >> I *think* that most, if not all, of the loaders for a particular entity >> could be reworked to share most of the same information. >> >> IIRC, the only real differences are the batch size and the actual query >> that gets generated. >> >> I'm happy to look into this, but I don't want to duplicate what others >> are doing. It a bit difficult collaborating on this since our time zones >> don't overlap very well (and, unfortunately, I'm not a morning person). >> >> I have some other work I need to attend to today. Please let me know if I >> should dig into this next week. >> >> On Fri, May 4, 2018 at 12:30 PM, Guillaume Smet > > wrote: >> >>> Don't know about the history but with the entity I checked out in the >>> OP's case, each EntityLoader is approximately 200 KB. >>> >>> The entity contains a lot of columns so that's no surprise. >>> >>> What doesn't help is that I don't think the 13 EntityLoaders/entity >>> share a lot of things so they are just adding up. >>> >>> On Fri, May 4, 2018 at 9:15 PM, Sanne Grinovero >>> wrote: >>> On 4 May 2018 at 19:45, Gail Badner wrote: > I don't think the batching strategies actually changed when we moved to load > plans. > > Is the problem that each loader is taking considerably more memory? Yes that's right. > > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet < guillaume.s...@gmail.com> > wrote: >> >> On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero < sa...@hibernate.org> >> wrote: >> >> > thanks for investigating. Steve in chat mentioned the option of >> > creating these EntityLoaders lazily: as while they need to be reused, >> > it's possible many of them are never really used. >> > >> > I think that's a very interesting observation, is that what you plan to >> > explore? >> > >> >> Yes, it's one of the things we should explore. >> >> >> > I see two difficulties: >> > - you'd need thredsafe code, yet we don't want barriers on this area. >> > Will probably need features such as lazy set, yet these would need >> > even more objects allocated if you're not careful. >> > - we always prefer fail-at-boot validation, so we'd need to make sure >> > even if you don't build them all, we can at least make sure we'll be >> > able to build them later. >> > >> >> Well, the idea is to prototype something on the memory usage front, then >> we >> can discuss further what we want to do with it and if it's worth refining >> it and spending more time on it. >> >> If we end up choosing this path, be sure we'll be bugging you for advice >> on >> concurrency and performance :). >> >> >> > Additionally there's the idea to see if any such Loader instance could >> > be reused - probably using a cache which we clear at end of bootstrap >> > - yet since Steve mentioned this would be a significant change let's >> > not try to push it in 5.3. >> > >> > Finally, a simple one: I have the impression that loader's Map could >> > be made into a plain array, or a custom structure which would then be >> > ideal to support lazy creation of loaders. With the exception of two >> > special strings - which will needt to be treated specially - all other >> > keys are enums. >> > >> >> Yeah, I'll play with a few ideas and see what I can do. I'm not yet sure >> if >> not knowing the code base will be a thing in my favor or not. At least I >> don't have preconceived ideas :]. >> >> >> > Whatever you choose to try, let's keep significant changes for 6 only :) >> > >> >> Steve was saying that this whole thing wouldn't be an issue in 6. >> >> -- >> Guillaume >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev > > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
As already mentioned, I plan to take a look at this next week. I also thought we could share most of the info between loaders but Steve was thinking it might not be possible without massive changes. I'll take a look and report back to all of you. I might be way over my head as I don't know the codebase very well but I already did a very similar work for HV with great results so it's worth a shot. -- Guillaume On Fri, May 4, 2018 at 10:31 PM, Gail Badner wrote: > I *think* that most, if not all, of the loaders for a particular entity > could be reworked to share most of the same information. > > IIRC, the only real differences are the batch size and the actual query > that gets generated. > > I'm happy to look into this, but I don't want to duplicate what others are > doing. It a bit difficult collaborating on this since our time zones don't > overlap very well (and, unfortunately, I'm not a morning person). > > I have some other work I need to attend to today. Please let me know if I > should dig into this next week. > > On Fri, May 4, 2018 at 12:30 PM, Guillaume Smet > wrote: > >> Don't know about the history but with the entity I checked out in the >> OP's case, each EntityLoader is approximately 200 KB. >> >> The entity contains a lot of columns so that's no surprise. >> >> What doesn't help is that I don't think the 13 EntityLoaders/entity share >> a lot of things so they are just adding up. >> >> On Fri, May 4, 2018 at 9:15 PM, Sanne Grinovero >> wrote: >> >>> On 4 May 2018 at 19:45, Gail Badner wrote: >>> > I don't think the batching strategies actually changed when we moved >>> to load >>> > plans. >>> > >>> > Is the problem that each loader is taking considerably more memory? >>> >>> Yes that's right. >>> >>> > >>> > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet < >>> guillaume.s...@gmail.com> >>> > wrote: >>> >> >>> >> On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero >> > >>> >> wrote: >>> >> >>> >> > thanks for investigating. Steve in chat mentioned the option of >>> >> > creating these EntityLoaders lazily: as while they need to be >>> reused, >>> >> > it's possible many of them are never really used. >>> >> > >>> >> > I think that's a very interesting observation, is that what you >>> plan to >>> >> > explore? >>> >> > >>> >> >>> >> Yes, it's one of the things we should explore. >>> >> >>> >> >>> >> > I see two difficulties: >>> >> > - you'd need thredsafe code, yet we don't want barriers on this >>> area. >>> >> > Will probably need features such as lazy set, yet these would need >>> >> > even more objects allocated if you're not careful. >>> >> > - we always prefer fail-at-boot validation, so we'd need to make >>> sure >>> >> > even if you don't build them all, we can at least make sure we'll be >>> >> > able to build them later. >>> >> > >>> >> >>> >> Well, the idea is to prototype something on the memory usage front, >>> then >>> >> we >>> >> can discuss further what we want to do with it and if it's worth >>> refining >>> >> it and spending more time on it. >>> >> >>> >> If we end up choosing this path, be sure we'll be bugging you for >>> advice >>> >> on >>> >> concurrency and performance :). >>> >> >>> >> >>> >> > Additionally there's the idea to see if any such Loader instance >>> could >>> >> > be reused - probably using a cache which we clear at end of >>> bootstrap >>> >> > - yet since Steve mentioned this would be a significant change let's >>> >> > not try to push it in 5.3. >>> >> > >>> >> > Finally, a simple one: I have the impression that loader's Map could >>> >> > be made into a plain array, or a custom structure which would then >>> be >>> >> > ideal to support lazy creation of loaders. With the exception of two >>> >> > special strings - which will needt to be treated specially - all >>> other >>> >> > keys are enums. >>> >> > >>> >> >>> >> Yeah, I'll play with a few ideas and see what I can do. I'm not yet >>> sure >>> >> if >>> >> not knowing the code base will be a thing in my favor or not. At >>> least I >>> >> don't have preconceived ideas :]. >>> >> >>> >> >>> >> > Whatever you choose to try, let's keep significant changes for 6 >>> only :) >>> >> > >>> >> >>> >> Steve was saying that this whole thing wouldn't be an issue in 6. >>> >> >>> >> -- >>> >> Guillaume >>> >> ___ >>> >> hibernate-dev mailing list >>> >> hibernate-dev@lists.jboss.org >>> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> > >>> > >>> ___ >>> hibernate-dev mailing list >>> hibernate-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> >> >> > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
I *think* that most, if not all, of the loaders for a particular entity could be reworked to share most of the same information. IIRC, the only real differences are the batch size and the actual query that gets generated. I'm happy to look into this, but I don't want to duplicate what others are doing. It a bit difficult collaborating on this since our time zones don't overlap very well (and, unfortunately, I'm not a morning person). I have some other work I need to attend to today. Please let me know if I should dig into this next week. On Fri, May 4, 2018 at 12:30 PM, Guillaume Smet wrote: > Don't know about the history but with the entity I checked out in the OP's > case, each EntityLoader is approximately 200 KB. > > The entity contains a lot of columns so that's no surprise. > > What doesn't help is that I don't think the 13 EntityLoaders/entity share > a lot of things so they are just adding up. > > On Fri, May 4, 2018 at 9:15 PM, Sanne Grinovero > wrote: > >> On 4 May 2018 at 19:45, Gail Badner wrote: >> > I don't think the batching strategies actually changed when we moved to >> load >> > plans. >> > >> > Is the problem that each loader is taking considerably more memory? >> >> Yes that's right. >> >> > >> > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet < >> guillaume.s...@gmail.com> >> > wrote: >> >> >> >> On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero >> >> wrote: >> >> >> >> > thanks for investigating. Steve in chat mentioned the option of >> >> > creating these EntityLoaders lazily: as while they need to be reused, >> >> > it's possible many of them are never really used. >> >> > >> >> > I think that's a very interesting observation, is that what you plan >> to >> >> > explore? >> >> > >> >> >> >> Yes, it's one of the things we should explore. >> >> >> >> >> >> > I see two difficulties: >> >> > - you'd need thredsafe code, yet we don't want barriers on this >> area. >> >> > Will probably need features such as lazy set, yet these would need >> >> > even more objects allocated if you're not careful. >> >> > - we always prefer fail-at-boot validation, so we'd need to make >> sure >> >> > even if you don't build them all, we can at least make sure we'll be >> >> > able to build them later. >> >> > >> >> >> >> Well, the idea is to prototype something on the memory usage front, >> then >> >> we >> >> can discuss further what we want to do with it and if it's worth >> refining >> >> it and spending more time on it. >> >> >> >> If we end up choosing this path, be sure we'll be bugging you for >> advice >> >> on >> >> concurrency and performance :). >> >> >> >> >> >> > Additionally there's the idea to see if any such Loader instance >> could >> >> > be reused - probably using a cache which we clear at end of bootstrap >> >> > - yet since Steve mentioned this would be a significant change let's >> >> > not try to push it in 5.3. >> >> > >> >> > Finally, a simple one: I have the impression that loader's Map could >> >> > be made into a plain array, or a custom structure which would then be >> >> > ideal to support lazy creation of loaders. With the exception of two >> >> > special strings - which will needt to be treated specially - all >> other >> >> > keys are enums. >> >> > >> >> >> >> Yeah, I'll play with a few ideas and see what I can do. I'm not yet >> sure >> >> if >> >> not knowing the code base will be a thing in my favor or not. At least >> I >> >> don't have preconceived ideas :]. >> >> >> >> >> >> > Whatever you choose to try, let's keep significant changes for 6 >> only :) >> >> > >> >> >> >> Steve was saying that this whole thing wouldn't be an issue in 6. >> >> >> >> -- >> >> Guillaume >> >> ___ >> >> hibernate-dev mailing list >> >> hibernate-dev@lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > >> > >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Don't know about the history but with the entity I checked out in the OP's case, each EntityLoader is approximately 200 KB. The entity contains a lot of columns so that's no surprise. What doesn't help is that I don't think the 13 EntityLoaders/entity share a lot of things so they are just adding up. On Fri, May 4, 2018 at 9:15 PM, Sanne Grinovero wrote: > On 4 May 2018 at 19:45, Gail Badner wrote: > > I don't think the batching strategies actually changed when we moved to > load > > plans. > > > > Is the problem that each loader is taking considerably more memory? > > Yes that's right. > > > > > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet > > > wrote: > >> > >> On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero > >> wrote: > >> > >> > thanks for investigating. Steve in chat mentioned the option of > >> > creating these EntityLoaders lazily: as while they need to be reused, > >> > it's possible many of them are never really used. > >> > > >> > I think that's a very interesting observation, is that what you plan > to > >> > explore? > >> > > >> > >> Yes, it's one of the things we should explore. > >> > >> > >> > I see two difficulties: > >> > - you'd need thredsafe code, yet we don't want barriers on this area. > >> > Will probably need features such as lazy set, yet these would need > >> > even more objects allocated if you're not careful. > >> > - we always prefer fail-at-boot validation, so we'd need to make sure > >> > even if you don't build them all, we can at least make sure we'll be > >> > able to build them later. > >> > > >> > >> Well, the idea is to prototype something on the memory usage front, then > >> we > >> can discuss further what we want to do with it and if it's worth > refining > >> it and spending more time on it. > >> > >> If we end up choosing this path, be sure we'll be bugging you for advice > >> on > >> concurrency and performance :). > >> > >> > >> > Additionally there's the idea to see if any such Loader instance could > >> > be reused - probably using a cache which we clear at end of bootstrap > >> > - yet since Steve mentioned this would be a significant change let's > >> > not try to push it in 5.3. > >> > > >> > Finally, a simple one: I have the impression that loader's Map could > >> > be made into a plain array, or a custom structure which would then be > >> > ideal to support lazy creation of loaders. With the exception of two > >> > special strings - which will needt to be treated specially - all other > >> > keys are enums. > >> > > >> > >> Yeah, I'll play with a few ideas and see what I can do. I'm not yet sure > >> if > >> not knowing the code base will be a thing in my favor or not. At least I > >> don't have preconceived ideas :]. > >> > >> > >> > Whatever you choose to try, let's keep significant changes for 6 only > :) > >> > > >> > >> Steve was saying that this whole thing wouldn't be an issue in 6. > >> > >> -- > >> Guillaume > >> ___ > >> hibernate-dev mailing list > >> hibernate-dev@lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
On 4 May 2018 at 19:45, Gail Badner wrote: > I don't think the batching strategies actually changed when we moved to load > plans. > > Is the problem that each loader is taking considerably more memory? Yes that's right. > > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet > wrote: >> >> On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero >> wrote: >> >> > thanks for investigating. Steve in chat mentioned the option of >> > creating these EntityLoaders lazily: as while they need to be reused, >> > it's possible many of them are never really used. >> > >> > I think that's a very interesting observation, is that what you plan to >> > explore? >> > >> >> Yes, it's one of the things we should explore. >> >> >> > I see two difficulties: >> > - you'd need thredsafe code, yet we don't want barriers on this area. >> > Will probably need features such as lazy set, yet these would need >> > even more objects allocated if you're not careful. >> > - we always prefer fail-at-boot validation, so we'd need to make sure >> > even if you don't build them all, we can at least make sure we'll be >> > able to build them later. >> > >> >> Well, the idea is to prototype something on the memory usage front, then >> we >> can discuss further what we want to do with it and if it's worth refining >> it and spending more time on it. >> >> If we end up choosing this path, be sure we'll be bugging you for advice >> on >> concurrency and performance :). >> >> >> > Additionally there's the idea to see if any such Loader instance could >> > be reused - probably using a cache which we clear at end of bootstrap >> > - yet since Steve mentioned this would be a significant change let's >> > not try to push it in 5.3. >> > >> > Finally, a simple one: I have the impression that loader's Map could >> > be made into a plain array, or a custom structure which would then be >> > ideal to support lazy creation of loaders. With the exception of two >> > special strings - which will needt to be treated specially - all other >> > keys are enums. >> > >> >> Yeah, I'll play with a few ideas and see what I can do. I'm not yet sure >> if >> not knowing the code base will be a thing in my favor or not. At least I >> don't have preconceived ideas :]. >> >> >> > Whatever you choose to try, let's keep significant changes for 6 only :) >> > >> >> Steve was saying that this whole thing wouldn't be an issue in 6. >> >> -- >> Guillaume >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev > > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
I don't think the batching strategies actually changed when we moved to load plans. Is the problem that each loader is taking considerably more memory? On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet wrote: > On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero > wrote: > > > thanks for investigating. Steve in chat mentioned the option of > > creating these EntityLoaders lazily: as while they need to be reused, > > it's possible many of them are never really used. > > > > I think that's a very interesting observation, is that what you plan to > > explore? > > > > Yes, it's one of the things we should explore. > > > > I see two difficulties: > > - you'd need thredsafe code, yet we don't want barriers on this area. > > Will probably need features such as lazy set, yet these would need > > even more objects allocated if you're not careful. > > - we always prefer fail-at-boot validation, so we'd need to make sure > > even if you don't build them all, we can at least make sure we'll be > > able to build them later. > > > > Well, the idea is to prototype something on the memory usage front, then we > can discuss further what we want to do with it and if it's worth refining > it and spending more time on it. > > If we end up choosing this path, be sure we'll be bugging you for advice on > concurrency and performance :). > > > > Additionally there's the idea to see if any such Loader instance could > > be reused - probably using a cache which we clear at end of bootstrap > > - yet since Steve mentioned this would be a significant change let's > > not try to push it in 5.3. > > > > Finally, a simple one: I have the impression that loader's Map could > > be made into a plain array, or a custom structure which would then be > > ideal to support lazy creation of loaders. With the exception of two > > special strings - which will needt to be treated specially - all other > > keys are enums. > > > > Yeah, I'll play with a few ideas and see what I can do. I'm not yet sure if > not knowing the code base will be a thing in my favor or not. At least I > don't have preconceived ideas :]. > > > > Whatever you choose to try, let's keep significant changes for 6 only :) > > > > Steve was saying that this whole thing wouldn't be an issue in 6. > > -- > Guillaume > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero wrote: > thanks for investigating. Steve in chat mentioned the option of > creating these EntityLoaders lazily: as while they need to be reused, > it's possible many of them are never really used. > > I think that's a very interesting observation, is that what you plan to > explore? > Yes, it's one of the things we should explore. > I see two difficulties: > - you'd need thredsafe code, yet we don't want barriers on this area. > Will probably need features such as lazy set, yet these would need > even more objects allocated if you're not careful. > - we always prefer fail-at-boot validation, so we'd need to make sure > even if you don't build them all, we can at least make sure we'll be > able to build them later. > Well, the idea is to prototype something on the memory usage front, then we can discuss further what we want to do with it and if it's worth refining it and spending more time on it. If we end up choosing this path, be sure we'll be bugging you for advice on concurrency and performance :). > Additionally there's the idea to see if any such Loader instance could > be reused - probably using a cache which we clear at end of bootstrap > - yet since Steve mentioned this would be a significant change let's > not try to push it in 5.3. > > Finally, a simple one: I have the impression that loader's Map could > be made into a plain array, or a custom structure which would then be > ideal to support lazy creation of loaders. With the exception of two > special strings - which will needt to be treated specially - all other > keys are enums. > Yeah, I'll play with a few ideas and see what I can do. I'm not yet sure if not knowing the code base will be a thing in my favor or not. At least I don't have preconceived ideas :]. > Whatever you choose to try, let's keep significant changes for 6 only :) > Steve was saying that this whole thing wouldn't be an issue in 6. -- Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi Guillaume, thanks for investigating. Steve in chat mentioned the option of creating these EntityLoaders lazily: as while they need to be reused, it's possible many of them are never really used. I think that's a very interesting observation, is that what you plan to explore? I see two difficulties: - you'd need thredsafe code, yet we don't want barriers on this area. Will probably need features such as lazy set, yet these would need even more objects allocated if you're not careful. - we always prefer fail-at-boot validation, so we'd need to make sure even if you don't build them all, we can at least make sure we'll be able to build them later. It's worth remembering that the JVM can actually optimise ORM quite well as there are not many memory barriers in the most complex areas. Additionally there's the idea to see if any such Loader instance could be reused - probably using a cache which we clear at end of bootstrap - yet since Steve mentioned this would be a significant change let's not try to push it in 5.3. Finally, a simple one: I have the impression that loader's Map could be made into a plain array, or a custom structure which would then be ideal to support lazy creation of loaders. With the exception of two special strings - which will needt to be treated specially - all other keys are enums. Whatever you choose to try, let's keep significant changes for 6 only :) Thanks, Sanne On 4 May 2018 at 10:06, Guillaume Smet wrote: > Hi, > > So, I analyzed the dumps yesterday evening. The problem is real, meaning his > SessionFactory is consuming more than 1GB of memory for 600+ tables, some > with a lot of attributes. So for sure, the model is a big one, but it would > be nice to be more gentle with this type of configuration. I don't think > it's something new to 5.3 though as it's not the first time we have this > type of reports. > > From my observations, the problem mostly comes from: > LegacyBatchingEntityLoader > \_ loaders - EntityLoader > \_ staticLoadQuery - EntityLoadQueryDetails > \_ rootReturn - EntityReturnImpl > > The largest LegacyBatchingEntityLoader I have in the dump takes more than 2 > MB. > > Keep in mind that with a batch size of 50, we have 13 EntityLoaders for > batch sizes of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 20 and 50, each loader > taking ~ 200 KB. > > We discussed some ideas yesterday with Steve. > > I'll try to experiment a few ideas next week to see if I can come up with > some mitigation. I'm not that familiar with the ORM source code so I'm full > of crazy ideas to try - most of them will probably prove to be bad (Steve > already shut some of them) but maybe one will survive and be beneficial to > the problem at hand. > > I'll keep you posted if I can come up with something. > > -- > Guillaume > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi, So, I analyzed the dumps yesterday evening. The problem is real, meaning his SessionFactory is consuming more than 1GB of memory for 600+ tables, some with a lot of attributes. So for sure, the model is a big one, but it would be nice to be more gentle with this type of configuration. I don't think it's something new to 5.3 though as it's not the first time we have this type of reports. >From my observations, the problem mostly comes from: LegacyBatchingEntityLoader \_ loaders - EntityLoader \_ staticLoadQuery - EntityLoadQueryDetails \_ rootReturn - EntityReturnImpl The largest LegacyBatchingEntityLoader I have in the dump takes more than 2 MB. Keep in mind that with a batch size of 50, we have 13 EntityLoaders for batch sizes of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 20 and 50, each loader taking ~ 200 KB. We discussed some ideas yesterday with Steve. I'll try to experiment a few ideas next week to see if I can come up with some mitigation. I'm not that familiar with the ORM source code so I'm full of crazy ideas to try - most of them will probably prove to be bad (Steve already shut some of them) but maybe one will survive and be beneficial to the problem at hand. I'll keep you posted if I can come up with something. -- Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
On 3 May 2018 at 19:33, Gail Badner wrote: > OK, let me know if I should jump in... Thanks Gail, I think you can relax for now :) - knowing that 4.3 introduced the load plans explains a good deal of the difference - Guillaume is checking details too - the Red Hat performance team is going to check general allocation with the usual benchmarks (not specifically this issue, but still reassuring enough) > > On Thu, May 3, 2018 at 10:59 AM, Steve Ebersole wrote: > >> Yeah, we have been having a chat about this on HipChat >> >> On Thu, May 3, 2018 at 12:54 PM Gail Badner wrote: >> >>> I just looked at that thread and it's comparing 3.6 to 5.3.CR2... >>> >>> On Thu, May 3, 2018 at 4:17 AM, Vlad Mihalcea >>> wrote: >>> >>> > I asked because someone might have an idea about some change that was >>> doe >>> > in 5.3 that might affect it. >>> > For the moment, I haven't started doing anything about it. >>> > >>> > Vlad >>> > >>> > On Thu, May 3, 2018 at 2:01 PM, Guillaume Smet < >>> guillaume.s...@gmail.com> >>> > wrote: >>> > >>> > > On Thu, May 3, 2018 at 9:35 AM, Vlad Mihalcea < >>> mihalcea.v...@gmail.com> >>> > > wrote: >>> > > >>> > >> I think it's worth investigating the cause and see how we could >>> address >>> > it >>> > >> in the next release. >>> > >> >>> > > >>> > > Sure, something looks fishy there. >>> > > >>> > > Will you do it or do you expect someone else to do it? It's not very >>> > clear >>> > > from your email. >>> > > >>> > > -- >>> > > Guillaume >>> > > >>> > > >>> > ___ >>> > hibernate-dev mailing list >>> > hibernate-dev@lists.jboss.org >>> > https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> > >>> ___ >>> hibernate-dev mailing list >>> hibernate-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> >> > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
OK, let me know if I should jump in... On Thu, May 3, 2018 at 10:59 AM, Steve Ebersole wrote: > Yeah, we have been having a chat about this on HipChat > > On Thu, May 3, 2018 at 12:54 PM Gail Badner wrote: > >> I just looked at that thread and it's comparing 3.6 to 5.3.CR2... >> >> On Thu, May 3, 2018 at 4:17 AM, Vlad Mihalcea >> wrote: >> >> > I asked because someone might have an idea about some change that was >> doe >> > in 5.3 that might affect it. >> > For the moment, I haven't started doing anything about it. >> > >> > Vlad >> > >> > On Thu, May 3, 2018 at 2:01 PM, Guillaume Smet < >> guillaume.s...@gmail.com> >> > wrote: >> > >> > > On Thu, May 3, 2018 at 9:35 AM, Vlad Mihalcea < >> mihalcea.v...@gmail.com> >> > > wrote: >> > > >> > >> I think it's worth investigating the cause and see how we could >> address >> > it >> > >> in the next release. >> > >> >> > > >> > > Sure, something looks fishy there. >> > > >> > > Will you do it or do you expect someone else to do it? It's not very >> > clear >> > > from your email. >> > > >> > > -- >> > > Guillaume >> > > >> > > >> > ___ >> > hibernate-dev mailing list >> > hibernate-dev@lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Yeah, we have been having a chat about this on HipChat On Thu, May 3, 2018 at 12:54 PM Gail Badner wrote: > I just looked at that thread and it's comparing 3.6 to 5.3.CR2... > > On Thu, May 3, 2018 at 4:17 AM, Vlad Mihalcea > wrote: > > > I asked because someone might have an idea about some change that was doe > > in 5.3 that might affect it. > > For the moment, I haven't started doing anything about it. > > > > Vlad > > > > On Thu, May 3, 2018 at 2:01 PM, Guillaume Smet > > > wrote: > > > > > On Thu, May 3, 2018 at 9:35 AM, Vlad Mihalcea > > > > wrote: > > > > > >> I think it's worth investigating the cause and see how we could > address > > it > > >> in the next release. > > >> > > > > > > Sure, something looks fishy there. > > > > > > Will you do it or do you expect someone else to do it? It's not very > > clear > > > from your email. > > > > > > -- > > > Guillaume > > > > > > > > ___ > > hibernate-dev mailing list > > hibernate-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
FYI, those batch loaders were updated to use "load plans" in 4.3. I suspect that might have something to do with it. Please let me know if I should look into this. On Thu, May 3, 2018 at 10:54 AM, Gail Badner wrote: > I just looked at that thread and it's comparing 3.6 to 5.3.CR2... > > On Thu, May 3, 2018 at 4:17 AM, Vlad Mihalcea > wrote: > >> I asked because someone might have an idea about some change that was doe >> in 5.3 that might affect it. >> For the moment, I haven't started doing anything about it. >> >> Vlad >> >> On Thu, May 3, 2018 at 2:01 PM, Guillaume Smet >> wrote: >> >> > On Thu, May 3, 2018 at 9:35 AM, Vlad Mihalcea >> > wrote: >> > >> >> I think it's worth investigating the cause and see how we could >> address it >> >> in the next release. >> >> >> > >> > Sure, something looks fishy there. >> > >> > Will you do it or do you expect someone else to do it? It's not very >> clear >> > from your email. >> > >> > -- >> > Guillaume >> > >> > >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
I just looked at that thread and it's comparing 3.6 to 5.3.CR2... On Thu, May 3, 2018 at 4:17 AM, Vlad Mihalcea wrote: > I asked because someone might have an idea about some change that was doe > in 5.3 that might affect it. > For the moment, I haven't started doing anything about it. > > Vlad > > On Thu, May 3, 2018 at 2:01 PM, Guillaume Smet > wrote: > > > On Thu, May 3, 2018 at 9:35 AM, Vlad Mihalcea > > wrote: > > > >> I think it's worth investigating the cause and see how we could address > it > >> in the next release. > >> > > > > Sure, something looks fishy there. > > > > Will you do it or do you expect someone else to do it? It's not very > clear > > from your email. > > > > -- > > Guillaume > > > > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
I asked because someone might have an idea about some change that was doe in 5.3 that might affect it. For the moment, I haven't started doing anything about it. Vlad On Thu, May 3, 2018 at 2:01 PM, Guillaume Smet wrote: > On Thu, May 3, 2018 at 9:35 AM, Vlad Mihalcea > wrote: > >> I think it's worth investigating the cause and see how we could address it >> in the next release. >> > > Sure, something looks fishy there. > > Will you do it or do you expect someone else to do it? It's not very clear > from your email. > > -- > Guillaume > > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
On Thu, May 3, 2018 at 9:35 AM, Vlad Mihalcea wrote: > I think it's worth investigating the cause and see how we could address it > in the next release. > Sure, something looks fishy there. Will you do it or do you expect someone else to do it? It's not very clear from your email. -- Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] SingleTableEntityPersister memory footprint in 5.3
Hi, On Discourse, a user told us that the memory footprint has increased dramatically between 5.2 and 5.3: https://discourse.hibernate.org/t/batch-fetch-style-recommendations/631/5?u=vlad I think it's worth investigating the cause and see how we could address it in the next release. Vlad ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
