RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
Yes, I remembered your suggestion. That is why I suggest to enable a compile option so that somebody can start formal benchmark measurement :-) Eddie Magenheimer, Dan (HP Labs Fort Collins) wrote: > Rid mangling change has been discussed many times on the list, > most recently: > > http://lists.xensource.com/archives/html/xen-ia64-devel/2005-11/msg00282.html > >> -Original Message- >> From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] On Behalf Of Xu, >> Anthony Sent: Monday, March 13, 2006 6:37 PM >> To: Dong, Eddie; Tian, Kevin; Akio Takebe; Masaki Kanno; >> xen-ia64-devel@lists.xensource.com >> Subject: RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page >> ref counter >> >>> From: Dong, Eddie >>> Sent: 2006年3月13日 22:12 >>> A minor suggestion for next in my mind is that we may add a simple >>> COMPILE option in Makefile or some .h file to be able to choice 1/3 >>> byte swap or 1/2 byte swap. People has some thoughts that 1/2 byte >>> swap may have better hash locality. >>> >>> Eddie. >> I second Eddie, >> >> I have some observations about this. >> Usually guest applications use almost the address space, the only >> different is rid. What I observed was if the lowest 17 bits of rid >> are same, the hash address is same. If we swap 1/3 byte, >> applications use the same address space but different rid may have >> the hash address in a majority of situations, which may make some >> collision chains very long. >> >> These are just some observations, I don't mean 1/2 byte swap is >> better than 1/3 byte swap.I think we need to add COMPILE option to >> get benchmark data first, and then make the decision. >> >> It's obviously not a big task but deserve to do. One thing we need >> to pay extra attention is the rid byte swap is done in assembly code >> in some fast_hyperpriops. >> >> Thanks, >> Anthony >> >> ___ >> Xen-ia64-devel mailing list >> Xen-ia64-devel@lists.xensource.com >> http://lists.xensource.com/xen-ia64-devel ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
Rid mangling change has been discussed many times on the list, most recently: http://lists.xensource.com/archives/html/xen-ia64-devel/2005-11/msg00282.html > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf > Of Xu, Anthony > Sent: Monday, March 13, 2006 6:37 PM > To: Dong, Eddie; Tian, Kevin; Akio Takebe; Masaki Kanno; > xen-ia64-devel@lists.xensource.com > Subject: RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & > page ref counter > > >From: Dong, Eddie > >Sent: 2006年3月13日 22:12 > >A minor suggestion for next in my mind is that we may add a > simple COMPILE option > >in Makefile or some .h file to be able to choice 1/3 byte > swap or 1/2 byte swap. > >People has some thoughts that 1/2 byte swap may have better > hash locality. > > > >Eddie. > I second Eddie, > > I have some observations about this. > Usually guest applications use almost the address space, the > only different is > rid. What I observed was if the lowest 17 bits of rid are > same, the hash address > is same. If we swap 1/3 byte, applications use the same > address space but different > rid may have the hash address in a majority of situations, > which may make some > collision chains very long. > > These are just some observations, I don't mean 1/2 byte swap > is better than 1/3 > byte swap.I think we need to add COMPILE option to get > benchmark data first, and > then make the decision. > > It's obviously not a big task but deserve to do. One thing we > need to pay extra > attention is the rid byte swap is done in assembly code in > some fast_hyperpriops. > > Thanks, > Anthony > > ___ > Xen-ia64-devel mailing list > Xen-ia64-devel@lists.xensource.com > http://lists.xensource.com/xen-ia64-devel > ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
>From: Dong, Eddie >Sent: 2006年3月13日 22:12 >A minor suggestion for next in my mind is that we may add a simple COMPILE >option >in Makefile or some .h file to be able to choice 1/3 byte swap or 1/2 byte >swap. >People has some thoughts that 1/2 byte swap may have better hash locality. > >Eddie. I second Eddie, I have some observations about this. Usually guest applications use almost the address space, the only different is rid. What I observed was if the lowest 17 bits of rid are same, the hash address is same. If we swap 1/3 byte, applications use the same address space but different rid may have the hash address in a majority of situations, which may make some collision chains very long. These are just some observations, I don't mean 1/2 byte swap is better than 1/3 byte swap.I think we need to add COMPILE option to get benchmark data first, and then make the decision. It's obviously not a big task but deserve to do. One thing we need to pay extra attention is the rid byte swap is done in assembly code in some fast_hyperpriops. Thanks, Anthony ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
Really a good job! A minor suggestion for next in my mind is that we may add a simple COMPILE option in Makefile or some .h file to be able to choice 1/3 byte swap or 1/2 byte swap. People has some thoughts that 1/2 byte swap may have better hash locality. Eddie. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
>From: Akio Takebe [mailto:[EMAIL PROTECTED] >Sent: 2006年3月13日 17:45 >>For refcount: >>- You may want to change PGT_va_shift to 32 like x86-64 since >"unsigned >>long type_info" is 64bit width on IA64. Or you either can define it as >"u32 >>type_info" to save space since higher half is not used by your patch. >> >I thought I wanted to be the same struct page as x86. >So I think changing PGT_va_shift to 32 is better >since "u32 type_info" cannot be saved space. >(because inuse and free are union.) Correct. > >>For domain destroy: >>- Also need to free metaphysical rid, which is null by far. Current >>metaphysical rid allocation policy is >monotonic-incremental-allocation-no- >>free >> style. Though it's unlikely to see exhaust of this area (one block >>reserved by >> far), it's better to change it to a cleaner, more efficient policy. You >>may at >>least put a call there if delayed with lower priority. >> >As you said, VHPT and TLB flush may be not necessary, >but we use these flush for safe destroy. >So Kan made a feature of allocating rid, then we may be able to remove >these flush. > Great. Thanks, Kevin ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
Hi, Kevin and Anthony Thank you for your comments. >For refcount: >- You may want to change PGT_va_shift to 32 like x86-64 since "unsigned >long type_info" is 64bit width on IA64. Or you either can define it as "u32 >type_info" to save space since higher half is not used by your patch. > I thought I wanted to be the same struct page as x86. So I think changing PGT_va_shift to 32 is better since "u32 type_info" cannot be saved space. (because inuse and free are union.) >For domain destroy: >- Also need to free metaphysical rid, which is null by far. Current >metaphysical rid allocation policy is monotonic-incremental-allocation-no- >free > style. Though it's unlikely to see exhaust of this area (one block >reserved by > far), it's better to change it to a cleaner, more efficient policy. You >may at >least put a call there if delayed with lower priority. > As you said, VHPT and TLB flush may be not necessary, but we use these flush for safe destroy. So Kan made a feature of allocating rid, then we may be able to remove these flush. Best Regards, Akio Takebe ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
>From: Masaki Kanno >Sent: 2006年3月10日 9:38 >Hi, > >We resend these patches. We made these patches to the latest >changeset:9161, >and reflected comments. >We tested the creation and the destruction of domU repeatedly 100 times. >As a result, there was not the memory which was not freed in xen/ia64. >Please apply these patches to the xen-ia64-unstable tree. > >Signed-off-by: Akio Takebe <[EMAIL PROTECTED]> >Signed-off-by: Masaki Kanno <[EMAIL PROTECTED]> > >Best regards, > Kan, and Fujitsu team Hi, Kan and Akio, Good work to see it in tree now. Now several minor comments :-) For refcount: - You may want to change PGT_va_shift to 32 like x86-64 since "unsigned long type_info" is 64bit width on IA64. Or you either can define it as "u32 type_info" to save space since higher half is not used by your patch. For domain destroy: - Also need to free metaphysical rid, which is null by far. Current metaphysical rid allocation policy is monotonic-incremental-allocation-no-free style. Though it's unlikely to see exhaust of this area (one block reserved by far), it's better to change it to a cleaner, more efficient policy. You may at least put a call there if delayed with lower priority. - Maybe you can use xenpage_list to track p2m pages instead of adding a new pg_list and BUG_ON for the former? More to consider for future of inter-domain reference: I haven't looked into Isaku's p2m implementation yet. However when dom0 is converted without p==m, foreign page map will also trap into Xen for inc/dec refcount of foreign pages. You may co-work with Isaku to see whether similar issue can be addressed. Thanks, Kevin ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
>From: Masaki Kanno [mailto:[EMAIL PROTECTED] >Sent: 2006年3月11日 13:37 >As for this patch, page faults occurs in living domains to flush all VHPT >and all TBL. That is I think that performance of living domains becomes bad >temporarily. However, I thought that a serious problem did not occur so that >entries was inserted in VHPT and TLB again. >Because xen/ia64 was not able to destroy a domain for a long time, I made >this patch to be able to destroy a domain as possible simply. >I think that I have to remake an allocation logic of RID to take the benefit >of RID partition. I think that this remaking is assignment given me. > Kanno, It's great for you to do this! More thinking about this, Currently, the rid range allocated for a domain has fixed relationship to this domain's ID. I am not sure the domain ID allocation mechanism. For example, when you destroy dom3 and create a new one, the new domain's ID is 3 or 4? I'm not sure. If the new domain' ID is 3, Maybe we need to breakdown the relationship between domain ID and the allocated rid ragne to reduce the VHPT and machine TC flush. Thanks Anthony ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
Hi Anthony, Thanks for your comments. Xu, Anthony wrote: >This patch is really good, community has been waiting domain destroy patch for >a long time. > >Is it appropriate time to flush vhpt and machine tlb when destroying domain? >I have below concerns, In fact, I have concerns same as you. >1. This may counteract the benefit of rid partition. In my mind these flushes >are done only when rid reuse happens when allocating rid for a new domain. As for this patch, page faults occurs in living domains to flush all VHPT and all TBL. That is I think that performance of living domains becomes bad temporarily. However, I thought that a serious problem did not occur so that entries was inserted in VHPT and TLB again. Because xen/ia64 was not able to destroy a domain for a long time, I made this patch to be able to destroy a domain as possible simply. I think that I have to remake an allocation logic of RID to take the benefit of RID partition. I think that this remaking is assignment given me. Best regards, Kan >2. When migration is enabled, VMM may need to flush vhpt and TC in all >processors. > >Thanks, >-Anthony > >>-Original Message- >>From: [EMAIL PROTECTED] >>[mailto:[EMAIL PROTECTED] On Behalf Of Masaki Kanno >>Sent: 2006定3埖10晩 9:38 >>To: xen-ia64-devel@lists.xensource.com >>Subject: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter >> >>Hi, >> >>We resend these patches. We made these patches to the latest changeset:9161, >>and reflected comments. >>We tested the creation and the destruction of domU repeatedly 100 times. >>As a result, there was not the memory which was not freed in xen/ia64. >>Please apply these patches to the xen-ia64-unstable tree. >> >>Signed-off-by: Akio Takebe <[EMAIL PROTECTED]> >>Signed-off-by: Masaki Kanno <[EMAIL PROTECTED]> >> >>Best regards, >> Kan, and Fujitsu team > > ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
Le Vendredi 10 Mars 2006 16:34, Alex Williamson a écrit : > On Fri, 2006-03-10 at 10:37 +0900, Masaki Kanno wrote: > > Hi, > > > > We resend these patches. We made these patches to the latest > > changeset:9161, and reflected comments. > > We tested the creation and the destruction of domU repeatedly 100 times. > > As a result, there was not the memory which was not freed in xen/ia64. > > Please apply these patches to the xen-ia64-unstable tree. > >Applied. Nice feature added! Tristan. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
On Fri, 2006-03-10 at 10:37 +0900, Masaki Kanno wrote: > Hi, > > We resend these patches. We made these patches to the latest changeset:9161, > and reflected comments. > We tested the creation and the destruction of domU repeatedly 100 times. > As a result, there was not the memory which was not freed in xen/ia64. > Please apply these patches to the xen-ia64-unstable tree. Applied. -- Alex Williamson HP Linux & Open Source Lab ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
RE: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter
This patch is really good, community has been waiting domain destroy patch for a long time. Is it appropriate time to flush vhpt and machine tlb when destroying domain? I have below concerns, 1. This may counteract the benefit of rid partition. In my mind these flushes are done only when rid reuse happens when allocating rid for a new domain. 2. When migration is enabled, VMM may need to flush vhpt and TC in all processors. Thanks, -Anthony >-Original Message- >From: [EMAIL PROTECTED] >[mailto:[EMAIL PROTECTED] On Behalf Of Masaki Kanno >Sent: 2006年3月10日 9:38 >To: xen-ia64-devel@lists.xensource.com >Subject: [Xen-ia64-devel] [PATCH] [RESEND] domU destroy & page ref counter > >Hi, > >We resend these patches. We made these patches to the latest changeset:9161, >and reflected comments. >We tested the creation and the destruction of domU repeatedly 100 times. >As a result, there was not the memory which was not freed in xen/ia64. >Please apply these patches to the xen-ia64-unstable tree. > >Signed-off-by: Akio Takebe <[EMAIL PROTECTED]> >Signed-off-by: Masaki Kanno <[EMAIL PROTECTED]> > >Best regards, > Kan, and Fujitsu team ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
