Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Mon, 13 Mar 2017, David Carrillo-Cisneros wrote: > >> I am ok removing the perf-like CPU filtering from the requirements. > > > > So if I'm not missing something then ALL remaining requirements can be > > solved with the RDT integrated monitoring mechanics, right? > > > > Right. Excellent.
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Mon, 13 Mar 2017, David Carrillo-Cisneros wrote: > >> I am ok removing the perf-like CPU filtering from the requirements. > > > > So if I'm not missing something then ALL remaining requirements can be > > solved with the RDT integrated monitoring mechanics, right? > > > > Right. Excellent.
Re: [PATCH 1/1] x86/cqm: Cqm requirements
>> I am ok removing the perf-like CPU filtering from the requirements. > > So if I'm not missing something then ALL remaining requirements can be > solved with the RDT integrated monitoring mechanics, right? > Right.
Re: [PATCH 1/1] x86/cqm: Cqm requirements
>> I am ok removing the perf-like CPU filtering from the requirements. > > So if I'm not missing something then ALL remaining requirements can be > solved with the RDT integrated monitoring mechanics, right? > Right.
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Fri, 10 Mar 2017, David Carrillo-Cisneros wrote: > > Fine. So we need this for ONE particular use case. And if that is not well > > documented including the underlying mechanics to analyze the data then this > > will be a nice source of confusion for Joe User. > > > > I still think that this can be done differently while keeping the overhead > > small. > > > > You look at this from the existing perf mechanics which require high > > overhead context switching machinery. But that's just wrong because that's > > not how the cache and bandwidth monitoring works. > > > > Contrary to the other perf counters, CQM and MBM are based on a context > > selectable set of counters which do not require readout and reconfiguration > > when the switch happens. > > > > Especially with CAT in play, the context switch overhead is there already > > when CAT partitions need to be switched. So switching the RMID at the same > > time is basically free, if we are smart enough to do an equivalent to the > > CLOSID context switch mechanics and ideally combine both into a single MSR > > write. > > > > With that the low overhead periodic sampling can read N counters which are > > related to the monitored set and provide N separate results. For bandwidth > > the aggregation is a simple ADD and for cache residency it's pointless. > > > > Just because perf was designed with the regular performance counters in > > mind (way before that CQM/MBM stuff came around) does not mean that we > > cannot change/extend that if it makes sense. > > > > And looking at the way Cache/Bandwidth allocation and monitoring works, it > > makes a lot of sense. Definitely more than shoving it into the current mode > > of operandi with duct tape just because we can. > > > > You made a point. The use case I described can be better served with > the low overhead monitoring groups that Fenghua is working on. Then > that info can be merged with the per-CPU profile collected for non-RDT > events. > > I am ok removing the perf-like CPU filtering from the requirements. So if I'm not missing something then ALL remaining requirements can be solved with the RDT integrated monitoring mechanics, right? Thanks, tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Fri, 10 Mar 2017, David Carrillo-Cisneros wrote: > > Fine. So we need this for ONE particular use case. And if that is not well > > documented including the underlying mechanics to analyze the data then this > > will be a nice source of confusion for Joe User. > > > > I still think that this can be done differently while keeping the overhead > > small. > > > > You look at this from the existing perf mechanics which require high > > overhead context switching machinery. But that's just wrong because that's > > not how the cache and bandwidth monitoring works. > > > > Contrary to the other perf counters, CQM and MBM are based on a context > > selectable set of counters which do not require readout and reconfiguration > > when the switch happens. > > > > Especially with CAT in play, the context switch overhead is there already > > when CAT partitions need to be switched. So switching the RMID at the same > > time is basically free, if we are smart enough to do an equivalent to the > > CLOSID context switch mechanics and ideally combine both into a single MSR > > write. > > > > With that the low overhead periodic sampling can read N counters which are > > related to the monitored set and provide N separate results. For bandwidth > > the aggregation is a simple ADD and for cache residency it's pointless. > > > > Just because perf was designed with the regular performance counters in > > mind (way before that CQM/MBM stuff came around) does not mean that we > > cannot change/extend that if it makes sense. > > > > And looking at the way Cache/Bandwidth allocation and monitoring works, it > > makes a lot of sense. Definitely more than shoving it into the current mode > > of operandi with duct tape just because we can. > > > > You made a point. The use case I described can be better served with > the low overhead monitoring groups that Fenghua is working on. Then > that info can be merged with the per-CPU profile collected for non-RDT > events. > > I am ok removing the perf-like CPU filtering from the requirements. So if I'm not missing something then ALL remaining requirements can be solved with the RDT integrated monitoring mechanics, right? Thanks, tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
> > Fine. So we need this for ONE particular use case. And if that is not well > documented including the underlying mechanics to analyze the data then this > will be a nice source of confusion for Joe User. > > I still think that this can be done differently while keeping the overhead > small. > > You look at this from the existing perf mechanics which require high > overhead context switching machinery. But that's just wrong because that's > not how the cache and bandwidth monitoring works. > > Contrary to the other perf counters, CQM and MBM are based on a context > selectable set of counters which do not require readout and reconfiguration > when the switch happens. > > Especially with CAT in play, the context switch overhead is there already > when CAT partitions need to be switched. So switching the RMID at the same > time is basically free, if we are smart enough to do an equivalent to the > CLOSID context switch mechanics and ideally combine both into a single MSR > write. > > With that the low overhead periodic sampling can read N counters which are > related to the monitored set and provide N separate results. For bandwidth > the aggregation is a simple ADD and for cache residency it's pointless. > > Just because perf was designed with the regular performance counters in > mind (way before that CQM/MBM stuff came around) does not mean that we > cannot change/extend that if it makes sense. > > And looking at the way Cache/Bandwidth allocation and monitoring works, it > makes a lot of sense. Definitely more than shoving it into the current mode > of operandi with duct tape just because we can. > You made a point. The use case I described can be better served with the low overhead monitoring groups that Fenghua is working on. Then that info can be merged with the per-CPU profile collected for non-RDT events. I am ok removing the perf-like CPU filtering from the requirements. Thanks, David
Re: [PATCH 1/1] x86/cqm: Cqm requirements
> > Fine. So we need this for ONE particular use case. And if that is not well > documented including the underlying mechanics to analyze the data then this > will be a nice source of confusion for Joe User. > > I still think that this can be done differently while keeping the overhead > small. > > You look at this from the existing perf mechanics which require high > overhead context switching machinery. But that's just wrong because that's > not how the cache and bandwidth monitoring works. > > Contrary to the other perf counters, CQM and MBM are based on a context > selectable set of counters which do not require readout and reconfiguration > when the switch happens. > > Especially with CAT in play, the context switch overhead is there already > when CAT partitions need to be switched. So switching the RMID at the same > time is basically free, if we are smart enough to do an equivalent to the > CLOSID context switch mechanics and ideally combine both into a single MSR > write. > > With that the low overhead periodic sampling can read N counters which are > related to the monitored set and provide N separate results. For bandwidth > the aggregation is a simple ADD and for cache residency it's pointless. > > Just because perf was designed with the regular performance counters in > mind (way before that CQM/MBM stuff came around) does not mean that we > cannot change/extend that if it makes sense. > > And looking at the way Cache/Bandwidth allocation and monitoring works, it > makes a lot of sense. Definitely more than shoving it into the current mode > of operandi with duct tape just because we can. > You made a point. The use case I described can be better served with the low overhead monitoring groups that Fenghua is working on. Then that info can be merged with the per-CPU profile collected for non-RDT events. I am ok removing the perf-like CPU filtering from the requirements. Thanks, David
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Thu, 9 Mar 2017, David Carrillo-Cisneros wrote: > On Thu, Mar 9, 2017 at 3:01 AM, Thomas Gleixnerwrote: > > On Wed, 8 Mar 2017, David Carrillo-Cisneros wrote: > >> On Wed, Mar 8, 2017 at 12:30 AM, Thomas Gleixner > >> wrote: > >> > Same applies for per CPU measurements. > >> > >> For CPU measurements. We need perf-like CPU filtering to support tools > >> that perform low overhead monitoring by polling CPU events. These > >> tools approximate per-cgroup/task events by reconciling CPU events > >> with logs of what job run when in what CPU. > > > > Sorry, but for CQM that's just voodoo analysis. > > I'll argue that. Yet, perf-like CPU is also needed for MBM, a less > contentious scenario, I believe. MBM is a different playground (albeit related due to the RMID stuff). > It does not work well for a single run (your example). But for the > example I gave, one can just rely on Random Sampling, Law of Large > Numbers, and Central Limit Theorem. Fine. So we need this for ONE particular use case. And if that is not well documented including the underlying mechanics to analyze the data then this will be a nice source of confusion for Joe User. I still think that this can be done differently while keeping the overhead small. You look at this from the existing perf mechanics which require high overhead context switching machinery. But that's just wrong because that's not how the cache and bandwidth monitoring works. Contrary to the other perf counters, CQM and MBM are based on a context selectable set of counters which do not require readout and reconfiguration when the switch happens. Especially with CAT in play, the context switch overhead is there already when CAT partitions need to be switched. So switching the RMID at the same time is basically free, if we are smart enough to do an equivalent to the CLOSID context switch mechanics and ideally combine both into a single MSR write. With that the low overhead periodic sampling can read N counters which are related to the monitored set and provide N separate results. For bandwidth the aggregation is a simple ADD and for cache residency it's pointless. Just because perf was designed with the regular performance counters in mind (way before that CQM/MBM stuff came around) does not mean that we cannot change/extend that if it makes sense. And looking at the way Cache/Bandwidth allocation and monitoring works, it makes a lot of sense. Definitely more than shoving it into the current mode of operandi with duct tape just because we can. Thanks, tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Thu, 9 Mar 2017, David Carrillo-Cisneros wrote: > On Thu, Mar 9, 2017 at 3:01 AM, Thomas Gleixner wrote: > > On Wed, 8 Mar 2017, David Carrillo-Cisneros wrote: > >> On Wed, Mar 8, 2017 at 12:30 AM, Thomas Gleixner > >> wrote: > >> > Same applies for per CPU measurements. > >> > >> For CPU measurements. We need perf-like CPU filtering to support tools > >> that perform low overhead monitoring by polling CPU events. These > >> tools approximate per-cgroup/task events by reconciling CPU events > >> with logs of what job run when in what CPU. > > > > Sorry, but for CQM that's just voodoo analysis. > > I'll argue that. Yet, perf-like CPU is also needed for MBM, a less > contentious scenario, I believe. MBM is a different playground (albeit related due to the RMID stuff). > It does not work well for a single run (your example). But for the > example I gave, one can just rely on Random Sampling, Law of Large > Numbers, and Central Limit Theorem. Fine. So we need this for ONE particular use case. And if that is not well documented including the underlying mechanics to analyze the data then this will be a nice source of confusion for Joe User. I still think that this can be done differently while keeping the overhead small. You look at this from the existing perf mechanics which require high overhead context switching machinery. But that's just wrong because that's not how the cache and bandwidth monitoring works. Contrary to the other perf counters, CQM and MBM are based on a context selectable set of counters which do not require readout and reconfiguration when the switch happens. Especially with CAT in play, the context switch overhead is there already when CAT partitions need to be switched. So switching the RMID at the same time is basically free, if we are smart enough to do an equivalent to the CLOSID context switch mechanics and ideally combine both into a single MSR write. With that the low overhead periodic sampling can read N counters which are related to the monitored set and provide N separate results. For bandwidth the aggregation is a simple ADD and for cache residency it's pointless. Just because perf was designed with the regular performance counters in mind (way before that CQM/MBM stuff came around) does not mean that we cannot change/extend that if it makes sense. And looking at the way Cache/Bandwidth allocation and monitoring works, it makes a lot of sense. Definitely more than shoving it into the current mode of operandi with duct tape just because we can. Thanks, tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Thu, Mar 9, 2017 at 3:01 AM, Thomas Gleixnerwrote: > On Wed, 8 Mar 2017, David Carrillo-Cisneros wrote: >> On Wed, Mar 8, 2017 at 12:30 AM, Thomas Gleixner wrote: >> > Same applies for per CPU measurements. >> >> For CPU measurements. We need perf-like CPU filtering to support tools >> that perform low overhead monitoring by polling CPU events. These >> tools approximate per-cgroup/task events by reconciling CPU events >> with logs of what job run when in what CPU. > > Sorry, but for CQM that's just voodoo analysis. I'll argue that. Yet, perf-like CPU is also needed for MBM, a less contentious scenario, I believe. > > CPU default is CAT group 0 (20% of cache) > T1 belongs to CAT group 1 (40% of cache) > T2 belongs to CAT group 2 (40% of cache) > > Now you do low overhead samples of the CPU (all groups accounted) with 1 > second period. > > Lets assume that T1 runs 50% and T2 runs 20% the rest of the time is > utilized by random other things and the kernel itself (using CAT group 0). > > What is the accumulated value telling you? In this single example not much, only the sum of occupancies. But assume I have T1...T1 different jobs, and I randomly select a pair of those jobs to run together in a machine, (they become the T1 and T2 in your example). Then I repeat that hundreds of thousands of times. I can collect all data with (tasks run, time run, occupancy) and build a simple regression to estimate the expected occupancy (and some confidence interval). That inaccurate but approximate value is very useful to feed into a job scheduler. Furthermore, it can be correlated with values of other events that are currently sampled this way. > > How do you approximate that back to T1/T2 and the rest? Described above for large numbers and random samples. More sophisticated (voodo?) statistic techniques are employed in practice to account for almost all issues I could think of (selection bias, missing values, interaction between tasks, etc). They seem to work fine. > > How do you do that when the tasks are switching between the samples several > times? It does not work well for a single run (your example). But for the example I gave, one can just rely on Random Sampling, Law of Large Numbers, and Central Limit Theorem. Thanks, David
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Thu, Mar 9, 2017 at 3:01 AM, Thomas Gleixner wrote: > On Wed, 8 Mar 2017, David Carrillo-Cisneros wrote: >> On Wed, Mar 8, 2017 at 12:30 AM, Thomas Gleixner wrote: >> > Same applies for per CPU measurements. >> >> For CPU measurements. We need perf-like CPU filtering to support tools >> that perform low overhead monitoring by polling CPU events. These >> tools approximate per-cgroup/task events by reconciling CPU events >> with logs of what job run when in what CPU. > > Sorry, but for CQM that's just voodoo analysis. I'll argue that. Yet, perf-like CPU is also needed for MBM, a less contentious scenario, I believe. > > CPU default is CAT group 0 (20% of cache) > T1 belongs to CAT group 1 (40% of cache) > T2 belongs to CAT group 2 (40% of cache) > > Now you do low overhead samples of the CPU (all groups accounted) with 1 > second period. > > Lets assume that T1 runs 50% and T2 runs 20% the rest of the time is > utilized by random other things and the kernel itself (using CAT group 0). > > What is the accumulated value telling you? In this single example not much, only the sum of occupancies. But assume I have T1...T1 different jobs, and I randomly select a pair of those jobs to run together in a machine, (they become the T1 and T2 in your example). Then I repeat that hundreds of thousands of times. I can collect all data with (tasks run, time run, occupancy) and build a simple regression to estimate the expected occupancy (and some confidence interval). That inaccurate but approximate value is very useful to feed into a job scheduler. Furthermore, it can be correlated with values of other events that are currently sampled this way. > > How do you approximate that back to T1/T2 and the rest? Described above for large numbers and random samples. More sophisticated (voodo?) statistic techniques are employed in practice to account for almost all issues I could think of (selection bias, missing values, interaction between tasks, etc). They seem to work fine. > > How do you do that when the tasks are switching between the samples several > times? It does not work well for a single run (your example). But for the example I gave, one can just rely on Random Sampling, Law of Large Numbers, and Central Limit Theorem. Thanks, David
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Wed, 8 Mar 2017, David Carrillo-Cisneros wrote: > On Wed, Mar 8, 2017 at 12:30 AM, Thomas Gleixnerwrote: > > Same applies for per CPU measurements. > > For CPU measurements. We need perf-like CPU filtering to support tools > that perform low overhead monitoring by polling CPU events. These > tools approximate per-cgroup/task events by reconciling CPU events > with logs of what job run when in what CPU. Sorry, but for CQM that's just voodoo analysis. Lets look at an example: CPU default is CAT group 0 (20% of cache) T1 belongs to CAT group 1 (40% of cache) T2 belongs to CAT group 2 (40% of cache) Now you do low overhead samples of the CPU (all groups accounted) with 1 second period. Lets assume that T1 runs 50% and T2 runs 20% the rest of the time is utilized by random other things and the kernel itself (using CAT group 0). What is the accumulated value telling you? How do you approximate that back to T1/T2 and the rest? How do you do that when the tasks are switching between the samples several times? I really have idea how that should work and what the value of this would be. Thanks, tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Wed, 8 Mar 2017, David Carrillo-Cisneros wrote: > On Wed, Mar 8, 2017 at 12:30 AM, Thomas Gleixner wrote: > > Same applies for per CPU measurements. > > For CPU measurements. We need perf-like CPU filtering to support tools > that perform low overhead monitoring by polling CPU events. These > tools approximate per-cgroup/task events by reconciling CPU events > with logs of what job run when in what CPU. Sorry, but for CQM that's just voodoo analysis. Lets look at an example: CPU default is CAT group 0 (20% of cache) T1 belongs to CAT group 1 (40% of cache) T2 belongs to CAT group 2 (40% of cache) Now you do low overhead samples of the CPU (all groups accounted) with 1 second period. Lets assume that T1 runs 50% and T2 runs 20% the rest of the time is utilized by random other things and the kernel itself (using CAT group 0). What is the accumulated value telling you? How do you approximate that back to T1/T2 and the rest? How do you do that when the tasks are switching between the samples several times? I really have idea how that should work and what the value of this would be. Thanks, tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Wed, Mar 8, 2017 at 12:30 AM, Thomas Gleixnerwrote: > Stephane, > > On Tue, 7 Mar 2017, Stephane Eranian wrote: >> On Tue, Mar 7, 2017 at 12:04 PM, Luck, Tony wrote: >> >> That's all nice and good, but I still have no coherent explanation why >> >> measuring across allocation domains makes sense. >> > >> > Is this in reaction to this one? >> > >> >>> 5) Put multiple threads into a single measurement group >> > >> > If we fix it to say "threads from the same CAT group" does it fix things? >> > >> Inside a CAT partition, there may be multiple tasks split into different >> cgroups. We need the ability to monitor groups of tasks individually >> within that CAT partition. I think this is what this bullet is about. > > I completely understand that. That's fine and I never debated that one, but > the requirements list is too vague about what you want to measure. > >> >>> 5) Put multiple threads into a single measurement group > > That can be: > > A) threads within a CAT group > > B) threads which belong to different CAT groups > > A) is fine. B) does not make any sense to me It's A). As Tony suggested in a previous email, we can rephrase it to: 5) Put a subset of threads from the same CAT group into a single measurement group. > > Same applies for per CPU measurements. For CPU measurements. We need perf-like CPU filtering to support tools that perform low overhead monitoring by polling CPU events. These tools approximate per-cgroup/task events by reconciling CPU events with logs of what job run when in what CPU. > > Thanks, > > tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Wed, Mar 8, 2017 at 12:30 AM, Thomas Gleixner wrote: > Stephane, > > On Tue, 7 Mar 2017, Stephane Eranian wrote: >> On Tue, Mar 7, 2017 at 12:04 PM, Luck, Tony wrote: >> >> That's all nice and good, but I still have no coherent explanation why >> >> measuring across allocation domains makes sense. >> > >> > Is this in reaction to this one? >> > >> >>> 5) Put multiple threads into a single measurement group >> > >> > If we fix it to say "threads from the same CAT group" does it fix things? >> > >> Inside a CAT partition, there may be multiple tasks split into different >> cgroups. We need the ability to monitor groups of tasks individually >> within that CAT partition. I think this is what this bullet is about. > > I completely understand that. That's fine and I never debated that one, but > the requirements list is too vague about what you want to measure. > >> >>> 5) Put multiple threads into a single measurement group > > That can be: > > A) threads within a CAT group > > B) threads which belong to different CAT groups > > A) is fine. B) does not make any sense to me It's A). As Tony suggested in a previous email, we can rephrase it to: 5) Put a subset of threads from the same CAT group into a single measurement group. > > Same applies for per CPU measurements. For CPU measurements. We need perf-like CPU filtering to support tools that perform low overhead monitoring by polling CPU events. These tools approximate per-cgroup/task events by reconciling CPU events with logs of what job run when in what CPU. > > Thanks, > > tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
Stephane, On Tue, 7 Mar 2017, Stephane Eranian wrote: > On Tue, Mar 7, 2017 at 12:04 PM, Luck, Tonywrote: > >> That's all nice and good, but I still have no coherent explanation why > >> measuring across allocation domains makes sense. > > > > Is this in reaction to this one? > > > >>> 5) Put multiple threads into a single measurement group > > > > If we fix it to say "threads from the same CAT group" does it fix things? > > > Inside a CAT partition, there may be multiple tasks split into different > cgroups. We need the ability to monitor groups of tasks individually > within that CAT partition. I think this is what this bullet is about. I completely understand that. That's fine and I never debated that one, but the requirements list is too vague about what you want to measure. > >>> 5) Put multiple threads into a single measurement group That can be: A) threads within a CAT group B) threads which belong to different CAT groups A) is fine. B) does not make any sense to me Same applies for per CPU measurements. Thanks, tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
Stephane, On Tue, 7 Mar 2017, Stephane Eranian wrote: > On Tue, Mar 7, 2017 at 12:04 PM, Luck, Tony wrote: > >> That's all nice and good, but I still have no coherent explanation why > >> measuring across allocation domains makes sense. > > > > Is this in reaction to this one? > > > >>> 5) Put multiple threads into a single measurement group > > > > If we fix it to say "threads from the same CAT group" does it fix things? > > > Inside a CAT partition, there may be multiple tasks split into different > cgroups. We need the ability to monitor groups of tasks individually > within that CAT partition. I think this is what this bullet is about. I completely understand that. That's fine and I never debated that one, but the requirements list is too vague about what you want to measure. > >>> 5) Put multiple threads into a single measurement group That can be: A) threads within a CAT group B) threads which belong to different CAT groups A) is fine. B) does not make any sense to me Same applies for per CPU measurements. Thanks, tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Tue, 7 Mar 2017, Stephane Eranian wrote: Hi, On Tue, Mar 7, 2017 at 12:04 PM, Luck, Tonywrote: That's all nice and good, but I still have no coherent explanation why measuring across allocation domains makes sense. Is this in reaction to this one? 5) Put multiple threads into a single measurement group If we fix it to say "threads from the same CAT group" does it fix things? Inside a CAT partition, there may be multiple tasks split into different cgroups. We need the ability to monitor groups of tasks individually within that CAT partition. I think this is what this bullet is about. The #8 covers that I think (or what we intended for 5..) ? 8) Can get measurements for subsets of tasks in a CAT group (to find the threads hogging the resources). Thanks, Vikas We'd like to have measurement groups use a single RMID ... if we allowed tasks from different CAT groups in the same measurement group we wouldn't be able to split the numbers back to report the right overall total for each of the CAT groups. -Tony
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Tue, 7 Mar 2017, Stephane Eranian wrote: Hi, On Tue, Mar 7, 2017 at 12:04 PM, Luck, Tony wrote: That's all nice and good, but I still have no coherent explanation why measuring across allocation domains makes sense. Is this in reaction to this one? 5) Put multiple threads into a single measurement group If we fix it to say "threads from the same CAT group" does it fix things? Inside a CAT partition, there may be multiple tasks split into different cgroups. We need the ability to monitor groups of tasks individually within that CAT partition. I think this is what this bullet is about. The #8 covers that I think (or what we intended for 5..) ? 8) Can get measurements for subsets of tasks in a CAT group (to find the threads hogging the resources). Thanks, Vikas We'd like to have measurement groups use a single RMID ... if we allowed tasks from different CAT groups in the same measurement group we wouldn't be able to split the numbers back to report the right overall total for each of the CAT groups. -Tony
Re: [PATCH 1/1] x86/cqm: Cqm requirements
Hi, On Tue, Mar 7, 2017 at 12:04 PM, Luck, Tonywrote: >> That's all nice and good, but I still have no coherent explanation why >> measuring across allocation domains makes sense. > > Is this in reaction to this one? > >>> 5) Put multiple threads into a single measurement group > > If we fix it to say "threads from the same CAT group" does it fix things? > Inside a CAT partition, there may be multiple tasks split into different cgroups. We need the ability to monitor groups of tasks individually within that CAT partition. I think this is what this bullet is about. > We'd like to have measurement groups use a single RMID ... if we > allowed tasks from different CAT groups in the same measurement > group we wouldn't be able to split the numbers back to report the > right overall total for each of the CAT groups. > > -Tony
Re: [PATCH 1/1] x86/cqm: Cqm requirements
Hi, On Tue, Mar 7, 2017 at 12:04 PM, Luck, Tony wrote: >> That's all nice and good, but I still have no coherent explanation why >> measuring across allocation domains makes sense. > > Is this in reaction to this one? > >>> 5) Put multiple threads into a single measurement group > > If we fix it to say "threads from the same CAT group" does it fix things? > Inside a CAT partition, there may be multiple tasks split into different cgroups. We need the ability to monitor groups of tasks individually within that CAT partition. I think this is what this bullet is about. > We'd like to have measurement groups use a single RMID ... if we > allowed tasks from different CAT groups in the same measurement > group we wouldn't be able to split the numbers back to report the > right overall total for each of the CAT groups. > > -Tony
RE: [PATCH 1/1] x86/cqm: Cqm requirements
On Tue, 7 Mar 2017, Luck, Tony wrote: > > That's all nice and good, but I still have no coherent explanation why > > measuring across allocation domains makes sense. > > Is this in reaction to this one? > > >> 5) Put multiple threads into a single measurement group > > If we fix it to say "threads from the same CAT group" does it fix things? > > We'd like to have measurement groups use a single RMID ... if we > allowed tasks from different CAT groups in the same measurement > group we wouldn't be able to split the numbers back to report the > right overall total for each of the CAT groups. Right. And the same applies to CPU measurements. If we have threads from 3 CAT groups running on a CPU then the aggregate value (except for bandwidth which can be computed by software) is useless. Thanks, tglx
RE: [PATCH 1/1] x86/cqm: Cqm requirements
On Tue, 7 Mar 2017, Luck, Tony wrote: > > That's all nice and good, but I still have no coherent explanation why > > measuring across allocation domains makes sense. > > Is this in reaction to this one? > > >> 5) Put multiple threads into a single measurement group > > If we fix it to say "threads from the same CAT group" does it fix things? > > We'd like to have measurement groups use a single RMID ... if we > allowed tasks from different CAT groups in the same measurement > group we wouldn't be able to split the numbers back to report the > right overall total for each of the CAT groups. Right. And the same applies to CPU measurements. If we have threads from 3 CAT groups running on a CPU then the aggregate value (except for bandwidth which can be computed by software) is useless. Thanks, tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Tue, 7 Mar 2017, Vikas Shivappa wrote: > Sending the cqm requirements as per Thomas comments in the previous > verson of cqm patch series - > https://marc.info/?l=linux-kernel=148374167726502 > > This is modified version of requirements sent by Tony in the same > thread with inputs from David and Stephan. > > Reviewed-by: Tony Luck> Reviewed-by: David Carrillo-Cisneros > Reviewed-by: Yu Fenghua > Reviewed-by: Stephane Eranian > Signed-off-by: Vikas Shivappa That's all nice and good, but I still have no coherent explanation why measuring across allocation domains makes sense. Repeating this requirement w/o explanation does not get us anywhere. For the record: I can understand the use case for bandwidth, but that's a pretty trivial act of summing it up in software, so there is no actual requirement to do that with a seperate RMID. For cache oocupancy it still does not make any sense unless there is some magic voodoo I'm not aware of to decipher the meaning of those numbers. Thanks, tglx
Re: [PATCH 1/1] x86/cqm: Cqm requirements
On Tue, 7 Mar 2017, Vikas Shivappa wrote: > Sending the cqm requirements as per Thomas comments in the previous > verson of cqm patch series - > https://marc.info/?l=linux-kernel=148374167726502 > > This is modified version of requirements sent by Tony in the same > thread with inputs from David and Stephan. > > Reviewed-by: Tony Luck > Reviewed-by: David Carrillo-Cisneros > Reviewed-by: Yu Fenghua > Reviewed-by: Stephane Eranian > Signed-off-by: Vikas Shivappa That's all nice and good, but I still have no coherent explanation why measuring across allocation domains makes sense. Repeating this requirement w/o explanation does not get us anywhere. For the record: I can understand the use case for bandwidth, but that's a pretty trivial act of summing it up in software, so there is no actual requirement to do that with a seperate RMID. For cache oocupancy it still does not make any sense unless there is some magic voodoo I'm not aware of to decipher the meaning of those numbers. Thanks, tglx
RE: [PATCH 1/1] x86/cqm: Cqm requirements
> That's all nice and good, but I still have no coherent explanation why > measuring across allocation domains makes sense. Is this in reaction to this one? >> 5) Put multiple threads into a single measurement group If we fix it to say "threads from the same CAT group" does it fix things? We'd like to have measurement groups use a single RMID ... if we allowed tasks from different CAT groups in the same measurement group we wouldn't be able to split the numbers back to report the right overall total for each of the CAT groups. -Tony
RE: [PATCH 1/1] x86/cqm: Cqm requirements
> That's all nice and good, but I still have no coherent explanation why > measuring across allocation domains makes sense. Is this in reaction to this one? >> 5) Put multiple threads into a single measurement group If we fix it to say "threads from the same CAT group" does it fix things? We'd like to have measurement groups use a single RMID ... if we allowed tasks from different CAT groups in the same measurement group we wouldn't be able to split the numbers back to report the right overall total for each of the CAT groups. -Tony